Evaluating L2 Learners’ Experience with GenAI-Powered Academic Reading Tool in Higher Education: A Small-Scale Exploratory Study

Generative artificial intelligence (GenAI)-powered tools have been widely used to enhance language learning skills. However, little is known about their role in shaping learners’ experiences during academic reading. To address this gap, our study engaged twelve postgraduate students in evaluating reading reports generated by Fullpicture, an academic reading tool powered by ChatGPT, over seven weeks. Students reflected on their experiences using the GenAI tool for academic reading and rated its effectiveness on a 6-point Likert scale. Based on the three-level model of reading comprehension, learners’ reported benefits of the tool include the literal level, which encompasses the facilitation of quick comprehension and the reduction of cognitive load; the inferential level, which includes the interpretation of the author’s tone and purpose and the reinforcement of personal beliefs; and the critical level, which involves the promotion of reflective reading practices and the generation of new perspectives for academic writing. The perceived challenges can also be categorized into three levels: selective summarization and misrepresentation of details at the literal level; overgeneralization of interpretations and irrelevance to personal reading and writing goals at the inferential level; and a lack of practical suggestions and weak evidence to support critiques at the critical level. The analysis of questionnaire results also revealed the supporting role of GenAI-powered tools for L2 learners in evaluating arguments, evidence quality, or the generalizability of research; however, their limited effectiveness was also identified, especially concerning the adequacy of discussion and theoretical frameworks. The study suggested that learners should collaborate with AI instead of fully relying on it to support academic reading.