Artificial Intelligence and Foreign Language Learning: ChatGPT’s Limitations in Summarizing Texts

—With the emergence of the widely accessible ChatGPT, teaching and assessment now face new challenges, particularly regarding the use of Artificial Intelligence (AI) tools. This paper presents a teaching experiment on the preparation of summaries of the short story “O tradutor ideal” [“The ideal translator”], by São Toméan writer Olinda Beja, based on answers provided by ChatGPT-4 and summaries produced by Chinese students in a Portuguese as a Foreign Language degree class. The methodology combines a qualitative analysis of student texts to identify AI’s influence with a longitudinal study of ChatGPT’s responses over a one-year period, aimed at observing the chatbot’s evolution. The study evaluates the potential and limitations of AI in summarizing lesser-known literary texts that are difficult to access online. Results highlight the importance of meeting challenges posed by new AI technologies without violating ethical principles. In the case of emerging literature in Portuguese, ChatGPT still shows several limitations, producing summaries that lack fidelity to the original text and literary and cultural depth. However, there has been an evolution in the responses of these Large Language Models (LLM) from April 2024 to April 2025, the period in which the study was carried out. The research highlights the need for critical reflection on the use of AI in educational and academic contexts, particularly in the teaching of literature, reinforcing the importance of adopting an ethical approach. It also emphasizes the need to closely examine the use of AI, particularly in research on Portuguese-speaking African literatures and summary writing.