Can AI Understand Literature? Columbia Researchers Put It to the Test
Despite significant advances in large language models (LLMs) such as ChatGPT—particularly in assisting with thinking, research, summarization, and the comprehension of complex technical texts—their ability to interpret storytelling and literature remains an open question. Issues related to interpretive nuance and depth continue to persist. Researchers at Columbia Engineering have addressed these challenges through a novel and ethically grounded evaluation framework. Their work received the Best Paper Award (2025) at Transactions of the Association for Computational Linguistics (TACL) , highlighting its methodological rigor and contribution to the field. “Before we can place real trust in LLMs’ analytical abilities, we need careful evidence of what they can and cannot do,” stated Kathleen McKeown, the Henry and Gertrude Rothschild Professor of Computer Science at Columbia Engineering. The research was led by Professor McKeown and Associate Professor Lydia Chilton. “If LLMs are to ...