[Thesis Defense] Yung-Sung Chuang: "Towards Factual and Trustworthy Large Language Models"

Thesis Advisor: James Glass

Thesis Committee: Yoon Kim, Jacob Andreas

Calendar Invitation: http://people.csail.mit.edu/yungsung/defense.ics

Speaker's Website: https://yung-sung.github.io

Abstract: 

Large Language Models (LLMs) have transformed how we interact with information, yet hallucinations, e.g., plausible but factually incorrect outputs, remain a critical barrier to their deployment in high-stakes applications. This thesis presents a comprehensive approach to understanding and mitigating hallucinations across several fundamental dimensions of knowledge in AI systems: parametric, contextual, and attribution knowledge.

We identify that hallucinations arise from different failure modes requiring distinct solutions. First, models may fail to leverage parametric knowledge already encoded in their weights. We introduce DoLa (Decoding by Contrasting Layers), which amplifies factual knowledge by dynamically contrasting predictions across transformer layers, improving factuality without training or external knowledge. Second, in retrieval-augmented generation settings, models often fail to properly use provided context. We develop Lookback Lens, which analyzes attention patterns to detect and reduce hallucinations. Third, even when models generate correct content, users need verifiable evidence. We present SelfCite, a self-supervised alignment method that enables LLMs to provide accurate sentence-level citations through a reward design of context ablation. Together, these methods form a roadmap towards better AI systems, working towards systems that are not only capable but also reliable, transparent, and trustworthy.