On 12 November 2025, the University of Cambridge’s Language Technology Lab hosted a seminar featuring Professor Sarah Wiegreffe of the University of Maryland. In her talk, “Interpretability as the Inverse Machine Learning Pipeline,” Professor Wiegreffe examined how interpretability methods can be mapped onto each stage of the standard machine learning workflow—data collection, model development, and evaluation—to yield deeper causal insights into language model behavior and guide more effective interventions.

Framing Interpretability within the Machine Learning Pipeline

Professor Wiegreffe began by reminding the audience that as natural language processing technologies are deployed in high-stakes domains such as education and healthcare, a surface-level benchmarking approach is no longer sufficient. Instead, she argued, interpretability must provide faithful, causal explanations of how models arrive at particular outputs. To organize this effort, she proposed “inverting” the usual pipeline: start from an observed model behavior that requires explanation, then trace backward through model internals and training dynamics, and ultimately to the data that shaped those behaviors.

Defining Mechanistic Interpretability

Delving into terminology, she reviewed her joint meta-analysis on the rise of “mechanistic interpretability” around 2020–21. While the term had been used broadly to signify any work examining model internals, Professor Wiegreffe and her coauthor advocated for a narrower, causal definition: methods that validate their explanations through targeted interventions on model components. She contrasted “open-box” analyses of attention weights or neurons with more rigorous approaches that demonstrate how manipulating a subnetwork produces quantifiable changes in model outputs.

Case Study: Understanding Multiple-Choice Robustness

To illustrate the inverted pipeline in practice, Professor Wiegreffe described a collaboration with the AI2 team on multiple-choice tasks. Noting that benchmarks often conflate format-following with domain knowledge, her group designed a synthetic “copying colors” task—e.g., “A banana is yellow; what color is a banana?”—to isolate format competence. They showed that some models fail even this simple mapping, revealing that poor benchmark performance may stem more from format brittleness than lack of underlying knowledge. Tracking this behavior across checkpoints, they demonstrated how an early, in-loop evaluation on the synthetic task can serve as a canary for when to invest in more expensive downstream benchmarks.

Relating Internals to Pre-Training Data

Shifting focus to large-scale models, Professor Wiegreffe highlighted work on linear relation embeddings, where a simple learned matrix can approximate a model’s prediction of facts (e.g., “capital of France → Paris”). Her team correlated the strength of these linear features with co-occurrence frequencies in the pre-training corpus. By reconstructing training batches and counting term pairs at each checkpoint, they found that once country–capital pairs exceed a frequency threshold, models reliably form the corresponding linear circuits—underscoring how data distribution shapes internal mechanisms.

Towards Actionable Interpretability

In closing, Professor Wiegreffe emphasized that the ultimate goal of interpretability is to enable targeted improvements. Whether by curating pre-training data, refining architectures during training, or steering models at inference time, causal insights can guide precise interventions. She noted that while fine-tuning remains a strong baseline for behavior correction, a deeper mechanistic understanding can suggest more efficient or robust alternatives.

The seminar concluded with a lively Q&A on the practical trade-offs between benchmark design, fine-tuning baselines, and novel interpretability methods—underscoring the growing importance of causal, faithful explanations in advancing reliable AI.

The Language Technology Lab at the University of Cambridge drives cutting-edge research in natural language processing and speech technologies. By integrating computational linguistics, machine learning, and human-computer interaction, the Lab advances tools for translation, information extraction, and dialogue systems, empowering applications across education, healthcare, and digital accessibility.

The Conf is a platform that reports on scholarly conferences, symposia, roundtables, book talks, and other academic events. It is managed by a group of students from leading American and European universities and is published by Alma Mater Europaea University, Location Vienna.