Naomi Saphra
Naomi Saphra

Research Fellow

About Me

I am a research fellow at the Kempner Institute at Harvard University. I am interested in NLP training dynamics: how models learn to encode linguistic patterns or other structure and how we can encode useful inductive biases into the training process. Recently, I have begun collaborating with natural and social scientists to use interpretability to understand the world around us. I have become particularly interested in fish. Previously, I earned a PhD from the University of Edinburgh on Training Dynamics of Neural Language Models; worked at NYU, Google and Facebook; and attended Johns Hopkins and Carnegie Mellon University. Outside of research, I play roller derby under the name Gaussian Retribution, perform standup comedy, and shepherd disabled programmers into the world of code dictation.

Download CV
Interests
  • Language modeling
  • Interpretability
  • Training Dynamics
  • Generalization
  • AI for Scientific Understanding
Education
  • PhD in Informatics

    University of Edinburgh

  • MEng in Computer Science

    Johns Hopkins University

  • BSc Artificial Intelligence

    Carnegie Mellon University

My Research

My core agenda focuses on a single goal: to completely and comprehensively understand language model training. This objective combines linguistics, optimization, learning dynamics, science of deep learning, interpretability, and behavioral analysis. Recently, I have begun using similar approaches to study scientific discovery models and enhance broader scientific understanding.

My current publication list is available on my Google Scholar.

Recent Posts

Against Monodomainism

A petty rant on the exceptional treatment of computer vision applications, directed at the machine learning community.

Featured Publications
Recent Publications
(2025). How to visualize training dynamics in neural networks. Blog Post Track at International Conference on Learning Representations (ICLR BlogPosts).
(2025). PolyPythias: Stability and Outliers across Fifty Language Model Pre-Training Runs. International Conference on Learning Representations (ICLR).
(2025). Recite, Reconstruct, Recollect: Memorization in LMs as a Multifaceted Phenomenon. International Conference on Learning Representations (ICLR).
(2025). Distributional Scaling Laws for Emergent Capabilities.