How to visualize training dynamics in neural networks
May 1, 2025
Sometimes I am a Tree: Data drives fragile hierarchical generalization
Feb 1, 2025
Sudden Drops in the Loss: Syntax Acquisition, Phase Transitions, and Simplicity Bias in MLMs
Jan 1, 2024
Loss in the Crowd: Hidden Breakthroughs in Language Model Training
Jan 1, 2024
Causation Does Not Imply Correlation: A Study of Circuit Mechanisms and Model Behaviors
Jan 1, 2024
The Parable of the Prinia's Egg: An Allegory for AI Science
I discuss what counts as strong evidence for an explanation of model behavior.
Sep 17, 2023
Linear Connectivity Reveals Generalization Strategies
Jan 1, 2023
Interpretability Creationism
Jan 1, 2023
Delays, Detours, and Forks in the Road: Latent State Models of Training Dynamics
Jan 1, 2023
Interpretability Creationism
Nothing in Deep Learning Makes Sense Except in the Light of SGD.
Jun 7, 2022