The Parable of the Prinia's Egg: An Allegory for AI Science
I discuss what counts as strong evidence for an explanation of model behavior.
I am a research fellow at the Kempner Institute at Harvard University. I am interested in NLP training dynamics: how models learn to encode linguistic patterns or other structure and how we can encode useful inductive biases into the training process. Recently, I have begun collaborating with natural and social scientists to use interpretability to understand the world around us. I have become particularly interested in fish. Previously, I earned a PhD from the University of Edinburgh on Training Dynamics of Neural Language Models; worked at NYU, Google and Facebook; and attended Johns Hopkins and Carnegie Mellon University. Outside of research, I play roller derby under the name Gaussian Retribution, perform standup comedy, and shepherd disabled programmers into the world of code dictation.
PhD in Informatics
University of Edinburgh
MEng in Computer Science
Johns Hopkins University
BSc Artificial Intelligence
Carnegie Mellon University
My core agenda focuses on a single goal: to completely and comprehensively understand language model training. This objective combines linguistics, optimization, learning dynamics, science of deep learning, interpretability, and behavioral analysis. Recently, I have begun using similar approaches to study scientific discovery models and enhance broader scientific understanding.
My current publication list is available on my Google Scholar.
I discuss what counts as strong evidence for an explanation of model behavior.
Nothing in Deep Learning Makes Sense Except in the Light of SGD.
A petty rant on the exceptional treatment of computer vision applications, directed at the machine learning community.
In August of 2015, my hands stopped working. This is what happened next.