PolyPythias: Stability and Outliers across Fifty Language Model Pre-Training Runs

May 1, 2025·

Oskar Van Der Wal

,

Pietro Lesci

,

Max Müller-Eberstein

Naomi Saphra

Naomi Saphra

,

Hailey Schoelkopf

,

Willem Zuidema

,

Stella Biderman

· 0 min read

Type

Conference paper

Publication

International Conference on Learning Representations (ICLR)

Last updated on May 1, 2025

Naomi Saphra

Authors

Research Fellow

← How to visualize training dynamics in neural networks May 1, 2025

Recite, Reconstruct, Recollect: Memorization in LMs as a Multifaceted Phenomenon May 1, 2025 →