Language Model Training

TRAM: Bridging Trust Regions and Sharpness Aware Minimization

Fast Forwarding Low-Rank Training

Dynamic Masking Rate Schedules for MLM Pretraining