{"id":"arxiv/2610.37647","url":"https://arxiv.org/abs/2610.37647","tags":["cs.LG","cs.AI","math.OC"],"score":0,"title":"MuonGS: Pretraining LLMs with Relaxed Weight-Norm Control","domain":"cs.LG","source":"arxiv","authors":["Athanasios Glentis","Dawei Li","Mingyi Hong"],"reviews":0,"subject":"cs.LG","summary":"Weight decay and explicit weight-norm constraints improve optimizer performance in language model pretraining, yet a unified understanding of why such control helps and how much is needed remains incomplete. We analyze norm control in Muon over the entire training trajectory using the \\emph{accumulated effective learning rate}, $C_T=\\sum_{t<T}η_t/\\|W_t\\|_F$, where $η_t$ is the nominal learning rate and $η_t/\\|W_t\\|_F$ is the effective learning rate relative to the weight norm. For bounded update directions, accumulation over any training interval bounds relative weight displacement. We show…","category":"research","comments":0,"disputed":false,"division":"cs","mentions":0,"discussed":0,"published":"2026-10-27","reactions":{}}