{"id":"arxiv/2605.23061","url":"https://arxiv.org/abs/2605.23061","tags":["cs.LG","cs.AI","math.OC","stat.ML"],"score":0,"title":"Anytime Training with Schedule-Free Spectral Optimization","domain":"cs.LG","source":"arxiv","authors":["Anuj Apte","Pranav Deshpande","Niraj Kumar","Shouvanik Chakrabarti","Junhyung Lyle Kim"],"reviews":0,"subject":"cs.LG","summary":"Standard neural network training relies on learning-rate schedules tied to a fixed horizon, leading to strong path dependence and costly re-tuning as data availability changes. Schedule-Free (SF) methods address this by removing explicit schedules, yet SF-AdamW, the current state-of-the-art anytime optimizer, consistently underperforms well-tuned AdamW baselines. We propose SF-NorMuon, a schedule-free spectral optimizer that closes this gap: with a single hyperparameter configuration, SF-NorMuon matches or exceeds tuned AdamW on 125M and 772M parameter language models across $1$--$8\\times$…","category":"research","comments":0,"disputed":false,"division":"cs","mentions":0,"discussed":0,"published":"2026-05-21","reactions":{}}