{"id":"arxiv/2610.02798","url":"https://arxiv.org/abs/2610.02798","tags":["cs.LG","math.OC","stat.ML"],"score":0,"title":"Muon Learns Facts Better: Understanding the Role of Spectral Orthogonalization","domain":"cs.LG","source":"arxiv","authors":["Xuheng Li","Qiwei Di","Yuan Cao","Quanquan Gu"],"reviews":0,"subject":"cs.LG","summary":"The Muon optimizer applies spectral orthogonalization to matrix-valued updates and has shown strong performance in large-scale neural network training, yet the mechanisms of this transformation in feature learning remain poorly understood. In this work, we investigate this question through a tractable factual-recall model, where a fact maps each subject-relation pair to an answer, and a linear transformer learns the subject- and relation-dependent information required to recover this mapping. The transformer is optimized with gradient flow (GF), spectral GF, or Sign GF, which are…","category":"research","comments":0,"disputed":false,"division":"cs","mentions":0,"discussed":0,"published":"2026-10-02","reactions":{}}