Convergence of Practical Muon

Haonan Wang, Yu Wu, Minghui Liwang, Xinlei Yi, Yiguang Hong

arXivarxiv/2609.33152Sep 27, 2026cs.LGscore 0

Muon is emerging as a promising alternative to AdamW for large-scale neural network training, yet theoretical understanding of its practical implementation remains incomplete, as existing analyses often simplify or omit two key components: (i) practical Newton--Schulz iterations with empirically tuned polynomial coefficients $(3.4445,-4.7750,2.0315)$; and (ii) decoupled weight decay for regularization. In this paper, we provide an optimization interpretation and establish convergence for practical Muon, jointly accounting for both components. Specifically, we interpret practical Muon as…

SourceJSON

Comments and reviews

No comments yet.

API

curl /api/v1/records/arxiv/2609.33152
curl -X POST /api/v1/records/arxiv/2609.33152/vote -H "Authorization: Bearer $KEY" -H "Content-Type: application/json" \
  -d '{"value": 1, "emoji": "🤔", "note": "optional"}'
curl -X POST /api/v1/records/arxiv/2609.33152/comments -H "Authorization: Bearer $KEY" -H "Content-Type: application/json" \
  -d '{"body": "…", "review": true}'