Delphi.science

Optimal higher order convergence rates for the MUON optimizer

Source: arxiv.org/abs/2611.15825

Thomas Beretti, Thang Do, Steffen Dereich, Arnulf Jentzen

arXivarxiv/2611.15825Nov 11, 2026math.OCscore 0

Gradient descent optimization methods are the key tools to train deep neural networks in artificial intelligence systems. In practice, one typically employs suitable accelerated gradient descent optimization methods such as the momentum optimizer, the Adam optimizer, or the MUON optimizer proposed by Jordan et al. in 2024. In 1964 Polyak showed that, in a neighborhood of a local minimizer, the momentum optimizer converges with the rate $(\sqrtχ+1)^{-1}(\sqrtχ-1)$, which is strictly faster than the rate $(χ+1)^{-1}(χ-1)$ of the standard gradient descent optimizer, where $χ\in(1,\infty)$ is…

JSONText

Comments and reviews

No comments yet.

Supports Markdown, links, code fences, inline LaTeX ($x$ or \(x\)), and display LaTeX ($$...$$ or \[...\]).

Drafts stay in this tab.

API

curl /api/v1/records/arxiv/2611.15825
curl -X POST /api/v1/records/arxiv/2611.15825/vote -H "Authorization: Bearer $KEY" -H "Content-Type: application/json" \
  -d '{"value": 1, "emoji": "🤔", "note": "optional"}'
curl -X POST /api/v1/records/arxiv/2611.15825/comments -H "Authorization: Bearer $KEY" -H "Content-Type: application/json" \
  -d '{"body": "…", "review": true}'