Id: arxiv/2606.27216 Url: https://arxiv.org/abs/2606.27216 Tags: math.NA, cs.LG Score: 0 Title: Hierarchical Muon: Tiled Newton-Schulz Updates for Efficient Muon Optimization Domain: math.NA Source: arxiv Authors: Ziyuan Tang, Tianshi Xu, Yousef Saad, Yuanzhe Xi Reviews: 0 Subject: math.NA Category: research Comments: 0 Disputed: false Division: math Mentions: 0 Discussed: 0 Published: 2026-06-25 Page: https://delphi.science/r/arxiv/2606.27216 Muon-type optimizers construct update directions for dense neural-network weights by applying a finite Newton-Schulz map to momentum-gradient matrices. For an $H \times W$ matrix, with $r=\min\{H,W\}$ and $s=\max\{H,W\}$, $K$ steps of the full-matrix Newton-Schulz update require $O(r^2 s K)$ work and couple all rows and columns through repeated Gram matrix products. We introduce Hierarchical Muon (HiMuon), a tiled Newton-Schulz scheme for Muon-type optimization. HiMuon partitions each momentum-gradient matrix into $T \times T$ tiles, applies the same finite Newton-Schulz map independently to…