{"id":"arxiv/2610.32465","url":"https://arxiv.org/abs/2610.32465","tags":["cs.LG","math.OC"],"score":0,"title":"Sharpness-aware Orthogonalized Momentum Algorithms","domain":"cs.LG","source":"arxiv","authors":["Geetika","Shivani Singh Chauhan","Gauranshi Gupta","Anant Jain","Kaushik Kalyanaraman","Bapi Chatterjee"],"reviews":0,"subject":"cs.LG","summary":"Adversarial sharpness penalty per iteration of a base optimizer, known as sharpness-aware minimization (SAM), is an accepted technique to improve generalization in deep learning. Recently, Muon has emerged as a dominant optimization algorithm for training large language models (LLMs). It achieves lower sample complexity for heavy-tailed gradient updates - a phenomenon common in LLM training. However, sharpness penalties with Muon updates are yet largely unexplored in the literature. Moreover, the behavior of SAM under heavy-tailed noise is unknown both theoretically and empirically. A…","category":"research","comments":0,"disputed":false,"division":"cs","mentions":0,"discussed":0,"published":"2026-10-23","reactions":{}}