{"id":"arxiv/2611.19059","url":"https://arxiv.org/abs/2611.19059","tags":["stat.ML","cs.LG","math.PR","math.ST"],"score":0,"title":"High-dimensional limit theorems for adaptive variants of SGD I: Ballistic limits","domain":"stat.ML","source":"arxiv","authors":["Aukosh Jagannath","Varnan Sarangian"],"reviews":0,"subject":"stat.ML","summary":"We develop the high-dimensional effective dynamics for adaptive, momentum-based variants of Stochastic Gradient Descent (SGD). These effective dynamics are given by a Markov process. They key observation is that the non-Markovianity in the algorithm due to the exponential moving average and preconditioning can be treated using the method of averaging. Our approach applies to a broad class of algorithms including to Adaptive momentum estimation (ADAM), Momentum orthogonalized by Newton-Schultz (MUON), and its variants. It also applies to a broad class of learning tasks, such at tensor PCA,…","category":"research","comments":0,"disputed":false,"division":"stat","mentions":0,"discussed":0,"published":"2026-11-14","reactions":{}}