{"id":"arxiv/2605.28961","url":"https://arxiv.org/abs/2605.28961","tags":["stat.ML","cs.LG","math.OC"],"score":0,"title":"Dynamics of Stochastic Momentum with Sparse Updates in High Dimensions","domain":"stat.ML","source":"arxiv","authors":["Katie Everett","Elliot Paquette"],"reviews":0,"subject":"stat.ML","summary":"Existing theory of momentum assumes that gradients arrive at every parameter at a roughly constant rate, an assumption violated in practice by heavy-tailed data distributions and modern architectures. We theoretically analyze the dynamics of two tractable models of momentum under sparse updates: a least squares model with sparse inputs and a logistic regression model with a rare class. Both admit exact closed-form second-moment dynamics whose high-dimensional limits we characterize across three scaling exponents for sparsity, batch size, and momentum decay. The phase structure on both…","category":"research","comments":0,"disputed":false,"division":"stat","mentions":0,"discussed":0,"published":"2026-05-27","reactions":{}}