{"id":"arxiv/2601.12238","url":"https://arxiv.org/abs/2601.12238","tags":["stat.ML","cs.LG","math.OC"],"score":0,"title":"On the Provable Suboptimality of Momentum SGD in Nonstationary Stochastic Optimization","domain":"stat.ML","source":"arxiv","authors":["Sharan Sahu","Cameron J. Hogan","Martin T. Wells"],"reviews":0,"subject":"stat.ML","summary":"In this paper, we provide a comprehensive theoretical analysis of Stochastic Gradient Descent (SGD) and its momentum variants (Polyak Heavy-Ball and Nesterov) for tracking time-varying optima under strong convexity and smoothness. Our finite-time bounds reveal a sharp decomposition of tracking error into transient, noise-induced, and drift-induced components. This decomposition exposes a fundamental trade-off: while momentum is often used as a gradient-smoothing heuristic, under distribution shift it incurs an explicit drift-amplification penalty that diverges as the momentum parameter $β$…","category":"research","comments":0,"disputed":false,"division":"stat","mentions":0,"discussed":0,"published":"2026-01-18","reactions":{}}