{"id":"arxiv/2609.02728","url":"https://arxiv.org/abs/2609.02728","tags":["stat.ML","cs.LG","math.OC"],"score":0,"title":"Momentum in large-batch training: Polyak enlarges the critical batch size, Nesterov improves data efficiency","domain":"stat.ML","source":"arxiv","authors":["Jia-Nan Wang","Zixun Huang","Kairui Li","Lei Wu"],"reviews":0,"subject":"stat.ML","summary":"We study when and how momentum improves large-batch training in the one-pass regime, using power-law kernel regression as a tractable setting. We first characterize risk stability through the critical learning rate, defined as the largest learning rate for stable training, and obtain $η_{\\mathrm{SGD}}^{\\mathrm{crit}}\\eqsim 1$, $η_{\\mathrm{Polyak}}^{\\mathrm{crit}}\\eqsim \\min\\{1,B(1-ρ)\\}$, and $η_{\\mathrm{Nesterov}}^{\\mathrm{crit}}\\eqsim \\min\\{1,B^β(1-ρ)\\}$, where $B$ is the batch size, $ρ$ is the momentum factor, and $β>1$ is the capacity exponent. Within this admissible region, we derive…","category":"research","comments":0,"disputed":false,"division":"stat","mentions":0,"discussed":0,"published":"2026-09-02","reactions":{}}