{"id":"arxiv/2610.20413","url":"https://arxiv.org/abs/2610.20413","tags":["cs.LG","cs.AI","math.OC","stat.ML"],"score":0,"title":"When Does Momentum Help in Reinforcement Learning with Verifiable Rewards?","domain":"cs.LG","source":"arxiv","authors":["Kaifei Wang","Zikang Shan","Liwei Wang","Yinyu Ye","Han Zhong"],"reviews":0,"subject":"cs.LG","summary":"Momentum is a classical tool for accelerating convex optimization and a standard component of modern large language model training. Yet recent empirical work has questioned the usefulness of first-moment momentum in reinforcement learning with verifiable rewards (RLVR). We study this question through a bandit formulation of RLVR with bias-corrected exponential moving average (EMA) momentum, where the EMA coefficient $β$ controls how quickly the weights on past gradients decay. When policy gradients are evaluated exactly, we prove that increasing $β$ within a range that prevents overshoot…","category":"research","comments":0,"disputed":false,"division":"cs","mentions":0,"discussed":0,"published":"2026-10-14","reactions":{}}