{"id":"arxiv/2609.39837","url":"https://arxiv.org/abs/2609.39837","tags":["cs.LG","math.OC"],"score":0,"title":"Fast Regularized Policy Mirror Descent with One-Step TD Updates","domain":"cs.LG","source":"arxiv","authors":["Qipei Chen","Wenye Li","Yule Sun","Ke Wei"],"reviews":0,"subject":"cs.LG","summary":"Policy mirror descent (PMD) enjoys fast convergence in regularized Markov decision processes (MDPs), but existing guarantees often rely on exact or increasingly accurate policy evaluation. We analyze PMD coupled with a persistent critic advanced by one temporal-difference (TD) update. For finite discounted MDPs, we establish global linear convergence in value for exact coordinate-wise Bellman updates, with any positive constant actor stepsize and arbitrary finite critic initialization. The proof combines a resolvent-based auxiliary distribution with a decaying Bellman-violation correction…","category":"research","comments":0,"disputed":false,"division":"cs","mentions":0,"discussed":0,"published":"2026-09-30","reactions":{}}