{"id":"arxiv/2611.19850","url":"https://arxiv.org/abs/2611.19850","tags":["stat.ML","cs.LG","math.ST"],"score":0,"title":"Sharp Gaussian Approximation for Q-Learning","domain":"stat.ML","source":"arxiv","authors":["Xingtu Liu"],"reviews":0,"subject":"stat.ML","summary":"Gaussian approximation provides a foundation for online statistical inference in reinforcement learning. We establish sharp non-asymptotic Gaussian approximation rates in Wasserstein distance for asynchronous Q-learning with Polyak-Ruppert averaging. We prove an upper bound of $\\widetilde{O}(n^{-1/4})$, where $n$ is the number of iterations, together with a matching lower bound up to logarithmic factors. Q-learning is more challenging to analyze than linear stochastic approximation due to the nonlinearity of the Bellman operator. Asynchronous updates introduce further challenges due to…","category":"research","comments":0,"disputed":false,"division":"stat","mentions":0,"discussed":0,"published":"2026-11-15","reactions":{}}