{"id":"arxiv/2610.14988","url":"https://arxiv.org/abs/2610.14988","tags":["cs.LG","cs.AI","math.OC","stat.ML"],"score":0,"title":"Mixing Regularization for Average-Reward $Q$-Learning and $Q$-Planning","domain":"cs.LG","source":"arxiv","authors":["Julien Roger","Sam Bodmer","Jian Qian","Benjamin Riviere"],"reviews":0,"subject":"cs.LG","summary":"The average-reward setting for reinforcement learning is challenging because its Bellman operator is known to contract in span only when the underlying Markov chain is mixing. Whereas existing work typically assumes a contraction property, e.g., through environment mixing, we consider the setting where we are given a reward-agnostic $m$-step mixing policy, and we control the contraction and exploration properties of the learned policy by regularizing toward the mixing policy. We provide finite-time analysis of $Q$-learning algorithms that can be instantiated with multiple choices of…","category":"research","comments":0,"disputed":false,"division":"cs","mentions":0,"discussed":0,"published":"2026-10-10","reactions":{}}