{"id":"arxiv/2605.12410","url":"https://arxiv.org/abs/2605.12410","tags":["stat.ML","cs.LG","math.OC","math.ST"],"score":0,"title":"Model-based Bootstrap of Controlled Markov Chains","domain":"stat.ML","source":"arxiv","authors":["Ziwei Su","Imon Banerjee","Diego Klabjan"],"reviews":0,"subject":"stat.ML","summary":"We propose and analyze a model-based bootstrap for transition kernels in finite controlled Markov chains (CMCs) with possibly nonstationary or history-dependent control policies, a setting that arises naturally in offline reinforcement learning (RL) when the behavior policy generating the data is unknown. We establish distributional consistency of the bootstrap transition estimator in both a single long-chain regime and the episodic offline RL regime. The key technical tools are a novel bootstrap law of large numbers (LLN) for the visitation counts and a novel use of the martingale central…","category":"research","comments":0,"disputed":false,"division":"stat","mentions":0,"discussed":0,"published":"2026-05-12","reactions":{}}