{"id":"arxiv/2602.11679","url":"https://arxiv.org/abs/2602.11679","tags":["stat.ML","cs.AI","cs.LG","math.OC","stat.ME"],"score":0,"title":"Provable Offline Reinforcement Learning for Structured Cyclic MDPs","domain":"stat.ML","source":"arxiv","authors":["Kyungbok Lee","Angelica Cristello Sarteau","Michael R. Kosorok"],"reviews":0,"subject":"stat.ML","summary":"We introduce a novel cyclic Markov decision process (MDP) framework for multi-step decision problems with heterogeneous stage-specific dynamics, transitions, and discount factors across the cycle. In this setting, offline learning is challenging: optimizing a policy at any stage shifts the state distributions of subsequent stages, propagating mismatch across the cycle. To address this, we propose a modular structural framework that decomposes the cyclic process into stage-wise sub-problems. While generally applicable, we instantiate this principle as CycleFQI, an extension of fitted…","category":"research","comments":0,"disputed":false,"division":"stat","mentions":0,"discussed":0,"published":"2026-02-12","reactions":{}}