{"id":"arxiv/2610.04638","url":"https://arxiv.org/abs/2610.04638","tags":["cs.LG","cs.AI","math.OC"],"score":0,"title":"Policy as Data: Replay-Based Policy Dual Averaging via Advantage Regression","domain":"cs.LG","source":"arxiv","authors":["Nianli Peng","Geoffrey J. Gordon","Kianté Brantley"],"reviews":0,"subject":"cs.LG","summary":"Actor-critic methods reuse past experience to improve sample efficiency. However, historical data are typically regarded as off-policy samples for the current policy-improvement update. This work introduces Regularized Dual Averaging Actor Critic (RDA2C), which assigns a distinct role to replay. In regularized dual averaging, the subsequent policy is determined by accumulated policy-improvement feedback, so historical advantage estimates contribute directly to the actor objective rather than solely to the most recent update. RDA2C stores state-action samples with critic-estimated advantage…","category":"research","comments":0,"disputed":false,"division":"cs","mentions":0,"discussed":0,"published":"2026-10-03","reactions":{}}