{"id":"arxiv/2605.20996","url":"https://arxiv.org/abs/2605.20996","tags":["cs.LG","math.OC"],"score":0,"title":"Beyond the Bellman Recursion: A Pontryagin-Guided Framework for Non-Exponential Discounting","domain":"cs.LG","source":"arxiv","authors":["Hojin Ko","Jeonggyu Huh"],"reviews":0,"subject":"cs.LG","summary":"Most value-based and actor--critic reinforcement learning methods rely on Bellman-style recursions, yet these recursions collapse under non-exponential discounting common in human preferences and survival processes. We show the breakdown is structural: exponential discounting sits at a fragile intersection of multiplicativity and time homogeneity, and violating either property breaks standard dynamic programming. To overcome this, we propose Pontryagin-Guided Direct Policy Optimization (PG-DPO), a variational framework that abandons recursion and couples the Pontryagin Maximum Principle with…","category":"research","comments":0,"disputed":false,"division":"cs","mentions":0,"discussed":0,"published":"2026-05-20","reactions":{}}