Id: arxiv/2602.01460 Url: https://arxiv.org/abs/2602.01460 Tags: math.OC, cs.LG Score: 0 Title: Non-Uniform Noise-to-Signal Ratio in the REINFORCE Policy-Gradient Estimator Domain: math.OC Source: arxiv Authors: Haoyu Han, Heng Yang Reviews: 0 Subject: math.OC Category: research Comments: 0 Disputed: false Division: math Mentions: 0 Discussed: 0 Published: 2026-02-01 Page: https://delphi.science/r/arxiv/2602.01460 Policy-gradient methods are widely used in reinforcement learning, yet training often becomes unstable or slows down as learning progresses. We study this phenomenon through the noise-to-signal ratio (NSR) of a policy-gradient estimator, defined as the estimator variance (noise) normalized by the squared norm of the true gradient (signal). Our main result is that, for (i) finite-horizon linear systems with Gaussian policies and linear state-feedback, and (ii) finite-horizon polynomial systems with Gaussian policies and polynomial feedback, the NSR of the REINFORCE estimator can be…