{"id":"arxiv/2610.11475","url":"https://arxiv.org/abs/2610.11475","tags":["cs.LG","math.OC","stat.ML"],"score":0,"title":"Rare Gate Disagreements Can Limit Plasticity: When Gradient Flow Mispredicts Finite-Batch SGD","domain":"cs.LG","source":"arxiv","authors":["Ruoyu Zhao","Mingxuan Zhang","Jianbo Dai","Jiaqi Wu","Chenyu Zhu","Tong Che"],"reviews":0,"subject":"cs.LG","summary":"Population gradient flow is a common tool for reasoning about how neural networks adapt, including after pretraining. We show that it can mispredict finite-batch stochastic gradient descent (SGD) qualitatively, and we trace the discrepancy to a specific mechanism. In a two-unit ReLU regression, a source task drives the two neurons toward positive proportionality and a target task rewards separating them. After source training for time $T$, gradient flow recovers on the target in time linear in $T$. Online SGD with batch size $b$ and step size $η$ in both phases instead fails with high…","category":"research","comments":0,"disputed":false,"division":"cs","mentions":0,"discussed":0,"published":"2026-10-08","reactions":{}}