{"id":"arxiv/2604.21016","url":"https://arxiv.org/abs/2604.21016","tags":["cs.LG","cs.AI","math.OC"],"score":0,"title":"SGD at the Edge of Stability: The Stochastic Sharpness Gap","domain":"cs.LG","source":"arxiv","authors":["Fangshuo Liao","Afroditi Kolomvaki","Anastasios Kyrillidis"],"reviews":0,"subject":"cs.LG","summary":"When training neural networks with full-batch gradient descent (GD) and step size $η$, the largest eigenvalue of the Hessian -- the sharpness $S(\\boldsymbolθ)$ -- rises to $2/η$ and hovers there, a phenomenon termed the Edge of Stability (EoS). \\citet{damian2023selfstab} showed that this behavior is explained by a self-stabilization mechanism driven by third-order structure of the loss, and that GD implicitly follows projected gradient descent (PGD) on the constraint $ S(\\boldsymbolθ)\\leq 2/η$. For mini-batch stochastic gradient descent (SGD), the sharpness stabilizes below $2/η$, with the…","category":"research","comments":0,"disputed":false,"division":"cs","mentions":0,"discussed":0,"published":"2026-04-22","reactions":{}}