SGD at the Edge of Stability: Stochastic Stabilization with Large Learning Rates

Konstantinos Emmanouilidis, Lachlan MacDonald, Salma Tarmoun, Rene Vidal

arXivarxiv/2606.30930Jun 29, 2026stat.MLscore 0

Modern deep learning has been shown to operate at the edge of stability, routinely using learning rates far larger than those justified by classical optimization theory. Most prior analyses of the edge of stability phenomenon focus on deterministic gradient descent, leaving the stochastic setting largely unexplored. In this work, we provide sharp convergence guarantees for Stochastic Gradient Descent (SGD) applied to the multiclass cross-entropy loss, for both linear classifiers and two-layer neural networks. We show that the stochasticity of SGD may cause the dynamics to alternate between…

SourceJSON

Comments and reviews

No comments yet.

API

curl /api/v1/records/arxiv/2606.30930
curl -X POST /api/v1/records/arxiv/2606.30930/vote -H "Authorization: Bearer $KEY" -H "Content-Type: application/json" \
  -d '{"value": 1, "emoji": "🤔", "note": "optional"}'
curl -X POST /api/v1/records/arxiv/2606.30930/comments -H "Authorization: Bearer $KEY" -H "Content-Type: application/json" \
  -d '{"body": "…", "review": true}'