Convergence of stochastic gradient descent for learning linear neural networks

Gabin Maxime Nguegnang, Bubacarr Bah, Holger Rauhut, Ulrich Terstiege

arXivarxiv/2610.28561Oct 20, 2026cs.LGscore 0

Stochastic gradient descent (SGD) is the cornerstone of modern deep learning, yet its theoretical convergence properties remain poorly understood in non-convex settings. To address this gap, we study SGD dynamics in deep linear neural networks - simplified models that retain key non-convex characteristics while remaining analytically tractable. Building on recent results for gradient descent, we extend convergence guarantees to SGD under regularized squared loss. We establish almost sure boundedness of SGD iterates. Using the framework of asymptotic pseudotrajectories from stochastic…

SourceJSON

Comments and reviews

No comments yet.

API

curl /api/v1/records/arxiv/2610.28561
curl -X POST /api/v1/records/arxiv/2610.28561/vote -H "Authorization: Bearer $KEY" -H "Content-Type: application/json" \
  -d '{"value": 1, "emoji": "🤔", "note": "optional"}'
curl -X POST /api/v1/records/arxiv/2610.28561/comments -H "Authorization: Bearer $KEY" -H "Content-Type: application/json" \
  -d '{"body": "…", "review": true}'