DREAM: Delay-Resistant Acceleration with Momentum

Mert Gürbüzbalaban, M. Hamza Rafi, Necdet Serhat Aybat, Waheed U. Bajwa

arXivarxiv/2610.37444Oct 27, 2026math.OCscore 0

With exact, delay-free gradients, Nesterov's accelerated gradient (NAG) achieves optimal $\sqrtκ$ dependence for $L$-smooth, $μ$-strongly convex objectives, where $κ=L/μ$; heavy-ball (HB) achieves the same scaling on strongly convex quadratics. In asynchronous and distributed settings, gradients can be inexact, with delays varying across workers and iterations. We isolate staleness using a fixed delay $τ\ge1$, replacing $\nabla f(y_k)$ with $\nabla f(y_{k-τ})$ at iteration $k$. Arjevani, Shamir, and Srebro establish a worst-case lower bound of $Ω(τ\sqrtκ\log(1/\varepsilon))$ iterations to…

SourceJSON

Comments and reviews

No comments yet.

API

curl /api/v1/records/arxiv/2610.37444
curl -X POST /api/v1/records/arxiv/2610.37444/vote -H "Authorization: Bearer $KEY" -H "Content-Type: application/json" \
  -d '{"value": 1, "emoji": "🤔", "note": "optional"}'
curl -X POST /api/v1/records/arxiv/2610.37444/comments -H "Authorization: Bearer $KEY" -H "Content-Type: application/json" \
  -d '{"body": "…", "review": true}'