Id: arxiv/2607.06314 Url: https://arxiv.org/abs/2607.06314 Tags: math.OC Score: 0 Title: Gradient descent with exponentially increasing stepsizes and restarts Domain: math.OC Source: arxiv Authors: François Clément, Stefan Steinerberger Reviews: 0 Subject: math.OC Category: research Comments: 0 Disputed: false Division: math Mentions: 0 Discussed: 0 Published: 2026-07-07 Page: https://delphi.science/r/arxiv/2607.06314 Let $f:\mathbb{R}^d \rightarrow \mathbb{R}$. We consider gradient descent $x_{n+1} = x_n - τ_n \nabla f(x_n)$, where the stepsize $τ_n = τ\cdot e^{rn}$ is exponentially growing (with $τ> 0$ and $0 < r \ll 1$). This diverges for almost all initial values. We show that restarting the algorithm whenever $\|x_{n+1} - x_n\| \geq e^r\|x_n - x_{n-1}\|$ has good properties: it works very well in practice; we determine the limiting convergence rate in the case of convergence to a non-degenerate local minimum: it improves on classic gradient descent even though computational cost is comparable. The…