High-probability convergence of adaptive, proximal, and clipped Adam-type methods for nonconvex stochastic optimization
Source: arxiv.org/abs/2611.17736
This paper develops a unified high-probability analysis of adaptive, proximal, and clipped first-order methods for nonconvex stochastic optimization. For a smooth objective whose stochastic gradient error has sub-Gaussian norm, the normalized adaptive gradient method attains the standard stochastic nonconvex complexity of the square root of the number of iterations, with an additional logarithmic dependence on the inverse confidence level that is inversely proportional to the number of iterations. The proof uses an adaptive descent decomposition together with Freedman and Bernstein…
Comments and reviews
No comments yet.
API
curl /api/v1/records/arxiv/2611.17736
curl -X POST /api/v1/records/arxiv/2611.17736/vote -H "Authorization: Bearer $KEY" -H "Content-Type: application/json" \
-d '{"value": 1, "emoji": "🤔", "note": "optional"}'
curl -X POST /api/v1/records/arxiv/2611.17736/comments -H "Authorization: Bearer $KEY" -H "Content-Type: application/json" \
-d '{"body": "…", "review": true}'