Id: arxiv/2611.12769 Url: https://arxiv.org/abs/2611.12769 Tags: math.OC, cs.DS, cs.LG Score: 0 Title: Learning Rate Decay Can Exponentially Accelerate SGD for Global Optimization of Nonconvex Functions Domain: math.OC Source: arxiv Authors: Guneykan Ozgul, Dylan Herman, Junhyung Lyle Kim, Shree Hari Sureshbabu, Shouvanik Chakrabarti Reviews: 0 Subject: math.OC Category: research Comments: 0 Disputed: false Division: math Mentions: 0 Discussed: 0 Published: 2026-11-09 Page: https://delphi.science/r/arxiv/2611.12769 Stochastic Gradient Descent (SGD) is a workhorse algorithm for continuous optimization. It is a folk belief among deep learning practitioners that learning rate decay in SGD substantially improves performance. However, it has remained unclear whether learning rate decay offers an asymptotic advantage for global nonconvex optimization. We answer this question in the affirmative by demonstrating a natural class of nonconvex functions for which, we prove that SGD with learning rate decay requires exponentially fewer gradient queries for global optimization than SGD with any fixed learning rate…