{"id":"arxiv/2604.14669","url":"https://arxiv.org/abs/2604.14669","tags":["cs.LG","math.DS","math.OC","stat.ML"],"score":0,"title":"Zeroth-Order Optimization at the Edge of Stability","domain":"cs.LG","source":"arxiv","authors":["Minhak Song","Liang Zhang","Bingcong Li","Niao He","Michael Muehlebach","Sewoong Oh"],"reviews":0,"subject":"cs.LG","summary":"Zeroth-order (ZO) methods are widely used when gradients are unavailable or prohibitively expensive, including black-box learning and memory-efficient fine-tuning of large models, yet their optimization dynamics in deep learning remain underexplored. In this work, we provide an explicit step size condition that exactly captures the (mean-square) linear stability of a family of ZO methods based on the standard two-point estimator. Our characterization reveals a sharp contrast with first-order (FO) methods: whereas FO stability is governed solely by the largest Hessian eigenvalue, mean-square…","category":"research","comments":0,"disputed":false,"division":"cs","mentions":0,"discussed":0,"published":"2026-04-16","reactions":{}}