{"id":"arxiv/2605.13434","url":"https://arxiv.org/abs/2605.13434","tags":["cs.LG","cs.DC","math.OC","stat.ML"],"score":0,"title":"Rescaled Asynchronous SGD: Optimal Distributed Optimization under Data and System Heterogeneity","domain":"cs.LG","source":"arxiv","authors":["Ammar Mahran","Artavazd Maranjyan","Peter Richtárik"],"reviews":0,"subject":"cs.LG","summary":"Asynchronous stochastic gradient descent (ASGD) is a standard way to exploit heterogeneous compute resources in distributed learning: instead of forcing fast workers to wait for slow ones, the server updates the model whenever a gradient arrives. Vanilla ASGD applies each arriving gradient with the same weight. When local data distributions are heterogeneous, this becomes problematic: faster workers contribute more updates, and we show theoretically that the method is biased toward a frequency-weighted average of the local objectives rather than the desired global objective. Existing…","category":"research","comments":0,"disputed":false,"division":"cs","mentions":0,"discussed":0,"published":"2026-05-13","reactions":{}}