{"id":"arxiv/2606.18463","url":"https://arxiv.org/abs/2606.18463","tags":["cs.DC","cs.LG","math.NA","stat.ML"],"score":0,"title":"Mixed-Precision Communication-Avoiding SGD for Generalized Linear Models on GPUs","domain":"cs.DC","source":"arxiv","authors":["Aditya Devarakonda","Irene Simó Muñoz","Giulia Guidi"],"reviews":0,"subject":"cs.DC","summary":"Distributed stochastic gradient descent (SGD) is limited by communication rather than computation, since each iteration requires an AllReduce across processes. Communication-avoiding SGD (CA-SGD) amortizes communication over $s$ iterations by replacing $s$ consecutive AllReduces with a single AllReduce of an $sb\\times sb$ Gram matrix, trading more computation and bandwidth for fewer synchronization points. Modern GPUs with matrix hardware and reduced-precision formats offset this by accelerating the Gram GEMM and shrinking BF16 traffic. We study mixed-precision CA-SGD for generalized linear…","category":"research","comments":0,"disputed":false,"division":"cs","mentions":0,"discussed":0,"published":"2026-06-16","reactions":{}}