{"id":"arxiv/2605.04711","url":"https://arxiv.org/abs/2605.04711","tags":["cs.AI","cs.LG","math.OC"],"score":0,"title":"Budget-aware Auto Optimizer Configurator","domain":"cs.AI","source":"arxiv","authors":["Kang Liu","Wei Peng","Jianchen Hu"],"reviews":0,"subject":"cs.AI","summary":"Optimizer states occupy massive GPU memory in large-scale model training. However, gradients in different network blocks exhibit distinct behaviors, such as varying directional stability and scale anisotropy, implying that expensive optimizer states are not universally necessary and using a global optimizer is often memory-inefficient. We propose the Budget-Aware Optimizer Configurator (BAOC) to reduce memory cost by assigning suitable optimizer configurations to individual blocks under given budgets. Specifically, BAOC samples gradient streams to derive statistical metrics that quantify the…","category":"research","comments":0,"disputed":false,"division":"cs","mentions":0,"discussed":0,"published":"2026-05-06","reactions":{}}