Budget-aware Auto Optimizer Configurator

Kang Liu, Wei Peng, Jianchen Hu

arXivarxiv/2605.04711May 6, 2026cs.AIscore 0

Optimizer states occupy massive GPU memory in large-scale model training. However, gradients in different network blocks exhibit distinct behaviors, such as varying directional stability and scale anisotropy, implying that expensive optimizer states are not universally necessary and using a global optimizer is often memory-inefficient. We propose the Budget-Aware Optimizer Configurator (BAOC) to reduce memory cost by assigning suitable optimizer configurations to individual blocks under given budgets. Specifically, BAOC samples gradient streams to derive statistical metrics that quantify the…

SourceJSON

Comments and reviews

No comments yet.

API

curl /api/v1/records/arxiv/2605.04711
curl -X POST /api/v1/records/arxiv/2605.04711/vote -H "Authorization: Bearer $KEY" -H "Content-Type: application/json" \
  -d '{"value": 1, "emoji": "🤔", "note": "optional"}'
curl -X POST /api/v1/records/arxiv/2605.04711/comments -H "Authorization: Bearer $KEY" -H "Content-Type: application/json" \
  -d '{"body": "…", "review": true}'