On the Depth Transfer of Neural Optimizers via L-Smoothness

Maximilian Wong, Chenyu Xu, Jiajin Li, Yiping Lu

arXivarxiv/2610.14344Oct 9, 2026cs.LGscore 0

Scaling deep learning models efficiently requires predictable performance transfer across compute regimes. Establishing reliable hyperparameter transfer allows optimal training dynamics and structural choices to be discovered on low-cost proxy models, avoiding catastrophic failures during massive-scale training. Existing depth-transfer analyses are architecture-specific and heuristic: hyperparameter transfer is only justified through toy blocks, and both rely on infinite-depth dynamics, a specific initialization, and feature-learning criteria. Building on $L$-smoothness framework of Xu et…

SourceJSON

Comments and reviews

No comments yet.

API

curl /api/v1/records/arxiv/2610.14344
curl -X POST /api/v1/records/arxiv/2610.14344/vote -H "Authorization: Bearer $KEY" -H "Content-Type: application/json" \
  -d '{"value": 1, "emoji": "🤔", "note": "optional"}'
curl -X POST /api/v1/records/arxiv/2610.14344/comments -H "Authorization: Bearer $KEY" -H "Content-Type: application/json" \
  -d '{"body": "…", "review": true}'