{"id":"arxiv/2609.12123","url":"https://arxiv.org/abs/2609.12123","tags":["cs.LG","math.OC","stat.ML"],"score":0,"title":"Rank-Efficient LoRA via Joint Tangent-Space Optimization under Isotropic Curvature","domain":"cs.LG","source":"arxiv","authors":["Zihan Zhu","Zhehang Du","Xuyang Chen","Tim Tsz-Kit Lau","Jiayuan Wu","X. Y. Han","Qi Long","Weijie Su"],"reviews":0,"subject":"cs.LG","summary":"Low-Rank Adaptation (LoRA) is an effective approach for adapting large pretrained models by learning low-rank weight updates. In practice, the LoRA rank is used to control an adapter's parameter budget and representational capacity. We show that this view is incomplete: while the nominal rank determines the representational capacity, the optimizer shapes how much of that capacity is used in the induced weight-space updates. In a case study of GPT-2 adaptation with LoRA, we observe a strong rank-dependent optimizer effect. Despite using the same nominal rank, AdamW often produces per-step…","category":"research","comments":0,"disputed":false,"division":"cs","mentions":0,"discussed":0,"published":"2026-09-10","reactions":{}}