{"id":"arxiv/2607.21975","url":"https://arxiv.org/abs/2607.21975","tags":["cs.LG","math.OC"],"score":0,"title":"On the Convergence of Stochastic Low-Rank Adaptation","domain":"cs.LG","source":"arxiv","authors":["Ru Wang","Chengchang Liu","John C. S. Lui"],"reviews":0,"subject":"cs.LG","summary":"Low-rank adaptation (LoRA) optimizes $J(B,A)=\\mathcal L(W_\\mathrm{base}+sBA)$ over two adapters $B \\in \\mathbb{R}^{m \\times r}$ and $A \\in \\mathbb{R}^{r \\times n}$ that form a low-rank update to a frozen pretrained weight matrix $W_\\mathrm{base} \\in \\mathbb{R}^{m \\times n}$. The prior analysis shows LoRA-GD takes $\\exp\\{\\mathcal{O}(ε^{-2})\\}$ oracle calls to find an $ε$-stationary point such that $\\|\\nabla J(B,A)\\|\\leq ε$ in the deterministic setting. We sharpen the analysis and show that $\\mathcal{O}(ε^{-4})$ full-gradient evaluations suffice for the same first-order criterion. We further…","category":"research","comments":0,"disputed":false,"division":"cs","mentions":0,"discussed":0,"published":"2026-07-24","reactions":{}}