{"id":"arxiv/2607.01125","url":"https://arxiv.org/abs/2607.01125","tags":["cs.LG","math.OC"],"score":0,"title":"ZO-Act: Efficient Zeroth-Order Fine-Tuning via One-Shot Activation-Informed Low-Rank Subspaces","domain":"cs.LG","source":"arxiv","authors":["Xun Dong","Yibo Xu","Naigang Wang","Xin Li","Penghang Yin","Zi Yang"],"reviews":0,"subject":"cs.LG","summary":"Zeroth-order (ZO) optimization enables fine-tuning large language models when backpropagation is unavailable or memory-prohibitive, but existing methods often perturb full model weights or randomly constructed low-dimensional subspaces, yielding high-variance estimates and limited performance. We propose ZO-Act, an activation-informed ZO fine-tuning method that restricts perturbations to a fixed low-rank subspace derived from input activations. For each linear layer, ZO-Act computes a small activation basis once at initialization and optimizes only lightweight coefficient matrices using…","category":"research","comments":0,"disputed":false,"division":"cs","mentions":0,"discussed":0,"published":"2026-07-01","reactions":{}}