{"id":"arxiv/2609.14485","url":"https://arxiv.org/abs/2609.14485","tags":["cs.LG","math.ST"],"score":0,"title":"Retrieval-Guided Fine-Tuning as Noisy Estimation: Risk bounds and Architectural Analysis","domain":"cs.LG","source":"arxiv","authors":["Bhargav Lad","Yifan Hao"],"reviews":0,"subject":"cs.LG","summary":"Retrieval-Guided Fine-Tuning (RAG-FT) incorporates retrieved data directly into the training objective, but the statistical consequences of noisy retrieval during training remain theoretically undercharacterized. We study this question by modeling RAG-FT as an estimation problem in a multi-task linear regression framework, using an OLS proxy for single-layer linear self-attention to obtain finite-sample risk bounds. Under homoscedastic retrieval noise, we show that retrieval failure decays exponentially with task separation relative to noise, and derive explicit finite-sample conditions…","category":"research","comments":0,"disputed":false,"division":"cs","mentions":0,"discussed":0,"published":"2026-09-13","reactions":{}}