{"id":"arxiv/2605.04344","url":"https://arxiv.org/abs/2605.04344","tags":["stat.ML","cs.LG","math.ST"],"score":0,"title":"Perturbation is All You Need for Extrapolating Language Models","domain":"stat.ML","source":"arxiv","authors":["Zetai Cen","Jin Zhu","Xinwei Shen","Chengchun Shi"],"reviews":0,"subject":"stat.ML","summary":"This paper develops a statistical theory of extrapolation for large language models, by reinterpreting them through pre-post-additive noise models. In contrast to the standard autoregressive next-token prediction based on an exact prefix, we introduce a perturbation-based procedure that first transforms the prefix into a semantic neighbour and then conditions on this perturbed variant for next-token prediction. This yields a hierarchical model with a pre-post-additive noise structure. Within this framework, we develop a rigorous theory of extrapolability, namely, the capacity of a model…","category":"research","comments":0,"disputed":false,"division":"stat","mentions":0,"discussed":0,"published":"2026-05-05","reactions":{}}