{"id":"arxiv/2609.35055","url":"https://arxiv.org/abs/2609.35055","tags":["cs.LG","math.OC","stat.ML"],"score":0,"title":"Universality and Generalization of Causal Transformers Across Context Lengths","domain":"cs.LG","source":"arxiv","authors":["Takashi Furuya","Maarten V. de Hoop","Gabriel Peyré"],"reviews":0,"subject":"cs.LG","summary":"Long contexts are central to modern transformer systems, but most expressivity results choose a different network for each fixed sequence length. We study whether one masked transformer can approximate causal token-to-token maps uniformly over sequences of arbitrary length sampling a fixed normalized horizon. To relate sampling resolutions, we model tokens by $α$-Hölder sequences or, more generally, a common modulus of continuity. Our notion of continuity across resolutions characterizes the causal families admitting uniform approximation on these compact input classes by a single…","category":"research","comments":0,"disputed":false,"division":"cs","mentions":0,"discussed":0,"published":"2026-09-28","reactions":{}}