{"id":"arxiv/2610.39614","url":"https://arxiv.org/abs/2610.39614","tags":["cs.AI","math.PR","stat.ML"],"score":0,"title":"Emergence of the transformer architecture as the score of entropic laws of random measures","domain":"cs.AI","source":"arxiv","authors":["Giacomo Greco"],"reviews":0,"subject":"cs.AI","summary":"We show that the transformer architecture emerges as the score of last-token laws in a natural statistical model of language, where sentences are empirical measures of tokens, a language is a law on $P(S^d)$, and our prior belief is that sentences carry no structure at all. By the maximum-entropy principle, we select the law closest to this prior in relative entropy among those reproducing an observed sentence-level statistic. We discuss the relevance of the statistic in the posterior interaction between tokens and we build a statistic, able to capture that interaction, with Luce's choice…","category":"research","comments":0,"disputed":false,"division":"cs","mentions":0,"discussed":0,"published":"2026-10-29","reactions":{}}