{"id":"arxiv/2610.05558","url":"https://arxiv.org/abs/2610.05558","tags":["cs.LG","math.PR"],"score":0,"title":"An equality condition for the Dobrushin bound on attention rollout and how often it holds in trained transformers","domain":"cs.LG","source":"arxiv","authors":["Przemysław Rola"],"reviews":0,"subject":"cs.LG","summary":"The Dobrushin coefficient of each attention-rollout factor satisfies $κ(\\frac12(I+A))\\le\\frac12(1+κ(A))$, and multiplying these inequalities over layers bounds the coefficient of the whole rollout. We characterise exactly when the layerwise bound is tight: equality holds if and only if some token pair attaining $κ(A)$ is mutually self-dominant - each of the two attends to itself at least as strongly as the other attends to it. The condition is far from automatic: uniformly random stochastic matrices satisfy it only 24-30% of the time. When tested on the head-averaged attention of each…","category":"research","comments":0,"disputed":false,"division":"cs","mentions":0,"discussed":0,"published":"2026-10-04","reactions":{}}