Id: arxiv/2607.24502 Url: https://arxiv.org/abs/2607.24502 Tags: math.DS, cs.LG Score: 0 Title: Self-Attention Dynamics with Rotary Position Embeddings: Twisted States and Explicit Consensus Rates on the Sphere Domain: math.DS Source: arxiv Authors: Hao Ye Reviews: 0 Subject: math.DS Category: research Comments: 0 Disputed: false Division: math Mentions: 0 Discussed: 0 Published: 2026-07-27 Page: https://delphi.science/r/arxiv/2607.24502 Rotary position embeddings (RoPE) modify attention scores through position-dependent rotations, but their effect on normalized token dynamics is not captured by the vanilla spherical self-attention model. We study the continuous-time dynamics obtained when queries and keys are rotated while values remain on the unit sphere. The resulting attention kernel is reversible and admits a sharp uniform softmax floor, yet the natural RoPE interaction energy has derivatives of both signs within one fixed nontrivial system. Every consensus state remains an equilibrium, and its transverse linearization…