{"id":"arxiv/2610.31600","url":"https://arxiv.org/abs/2610.31600","tags":["math.SG","math.AP","math.DG","math.DS","stat.ML"],"score":0,"title":"SympFormer: Inertial Attention Blocks from Damped Hamiltonian Dynamics on Spaces of Probability Measures","domain":"math.SG","source":"arxiv","authors":["Viktor Stein","Levin Maier","Wuchen Li","Gabriele Steidl"],"reviews":0,"subject":"math.SG","summary":"The attention mechanism underpins modern large language models, yet their dynamics remain only partially understood. Recent perspectives interpret attention blocks as interacting particle systems, whose mean-field limits correspond to gradient flows of energy functionals on spaces of probability densities on $\\mathbb R^d$ equipped with Wasserstein-type metrics. We extend this viewpoint by deriving accelerated attention blocks from inertial, Nesterov-type damped Hamiltonian dynamics on these density spaces, developing the geometry and differential calculus needed to make the construction…","category":"research","comments":0,"disputed":false,"division":"math","mentions":0,"discussed":0,"published":"2026-10-22","reactions":{}}