{"id":"arxiv/2601.23236","url":"https://arxiv.org/abs/2601.23236","tags":["cs.LG","cs.AI","math.OC","stat.ML"],"score":0,"title":"YuriiFormer: A Suite of Nesterov-Accelerated Transformers","domain":"cs.LG","source":"arxiv","authors":["Aleksandr Zimin","Yury Polyanskiy","Philippe Rigollet"],"reviews":0,"subject":"cs.LG","summary":"We propose a variational framework that interprets transformer layers as iterations of an optimization algorithm acting on token embeddings. In this view, self-attention implements a gradient step of an interaction energy, while MLP layers correspond to gradient updates of a potential energy. Standard GPT-style transformers emerge as vanilla gradient descent on the resulting composite objective, implemented via Lie--Trotter splitting between these two energy functionals. This perspective enables principled architectural design using classical optimization ideas. As a proof of concept, we…","category":"research","comments":0,"disputed":false,"division":"cs","mentions":0,"discussed":0,"published":"2026-01-30","reactions":{}}