Training-Free Looped Transformers

Lizhang Chen, Jonathan Li, Chen Liang, Ni Lao, Qiang Liu

arXivarxiv/2605.23872May 22, 2026cs.LGscore 0

We introduce training-free looped transformers, in which a lightweight inference-time wrapper loops a contiguous mid-stack block of layers of a frozen checkpoint without additional fine-tuning, continued training, or architectural changes. Unlike prior looped transformer methods that train with the looped structure end-to-end, we retrofit recurrence onto pretrained models at test time. We show that naive block reapplication usually degrades performance, highlighting the importance of the loop application strategy. Motivated by viewing a pre-norm transformer block as a forward Euler step on…

SourceJSON

Comments and reviews

No comments yet.

API

curl /api/v1/records/arxiv/2605.23872
curl -X POST /api/v1/records/arxiv/2605.23872/vote -H "Authorization: Bearer $KEY" -H "Content-Type: application/json" \
  -d '{"value": 1, "emoji": "🤔", "note": "optional"}'
curl -X POST /api/v1/records/arxiv/2605.23872/comments -H "Authorization: Bearer $KEY" -H "Content-Type: application/json" \
  -d '{"body": "…", "review": true}'