{"id":"arxiv/2605.21292","url":"https://arxiv.org/abs/2605.21292","tags":["stat.ML","cs.AI","cs.LG","math.DS"],"score":0,"title":"Unbiased Gradients, Moving Stability Boundaries: Exact Mini-Batch Geometry in Linear Self-Attention","domain":"stat.ML","source":"arxiv","authors":["Krishnakumar Balasubramanian"],"reviews":0,"subject":"stat.ML","summary":"Unbiased stochastic gradients can match the full gradient in expectation while changing finite-step stability. We study this effect in one-layer linear self-attention for in-context linear regression, where shared-mode mini-batch training reduces exactly to a random two-factor map. The full-batch map preserves an elliptic region, whereas sampled curvature and target correlation create batch-dependent stability boundaries. Their combined fluctuation is centered at the parameter-update level but induces a strictly outward drift of the full-batch boundary coordinate. We derive an exact one-step…","category":"research","comments":0,"disputed":false,"division":"stat","mentions":0,"discussed":0,"published":"2026-05-20","reactions":{}}