{"id":"arxiv/2606.29176","url":"https://arxiv.org/abs/2606.29176","tags":["cs.LG","math.DG","math.OC","stat.ML"],"score":0,"title":"Dead-Direction Conditioners: Gauge-Equivariant Preconditioning for Deep Networks","domain":"cs.LG","source":"arxiv","authors":["Tejas Pradeep Shirodkar"],"reviews":0,"subject":"cs.LG","summary":"A deep network's loss is invariant to continuous symmetries of its parameters: the logit shift, the ReLU rescaling, the LayerNorm scale, the per-head attention rotation. Adam's per-coordinate preconditioner drifts along each symmetry orbit, which pulls the trajectory off the symmetry quotient where the optimization lives and blurs the singular-learning rate the quotient makes readable. We build DDC, a Dead-Direction Conditioner that lifts a base optimizer into a $G$-equivariant one: it conditions the optimizer's state in the orbit decomposition of a $G$-invariant metric, so the trajectory…","category":"research","comments":0,"disputed":false,"division":"cs","mentions":0,"discussed":0,"published":"2026-06-28","reactions":{}}