{"id":"arxiv/2609.21039","url":"https://arxiv.org/abs/2609.21039","tags":["cs.LG","math.NA"],"score":0,"title":"Stiefel-AdamW: Geometry-Aware AdamW for Linear Factorization Blocks","domain":"cs.LG","source":"arxiv","authors":["Emanuele Zangrando","Marco Sutti","Francesco Tudisco"],"reviews":0,"subject":"cs.LG","summary":"A pervasive structural pattern in modern deep learning is the linear factorization block: a submodule of the form $W = BA$ in which two parameter matrices are multiplied directly, with no intervening nonlinearity. Such blocks appear in LoRA adapters, low-rank compressed layers, query-key products of self-attention, and share a common pathology: the factorization is non-unique, which can destabilize training and limit usable learning rates. Despite this, factorization blocks are typically optimized with standard Euclidean methods that ignore the underlying geometry. We introduce…","category":"research","comments":0,"disputed":false,"division":"cs","mentions":0,"discussed":0,"published":"2026-09-17","reactions":{}}