{"id":"arxiv/2605.05341","url":"https://arxiv.org/abs/2605.05341","tags":["cs.LG","cs.AI","math.OC","stat.ML"],"score":0,"title":"Feature Starvation as Geometric Instability in Sparse Autoencoders","domain":"cs.LG","source":"arxiv","authors":["Faris Chaudhry","Keisuke Yano","Anthea Monod"],"reviews":0,"subject":"cs.LG","summary":"Sparse autoencoders (SAEs) are used to disentangle the dense, polysemantic internal representations of large language models (LLMs) into interpretable, monosemantic concepts. However, standard $\\ell_1$-regularized SAEs suffer from feature starvation (dead neurons) and shrinkage bias, often requiring computationally expensive heuristic resampling and nondifferentiable hard-masking methods to bypass these challenges. We argue that feature starvation is not merely an empirical artifact of poor data diversity, but a fundamental optimization-geometric pathology of overcomplete dictionaries: the…","category":"research","comments":0,"disputed":false,"division":"cs","mentions":0,"discussed":0,"published":"2026-05-06","reactions":{}}