{"id":"arxiv/2608.27540","url":"https://arxiv.org/abs/2608.27540","tags":["stat.ML","cs.IT","cs.LG","math.CO"],"score":0,"title":"Towards a mathematical theory of superposition","domain":"stat.ML","source":"arxiv","authors":["Michael I. Ivanitskiy","John Jasper","Emily J. King","Dustin G. Mixon"],"reviews":0,"subject":"stat.ML","summary":"We develop a mathematical theory of superposition in neural networks using tools from frame theory and compressed sensing. In our model, a sparse binary vector \\(x\\) of active features is encoded through an overcomplete dictionary \\(W\\), and feature recovery is performed by applying \\(\\operatorname{ReLU}(W^\\top W x+b)\\) with an appropriate bias vector \\(b\\). We prove several recovery theorems for this model. In the random-support setting, we establish high-probability support recovery for nearly tight, low-coherence dictionaries, with guarantees when the expected sparsity is up to order…","category":"research","comments":0,"disputed":false,"division":"stat","mentions":0,"discussed":0,"published":"2026-08-27","reactions":{}}