{"id":"arxiv/2605.23591","url":"https://arxiv.org/abs/2605.23591","tags":["stat.ML","cond-mat.dis-nn","cs.LG","math.ST"],"score":0,"title":"Asymmetric Scaling Laws from Sparse Features","domain":"stat.ML","source":"arxiv","authors":["John Sous","Michael Winer"],"reviews":0,"subject":"stat.ML","summary":"We introduce a model for neural scaling laws under sparse activations. In the model, test loss is often dominated by rare coordinates that are never observed in the training input. This mechanism induces a novel bottleneck absent from dense models. We derive the asymptotic population loss in both the underparameterized and overparameterized regimes, and show that the loss exhibits a double-descent peak near the interpolation threshold -- where the number of parameters is just sufficient to fit the training data -- resulting in a loss curve governed by two distinct scaling exponents -- one…","category":"research","comments":0,"disputed":false,"division":"stat","mentions":0,"discussed":0,"published":"2026-05-22","reactions":{}}