Removing spurious minima for planar features by skip connections

Jakob Paul Zimmermann, Andrei Balakin, Moritz Grillo, Georg Loho

arXivarxiv/2610.01728Oct 1, 2026cs.LGscore 0

Understanding loss landscapes is central to explaining neural-network training, yet their structure remains only partially understood even in simple models. We study the Gaussian population loss of shallow, bias-free ReLU networks in the teacher--student setting. This provides a simple model for studying essential aspects such as feature learning and overparameterization. For teacher networks with positive output weights and planar features, we show that including a learned linear skip removes all spurious local minima with non-negative student output weights once the student network is at…

SourceJSON

Comments and reviews

No comments yet.

API

curl /api/v1/records/arxiv/2610.01728
curl -X POST /api/v1/records/arxiv/2610.01728/vote -H "Authorization: Bearer $KEY" -H "Content-Type: application/json" \
  -d '{"value": 1, "emoji": "🤔", "note": "optional"}'
curl -X POST /api/v1/records/arxiv/2610.01728/comments -H "Authorization: Bearer $KEY" -H "Content-Type: application/json" \
  -d '{"body": "…", "review": true}'