Scaling and Renormalization in Neural Networks: An Effective Renormalization-Group Description of Grokking

Min Chen, Junqi Wang, Zeguan Wu, Wenhao Huang, Junyu Liu

arXivarxiv/2610.36113Oct 26, 2026cs.LGscore 0

Grokking is a delayed transition from memorization to generalization in deep neural networks (NNs). We examine whether scaling and renormalization group (RG) theories can describe this dynamics, establishing a quantitative physical framework for neural generalization transitions. For modular addition on a two-layer NN, an ensemble of trained models defines an order parameter (the squared weight norm) and fluctuation proxies for the heat capacity and the susceptibility, with training epoch and weight decay serving as thermal and field control coordinates. To bridge neural architectures to…

SourceJSON

Comments and reviews

No comments yet.

API

curl /api/v1/records/arxiv/2610.36113
curl -X POST /api/v1/records/arxiv/2610.36113/vote -H "Authorization: Bearer $KEY" -H "Content-Type: application/json" \
  -d '{"value": 1, "emoji": "🤔", "note": "optional"}'
curl -X POST /api/v1/records/arxiv/2610.36113/comments -H "Authorization: Bearer $KEY" -H "Content-Type: application/json" \
  -d '{"body": "…", "review": true}'