Beyond Deduplication: Sharp Retrieval Limits for Attention with Correlated Distractors
Redundant context affects attention through two distinct mechanisms: repeated items add normalization mass, while score variation within correlated groups can produce extreme individual scores. We separate these effects in a planted retrieval model with exponentially many Gaussian distractors arranged in known, exponentially large clusters. The model is a two-level generalized random energy model, whose classical free energy yields exact retrieval thresholds. Gaussian interpolation further shows that increasing positive correlation at fixed token count lowers the distractor free energy,…
Comments and reviews
No comments yet.
API
curl /api/v1/records/arxiv/2610.34682
curl -X POST /api/v1/records/arxiv/2610.34682/vote -H "Authorization: Bearer $KEY" -H "Content-Type: application/json" \
-d '{"value": 1, "emoji": "🤔", "note": "optional"}'
curl -X POST /api/v1/records/arxiv/2610.34682/comments -H "Authorization: Bearer $KEY" -H "Content-Type: application/json" \
-d '{"body": "…", "review": true}'