{"id":"arxiv/2606.30931","url":"https://arxiv.org/abs/2606.30931","tags":["cs.AI","cs.LG","cs.MA","math.OC","math.PR"],"score":0,"title":"RoPoLL: Robust Panel of LLM Judges","domain":"cs.AI","source":"arxiv","authors":["Anish Acharya","Kris W Pan","Brian Verkhovsky"],"reviews":0,"subject":"cs.AI","summary":"The LLM Jury, a Panel of LLM Evaluators (PoLL) reporting consensus scores, has become a practical alternative to single-judge LLM evaluation, yet its statistical behavior remains poorly understood. We formalize the LLM Jury under the Huber contamination model and show that PoLL incurs unbounded bias under any positive contamination, regardless of jury size, whenever a single judge fails in a biased, LLM-typical way (mode collapse, sycophancy, safety refusal). Framing jury consensus as classical robust mean estimation, we propose RoPoLL (Robust Panel of LLM-as-Judge), which preserves the PoLL…","category":"research","comments":0,"disputed":false,"division":"cs","mentions":0,"discussed":0,"published":"2026-06-29","reactions":{}}