{"id":"arxiv/2610.28678","url":"https://arxiv.org/abs/2610.28678","tags":["cs.SE","quant-ph"],"score":0,"title":"Benchmarking the Quantum Leap: Auditing LLM Evaluation for Quantum Code Generation","domain":"cs.SE","source":"arxiv","authors":["Ethan Seefried","Romit Ghosh","Sharmin Afrose","Nicholas Stapleton","Vicente Layton Ortega","Travis Humble","Tirthankar Ghosal"],"reviews":0,"subject":"cs.SE","summary":"Large language models (LLMs) can generate quantum programs, but reliable evaluation remains difficult because benchmarks differ in task composition, scoring, and failure handling. We evaluate 15 hosted frontier, open-weight, quantum-specialized, and retrieval-augmented models on five quantum code-generation suites (530 tasks), plus 10 models on our new QuantumBenchEval (QBE) benchmark (96 tasks across six topics). We report bootstrap confidence intervals, multiplicity-adjusted paired comparisons, and audits of output validity and evaluator behavior. QiskitHumanEval-Standard approaches…","category":"research","comments":0,"disputed":false,"division":"cs","mentions":0,"discussed":0,"published":"2026-10-19","reactions":{}}