{"id":"arxiv/2609.19529","url":"https://arxiv.org/abs/2609.19529","tags":["stat.ML","cs.IT","cs.LG","math.PR","math.ST"],"score":0,"title":"Next-token functional estimation","domain":"stat.ML","source":"arxiv","authors":["Milind Nakul","Vidya Muthukumar","Ashwin Pananjady"],"reviews":0,"subject":"stat.ML","summary":"Suppose we observe the first $n$ points of a sequence of random variables having length $n+1$, and wish to estimate a functional of the unobserved final point and the empirical measure of the $n$ observed training points. Such next-token functionals include the probability that the next token is novel (also known as the surprise probability), the tail probability of the minimum distance between the next token and training points, and the test error of a classifier trained on the observed points. All of these quantities are classically estimated by the leave-one-out method, which is…","category":"research","comments":0,"disputed":false,"division":"stat","mentions":0,"discussed":0,"published":"2026-09-17","reactions":{}}