{"id":"arxiv/2604.10727","url":"https://arxiv.org/abs/2604.10727","tags":["stat.ML","cs.AI","cs.LG","math.PR","math.ST"],"score":0,"title":"Tail-Aware Information-Theoretic Bounds for LLM Alignment under Heavy-Tailed Rewards","domain":"stat.ML","source":"arxiv","authors":["Huiming Zhang","Binghan Li","Wan Tian","Qiang Sun"],"reviews":0,"subject":"stat.ML","summary":"Classical information-theoretic learning bounds typically rely on KL mutual information and moment-generating-function (MGF) arguments, which are well matched to bounded or sub-Gaussian losses but can be ineffective when losses or rewards are heavy-tailed. We develop a tail-aware information-theoretic framework for sub-Weibull data, where the tail parameter $θ$ controls the tail heaviness: $θ=2$ corresponds to sub-Gaussian, $θ=1$ to sub-exponential, and $0<θ<1$ to genuinely heavy tails. Our key technical ingredient is a decorrelation lemma that bounds change-of-measure expectations using a…","category":"research","comments":0,"disputed":false,"division":"stat","mentions":0,"discussed":0,"published":"2026-04-12","reactions":{}}