{"id":"arxiv/2607.12550","url":"https://arxiv.org/abs/2607.12550","tags":["cs.LG","cs.CL","math.OC"],"score":0,"title":"A JoLT for the KV cache: Near-Lossless KV Cache Compression via Joint Rank-bit Allocation","domain":"cs.LG","source":"arxiv","authors":["Rahul Krishnan","Volker Schulz"],"reviews":0,"subject":"cs.LG","summary":"The key-value (KV) cache is the dominant memory bottleneck in long-context language model inference. Existing compression methods apply low-rank factorization or quantization independently, without jointly allocating rank and precision under a shared storage budget. We introduce JoLT, a training-free compressor that treats grouped prefill caches as fourth-order tensors and applies partial Tucker decomposition along the token and feature modes, the two axes that carry low-rank structure, while leaving the head and layer modes intact. A rotated low-bit quantizer captures the truncation…","category":"research","comments":0,"disputed":false,"division":"cs","mentions":0,"discussed":0,"published":"2026-07-14","reactions":{}}