{"id":"arxiv/2607.03948","url":"https://arxiv.org/abs/2607.03948","tags":["cs.AI","cs.LG","math.OC"],"score":0,"title":"Online Linear Programming for Multi-Objective Routing in LLM Serving","domain":"cs.AI","source":"arxiv","authors":["Zixi Chen","Yinyu Ye","Zijie Zhou"],"reviews":0,"subject":"cs.AI","summary":"We study the online routing problem in large language model serving, where requests arrive sequentially and must be dispatched to parallel decode workers under tight batch-size and KV-cache constraints. Unlike widely used routing heuristics that are not tied to explicit service-level objectives (SLOs) and offer limited control over latency-throughput trade-offs, we introduce a multi-objective optimization framework that formulates routing as an online linear programming with interpretable decision rewards. We apply an efficient bid-price control policy based on the online linear programming…","category":"research","comments":0,"disputed":false,"division":"cs","mentions":0,"discussed":0,"published":"2026-07-04","reactions":{}}