Large language model (LLM) inference systems serve clients across multiple subscription tiers with heterogeneous latency requirements and revenue contributions. Existing schedulers optimize throughput or service-level objectives (SLO) attainment, treating all requests as equally valuable. In multi-tier settings, this leads to a fundamental mismatch: a scheduler that maximizes goodput may allocate substantial resources to low-tier requests while high-tier requests miss their stricter SLOs, resulting in suboptimal operator revenue. We present TierServe, a revenue-driven LLM inference scheduler that jointly leverages both subscription-tier priority and per-request SLO slack. TierServe prioritizes high-value requests while dynamically utilizing available slack to improve overall efficiency, aligning scheduling decisions with both business value and latency urgency. We evaluate TierServe across a range of workloads and load conditions. Results show that TierServe consistently improves system revenue over state-of-the-art baselines while maintaining competitive per-tier latency.
Large Language Models (LLMs) have shown strong performance in knowledge-intensive question answering, yet practical deployment on complex Knowledge Graph Question Answering (KGQA) tasks remains challenging. Existing KG-augmented methods often face a trade-off between cost and robustness: interactive search requires frequent LLM calls and incurs high token cost and latency, while rigid retrieval pipelines are vulnerable to early retrieval errors and provide little support for correction. To address this problem, we propose Cognitive Graph Reasoning (CGR), a budget-aware closed-loop framework for efficient KGQA. CGR consists of a query initialization module, System 1, a High-Density Evidence Interface, and System 2. System 1 combines lightweight neural graph search, dynamic pruning, and multi-source collaboration to efficiently construct relevant evidence paths. The High-Density Evidence Interface compresses these paths into compact contexts, and System 2 triggers feedback retrieval when the current evidence is insufficient, mitigating the error propagation problem of single-pass pipelines. Experiments on WebQSP, CWQ, and GrailQA show that CGR maintains or improves accuracy while reducing token consumption by up to 92.9 × . These results suggest that CGR is a practical KGQA solution for resource-constrained settings.
Agent workflows typically rely on exclusive sandbox allocation for resource management. However, prolonged waiting for large language model invocations leads to resource inefficiency. To address this issue, prior work has explored spatial sharing via overcommit and temporal sharing by reusing sandboxes across requests within the same agent. Nevertheless, these approaches achieve limited improvements due to resource contention and sparse invocation patterns. To overcome these limitations, we propose re-allocating idle sandboxes across agents to enable efficient resource sharing. We present aShare, an agent system that decouples computation from communication to enable idle resource re-allocation. It combines offline agent partitioning with online latency-aware scheduling to optimize resource efficiency. Experimental results demonstrate that aShare reduces resource consumption by 35.9
As long-context large language models become increasingly widespread, autoregressive decoding is increasingly bottlenecked by KV-cache footprint and memory bandwidth. Existing KV-cache compression methods typically rely on sparsification or low-bit quantization, but simply combining the two often fails to simultaneously preserve model quality and deliver real end-to-end speedup, since sparsification changes how compressed KV-cache states are represented and executed during decoding. To address this issue, we present JSQKV, a joint sparse-quant framework for KV-cache compression and decode acceleration that co-designs compression policy, compressed format, and execution path. Specifically, JSQKV combines attention-score-based differential sparsification with a dual-window online mechanism, Per-Token-Tile quantization with Hadamard rotation, and a bitmap-based sparse-quant format with a matching decode kernel. Through this joint design, compressed KV-cache data remains compact in memory and is more effectively translated into reduced memory traffic and end-to-end speedup. Experimental results show that, under matched compression budgets, JSQKV generally outperforms the sequential MUSTAFAR+KIVI baseline on representative LongBench tasks, with larger gains under more aggressive sparse-quant settings. On Llama-3-8B-Instruct, JSQKV improves the average score on six representative LongBench tasks from 39.83 to 43.12, while also improving end-to-end throughput by 44.2 × .
