RACE: Risk-Aware Cloud-Edge Orchestration with Online Adaptive Calibration for Trustworthy LLM Services

Edge-deployed large language models (LLMs) face a critical trustworthiness challenge: constrained by model capacity, edge LLMs exhibit substantially higher hallucination rates on knowledge-intensive queries than their cloud counterparts, yet existing edge-cloud scheduling methods route requests based solely on latency or load balancing, lacking any awareness of—or guarantees on—semantic correctness. Reliable routing hinges on quantifying the semantic risk of edge outputs and providing statistical guarantees on error rates. Semantic entropy effectively quantifies semantic risk, and online adaptive calibration—inspired by conformal prediction—offers principled error rate control under distribution shift. However, conventional semantic entropy computation requires multiple sampling passes ($\approx$6 s), which is infeasible on edge devices; moreover, static threshold calibration cannot adapt to distribution shift, causing actual error rates to far exceed the target. To overcome these bottlenecks, we propose RACE (Risk-Aware Cloud-Edge Orchestration), the first framework to integrate online adaptive threshold calibration with semantic risk budgeting, providing provable bounded ($T$-independent) regret and finite-sample coverage guarantees. RACE introduces three key innovations: a Semantic Entropy Probe (SEP) that predicts semantic uncertainty from prompt-only hidden states in a single forward pass ($\lt $1 ms), enabling real-time hallucination detection on edge devices; an online adaptive calibrator that fuses risk control with online convex optimization to dynamically adjust decision thresholds under distribution shift; and a semantic risk budget grounded in Lagrangian duality theory that bounds the cumulative hallucination rate below a user-specified tolerance $\alpha$. Experiments on MMLU and TriviaQA with GPT-5 as the cloud worker show that, with Phi-3-medium as the primary edge model, RACE achieves a cloud invocation rate of only 4.05–6.25% while keeping edge error rates below $\alpha {=}30\%$—the lowest invocation rate among the risk-controlled methods (Static-CP, ACI, L-ARC); experiments with the weaker Mistral-7B further expose the feasibility boundary of $\alpha$ and the role of the adaptation budget $\rho$.