Constraint-Aware Combinatorial Bandits: Theoretical Foundations and Network Applications

We study the problem of constraint-aware combinatorial multi-armed bandits (CMAB), a novel extension of combinatorial semi-bandits that maximizes total rewards while adhering to long-term cost constraints. In each round, the environment generates stochastic rewards and costs for each arm from unknown distributions. The learning agent selects a combinatorial set of arms that collectively gain rewards and incur costs, which are (partially) observed as feedback to improve future decisions. While prior work has focused on minimizing regret, it often overlooks cost constraints and diverse reward structures across applications. To address this, we introduce a general framework, constraint-aware CMAB with versatile reward functions (C2MAB-V). Unlike existing works that solve a difficult discrete constrained optimization (CO) problem to control constraint violation, we propose bypassing the original discrete CO problem and using a novel relaxation and rounding (RR) approach, which solves a relaxed continuous CO problem with improved approximation guarantees and computational efficiency. A new challenge arises since RR is originally designed for offline CO problems, and due to the flexibility of choosing the relaxation function F and the rounding procedure σ, it is unclear how to guarantee that RR can achieve low regret and low violation simultaneously. In this paper, we are the first to discover unified RR conditions and, equipped with these conditions, we prove that C2MAB-V has the following nice properties: sub-linear regret, fast diminishing violation, and good computational efficiency. We demonstrate the generality of our RR conditions by applying C2MAB-V to diverse applications with concrete (F,σ), i.e., mobile crowdsensing, network routing, and multi-LLM selection, with varying reward functions (e.g., linear, conjunctive, disjunctive, submodular) and feedback models (e.g., semi-bandit, cascading). Extensive experiments on synthetic and real-world datasets, including evaluations with nine LLMs on practical tasks, confirm that C2MAB-V outperforms baselines across these applications.