Solving Decentralized Continuous Markov Decision Problems with Structured Reward

We present an approximation method that solves a class of Decentralized hybrid Markov Decision Processes (DEC-HMDPs). These DEC-HMDPs have both discrete and continuous state variables and represent individual agents with continuous measurable state-space, such as resources. Adding to the natural complexity of decentralized problems, continuous state variables lead to a blowup in potential decision points. Representing value functions as Rectangular Piecewise Constant (RPWC) functions, we formalize and detail an extension to the Coverage Set Algorithm (CSA) [1] that solves transition independent DEC-HMDPs with controlled error. We apply our algorithm to a range of multi-robot exploration problems with continuous resource constraints.