Efficient Multivariate Bandit Algorithm with Path Planning

We solve the arms exponential exploding issues in the Multivariate-MAB when the arm dimension hierarchy is considered. We propose a framework called path planning, which utilizes paths in a graph to model reward success rate with m-way dimension interaction and adopts Thompson Sampling (TS) for a heuristic search. It is straightforward to combat the curse of dimensionality using a serial process that operates sequentially by focusing on one dimension per each process. Our proposed method utilizing tree models has advantages comparing with traditional models such as general linear regression. Real data and simulation studies validate our claim by achieving faster convergence speed, better efficient optimal arm allocation, and lower cumulative regret.