Loading Cost-Aware Model Caching and Request Routing for Cooperative Edge Inference

Most existing works on edge service caching and request routing fail to consider the influence of the service loading time. Meanwhile, the requests generated by end devices will change dynamically, which means that the caching strategy should adapt accordingly. In this paper, we investigate loading cost-aware joint model caching and request routing with cooperative edge computing, considering both the service loading time and the dynamic user requests. A system throughput maximization problem is formulated, which is proved to be NP-hard. Then, a randomized rounding-based online algorithm with M/(M - 2 lnN)-approximation ratio is proposed to solve it, where M and N are the numbers of end devices and deep neural network (DNN) models, respectively. Extensive experimental results demonstrate that our algorithm achieves more than 42.7% throughput gain than baseline algorithms.