Reinforcement Learning Based Collaborative Inference and Task Offloading Optimization for Cloud-Edge-End Systems

Deep Neural Network (DNN) has been widely used in intelligence applications due to its excellent performance in executing inference tasks. Since DNN tasks require a large amount of computation and high-resolution raw input, collaborative inference is proposed to partition the DNN model from the middle layer to minimize the end-to-end latency. However, when existing work attempts to combine collaborative inference with task offloading in cloud-edge-end systems, two challenges result in poor latency and throughput: excessive layers in the model make it difficult to find suitable partitions, and multiple decision variables make the reinforcement learning agent challenging to converge. This paper aims to reduce the long-term average end-to-end latency of DNN tasks by jointly optimizing task offloading, model partitioning, and resource allocation in dynamic environments. To solve the problem of failing to find suitable partitions, we propose a novel Optional Partition Point Compression (OPPC) algorithm, which selects the high-quality partition points based on layers’ output feature to reduce the difficulty of model partitioning. To improve the convergence performance of the agent, we propose a Reinforcement Learning based Collaborative Inference Optimization (RLCIO) algorithm. Unlike the existing reinforcement learning architecture, RLCIO decouples resource allocation using an Edge Computing Resource Allocation (ECRA) algorithm to reduce the agent’s decision variables. Simulation results show that the RLCIO algorithm performs better than five related schemes, reduces the average end-to-end latency of system tasks by 72% and improves the system throughput by 3.5x in the best case.