Learning-Based Edge-Device Collaborative DNN Inference in IoVT Networks.

Deep Neural Network (DNN) is a promising technology for Internet of Visual Things (IoVT) devices to extract their visual information from unstructured data. However, it is hard to deploy a complete DNN model at resource-constrained IoVT devices to fulfill their latency, energy, and inference accuracy demands. Exploiting the reachable and available computing resources of IoVT devices and MEC servers, we propose an edge-device collaborative DNN inference framework to empower resource-constrained IoVT devices to perform DNN-based inference. Especially, the DNN model partition separates the DNN model into two parts, which are deployed on both the IoVT devices and multi-access mobile edge computing (MEC) server for performing inference collaboratively. The DNN early exit and computation resource allocation are employed to accelerate the DNN inference while guaranteeing the inference accuracy. Moreover, a metric to measure the inference performance of average latency and accuracy (IPLA) is designed. Joint multi-user DNN partitioning, early exit point selection, and computation resource allocation are optimized to maximize the tradeoff performance of inference latency and accuracy. We model the optimized problem as a Markov Decision Process and propose a Deep Deterministic Policy Gradient-based edge-device collaborative DNN inference algorithm to solve the problem of huge state space and high-dimensional continuous actions. Experiments are conducted with the Alexnet model on the dataset of CIFAR-10 and Resnet-50 model on the dataset of ImageNet. Simulation results verify that the proposed algorithm speeds up the overall inference execution of IoVT devices while guaranteeing inference accuracy.