Combinatorial Client-Master Multiagent Deep Reinforcement Learning for Task Offloading in Mobile Edge Computing

Recently, there has been an explosion of mobile applications that performcomputationally intensive tasks such as video streaming, data mining, virtualreality, augmented reality, image processing, video processing, facerecognition, and online gaming. However, user devices (UDs), such as tabletsand smartphones, have a limited ability to perform the computation needs of thetasks. Mobile edge computing (MEC) has emerged as a promising technology tomeet the increasing computing demands of UDs. Task offloading in MEC is astrategy that meets the demands of UDs by distributing tasks between UDs andMEC servers. Deep reinforcement learning (DRL) is gaining attention intask-offloading problems because it can adapt to dynamic changes and minimizeonline computational complexity. However, the various types of continuous anddiscrete resource constraints on UDs and MEC servers pose challenges to thedesign of an efficient DRL-based task-offloading strategy. Existing DRL-basedtask-offloading algorithms focus on the constraints of the UDs, assuming theavailability of enough storage resources on the server. Moreover, existingmultiagent DRL (MADRL)–based task-offloading algorithms are homogeneous agentsand consider homogeneous constraints as a penalty in their reward function. Weproposed a novel combinatorial client-master MADRL (CCM_MADRL) algorithm fortask offloading in MEC (CCM_MADRL_MEC) that enables UDs to decide theirresource requirements and the server to make a combinatorial decision based onthe requirements of the UDs. CCM_MADRL_MEC is the first MADRL in taskoffloading to consider server storage capacity in addition to the constraintsin the UDs. By taking advantage of the combinatorial action selection,CCM_MADRL_MEC has shown superior convergence over existing MADDPG andheuristic algorithms.