Learning for Multiple Purposes: A Q-learning Enhanced Hybrid Metaheuristic for Parallel Drone Scheduling Traveling Salesman Problem

In recent years, combining the artificial intelligence techniques with traditional operations research approaches for solving combinatorial optimization problems is an interesting issue in Operations Research (OR) community. There have been numerous studies in the literature that combine machine learning technique with metaheuristic algorithms for a single purpose to improve the latter’s performance. In this paper, we attempted to integrate the reinforcement learning technique Q-learning into a ruin-and-recreate based metaheuristic for multiple purposes, and thus proposed a hybrid heuristic Q-learning aided slack induction by string removals (QSISRs). Incorporating drones into last-mile delivery alongside trucks and couriers is an important trend in the development of urban logistics distribution. The parallel drone scheduling traveling salesman problem (PDSTSP) arises when a certain proportion (from 20%–100%) customers are located within a drone’s flight range from the depot, and drones can directly serve customers from the depot. In this problem, drones and trucks operate independently, and no synchronization is required. The proposed QSISRs was applied to the PDSTSP to evaluate its performance. Numerical experiments validate the effectiveness of the Q-learning technique for algorithm performance enhancement, and also demonstrate the effectiveness of the proposed QSISRS for solving the PDSTSP. Furthermore, comparison results indicate that QSISRs is one of the state-of-the-art heuristic approaches for solving this problem.