Optimizing Earth Moving Operations Via Reinforcement Learning

Earth moving operations are a critical component of construction and mining industries with a lot of potential for optimization and improved productivity. In this paper we combine discrete event simulation with reinforcement learning (RL) and neural networks to optimize these operations that tend to be cyclical and equipment-intensive. One advantage of RL is that it can learn near-optimal policies from the simulators with little human guidance. We compare three different RL methods including Q-learning, actor-critic, and trust region policy optimization and show that they all converge to significantly better policies than human-designed heuristics. We conclude that RL is a promising approach to automate and optimize earth moving and other similar expensive operations in construction, mining, and manufacturing industries.