Efficient Pump Scheduling for Large‐Scale Multiproduct Pipelines Using Deep Reinforcement Learning

Optimizing pump scheduling in multiproduct pipelines can significantly reduce energy consumption and carbon emissions. For pump scheduling in multiproduct pipelines, to describe hydraulic losses more accurately, the model needs to adopt shorter discrete time intervals, which will lead to longer decision-making time. It combined with the large solution space of large-scale pipelines, will lead to low solution efficiency with dynamic programming methods and poor solution quality with heuristic optimization algorithms. Given that, this article develops a multiproduct refined oil transmission simulation system and employs the enhanced Proximal Policy Optimization (PPO) algorithm with action space shaping trick to optimize pump scheduling for large-scale multiproduct pipelines. The method of converting discrete action space to multi-discrete action space through action shaping can address PPO's low convergence efficiency issue resulting from the large discrete action space challenge common in large-scale multiproduct pipelines. The experimental results indicate that the proposed method, that is, PPO algorithm with multidiscrete action space exhibits significant advantages in terms of efficiency and robustness in large-scale pipelines compared to mainstream methods for pump scheduling such as dynamic programming (DP), genetic algorithms (GA), and ant colony optimization (ACO). Furthermore, we demonstrate the effectiveness of action space shaping in large-scale pipelines from the perspectives of exploration and exploitation. PPO algorithm with action space shaping interacts with the refined oil transmission pipeline system.image