Energy management strategy via maximum entropy reinforcement learning for an extended range logistics vehicle

The modern energy management strategy (EMS) plays a vital role in the energy efficiency of the extended range electric vehicle. However, some modern strategies such as model predictive control (MPC) and dynamic programming (DP) have limited practical potential because they are subject to the pre-known environment information and noise interference. The reinforcement learning (RL)control strategy can be adopted as online control to interact with the vehicle and the environment. In this study, a novel auxiliary power unit (APU) charging strategy with multi-object optimization is proposed to achieve high fuel conversion efficiency while maintaining battery charging health. The state-of-the-art algorithm, Soft Actor-Critic (SAC), is applied to achieve better exploration of the possible APU behaviour and solve the sensitivity and poor convergence problems from the current RL studies. Its performance is further verified by the results of the Deep Deterministic Policy Gradient (DDPG) algorithm and DP. Three innovative targets are selected as the RL rewards for optimization: the engine fuel rate, SOC charging trajectory, and the battery charging rate (C-rate). The first adoption of the battery C-rate monitoring in RL-based energy management strategy helps extend the battery lifespan from excessive discharge. The comparative results show that the SAC had a 36% faster convergence speed than DDPG while providing a smoother and more stable action space. The fuel consumption with SAC also outplays that of DDPG by around 3%, which achieves almost 95% of the global optimization result. The successful deployment of the SAC algorithm as an EMS indicates its standout ability in dealing with wide-range actions and states with high randomness, revealing the practical potential compared with the existing RL strategies. (C) 2022 Elsevier Ltd. All rights reserved.