Deep Reinforcement Learning Based Security-Constrained Battery Scheduling in Home Energy System

The home energy system today involves multiple renewable energy sources and battery energy storage systems, which can be considered as a microgrid. The battery energy storage system is a key component in the home energy system for the sake of filling the gap between the user demand and volatile energy supplies to maximize the techno and economic performances. However, the battery scheduling must suffer the stochastic nature of renewable energy resources and loads, which results in an intractable multi-period stochastic optimization problem with security constraints. An improved actor-critic-based reinforcement learning is proposed for this issue, where a distributional critic net is applied for faster and more accurate reward estimation under uncertainties, and a policy net incorporating protective secondary control is adopted to satisfy security constraints, preventing the unsafe state of batteries during the trial-and-error process. Numerical tests show that the proposed approach outperforms conventional reinforcement learning algorithms, as well as the rule-based battery scheduling approach while guaranteeing safe operation. The robustness and adaptability of the proposed method are also verified in case studies with different optimization tasks.