Deep Reinforcement Learning for Autonomous Control of Low Voltage Grids with Focus on Grid Stability in Future Power Grids

The effort to achieve a low-emission electrical grid in Germany poses challenges for distribution system operators. These include the expansion of volatile and decentralised re-newable energy resources (DER), such as photovoltaics, and the steady expansion of electromobility with high charging capacities in the low voltage (LV) grid. To enhance the grid stability and to avoid costly grid expansion, a autonomous control systems must be implemented. For this purpose a double deep Q-learning (DDQN) approach with multiple agents are being investigated. The reinforcement learning (RL) agents are being trained and validated on a LV grid with grid instabilities such as voltage drops, transformer overload and reverse power flow to the medium voltage level. The grid consists of 13 nodes, 1 string with 11 households, 11 PV systems with battery storage systems (BSS) and 11 electrical vehicle (EV). As results the voltage level drops are decrease from 0.89 p.u. to 0.95 p.u. compared between controlled and uncontrolled cases. Also, a reverse power flow can be fully prevented. The transformer utilisation can be decreased by a maximum of 1 p.u.. Moreover, the batteries are more often utilised for global grid stability purposes then optimise local self consumption.