Decentralized multi-objective cloud energy storage operation control with deep reinforcement learning

The high proportions of renewable energy resources in modern distribution power systems raise a great challenge for voltage regulations. Could Energy Storage (CES) aggregates different energy storage devices and can be utilized to regulate the system voltage. We formulate a multi-objective CES energy management problem using CES resources to regulate the system voltage and reduce the system energy losses. Then, we embed the CES model into the problem considering the CES characteristics. However, the problem is hard to solve using optimization methods due to the non-convex constraints and online control requirements. This research proposes a two-stage CES energy management framework using the deep reinforcement learning method to obtain control decisions. We first partition the networks into sub-networks considering the reactive power support effects. In the offline stage, we combine the imitation learning method with the multi-agent deep deterministic policy gradient algorithm to train the CES agents. The pre-trained agents are further transferred into the real environment and safely explore it in the online stage. The numerical results on the IEEE 33-bus case online test show that our framework achieves good performances of the voltage regulations and the system energy loss saving.