A reinforcement-probability Bayesian approach for strategic bidding and market clearing for renewable energy sources with uncertainty

With the participation of renewable energy sources (RESs) in the electricity market, the inherent uncertainty brings new challenges for strategic bidding and market clearing in the electricity market. In this paper, the market structure is formulated as a non-cooperative game among power suppliers, consumers, and the market operator. A reinforcement-probability Bayesian approach is proposed to obtain the optimal bidding strategy of RESs and achieve maximal social welfare at the market clearing, considering uncertain cost types of RESs and incomplete market information. First, a reinforcement-probability model is presented to obtain the optimal agent-type-strategy combination considering the uncertainty of cost type selection. A agent-type-probability algorithm is proposed for the model to update the mean of probability density function of cost types according to the expected profit. Second, an incremental-cost-based distributed algorithm is proposed to solve the market clearing problem, which accelerates the convergence rate without violating the system constraints via feedback from the scheduled power. Finally, a Bayesian probability model is proposed for RESs to learn the hidden information of the market and transform the incomplete information electricity market into a complete one. A Bayesian Nash equilibrium is reached in the model. To coordinate the information among market participants, the output of the Bayesian probability model is calculated by the Hysteretic Q-learning algorithm and used as the reward to update the agent-type-probability in the reinforcement-probability model. An illustrative example is presented to demonstrate the effectiveness of the proposed approach, which shows that the proposed approach has a better convergence performance and has a 2-6 times faster learning speed compared with the policy gradient and deep Q-network.