Markov decision process based value chain calculation of water distribution network scheduling

•Quantify the cumulative value of scheduling decisions in water distribution network.•Calculate expected value and state transition probability of scheduling decisions.•Conduct method validation with real world scheduling data and different strategies.•Provide direct support for scheduling optimization, minimizing energy consumption.