Optimal transmission scheduling for remote state estimation in CPSs with energy harvesting two-hop relay networks

This paper considers the optimal transmission power scheduling for accurate remote state estimation in cyber–physical systems (CPSs). The plant is modeled as a discrete-time stochastic linear system with sensor measurements transmitted to the remote estimator over an intelligent relay. A dynamic cooperative team is formulated in which the sensor and the relay sharing the same cost function are equipped with corresponding energy harvesters, and need to decide on the transmission power levels to minimize the average error covariance at the remote estimator. Since the nodes make simultaneous decisions to achieve a common goal, the sensor and the relay do not have access to the transmission power that will be selected by another node in advance, and hence they cannot build a value estimation of the cost function directly at each time point, which poses a challenge to determine the optimal transmission policies of both sides. To overcome this challenge, we provide a local-action value function and prove that both nodes select the transmission power with respect to the local-action value function without any loss of performance. In addition, a Markov decision process (MDP) and a multi-agent reinforcement learning algorithm are presented to obtain the optimal transmission power schedule over an infinite time horizon. Finally, simulation examples are provided to corroborate and illustrate the theoretical results.