Distributed Decision Making in Cyber Physical Network Systems

Cyber-physical network systems are ubiquitous and play an indispensable role in advancing our modern society, where more and more devices with sensing, computation, communication, actuation capabilities are connected to revolutionize the way urban systems operate. Regardless of the specific application, the central goal is to achieve a desired network collective behavior through the design of admissible distributed decision making algorithms for individual devices, harnessing their sensing, computation, communication and actuation capabilities. While various decision making tools have been developed, achieving efficient decision making in a network setting is in many aspects different from traditional decision making theory. First of all, the decision making process is distributed in nature. There is not just a single decision maker; instead, each individual node in the network act as a decision maker and their decisions will collectively shape the global network behavior. Secondly, individual nodes' decision making is constrained by the limited information available to them. What further complicates the problem is the often unreliable communication links and the uncertain and even time-varying disturbances. All these challenges call for new studies of decision making theory tailored for network systems. In this dissertation, we tackle these challenges under three specific problem settings: distributed optimization, distributed voltage control in power network, and distributed localized policy optimization in multi-agent Markov decision process. In the distributed optimization problem, there are a set of agents, each of which has a local convex cost function , and the …