The Stochastic Network Model

In this chapter, we specify the general network model. Specifically, we consider a controller that operates a network with the goal of minimizing certain time average cost, subject to resource constraint, and queue stability or delay constraint. The network is assumed to operate in slotted time, i.e., $$t\in \{0,1,2,...\}$$ . We assume there are $$r\ge 1$$ queues in the network, e.g., the amount of data to be transmitted in cellular networks or the amount of flexible jobs to be scheduled in a smart grid. We also assume that the network is supported by K types of resources, e.g., energy and token for data transmission. In every slot t, we use S(t) to denote the current network state, which indicates the current network parameters, such as a vector of conditions for each network link, or a collection of other relevant information about the current network channels and arrivals. We assume that S(t) takes M different random network states denoted as $$\mathcal {S} = \{s_1, s_2, \ldots , s_M\}$$ . We assume that the underlying governing rules of the state, e.g., the distribution of S(t) if it evolves in an i.i.d. way, are unknown.