In this paper we study partially observable discrete-time zero-sum games with Borel state spaces and unbounded reward functions. The optimality criterion is the first passage risk probability criterion. We first introduce a new auxiliary zero-sum game problem and establish the corresponding Shapley equation. Then via an approximation approach we obtain the existence of a solution to the Shapley equation under the first passage risk probability criterion. Moreover, applying the Shapley equation we prove the existence of a saddle-point equilibrium.

In this paper we study constrained discrete-time nonzero-sum games with a Borel state space and possibly unbounded cost functions. The transition law is absolutely continuous with respect to some probability measure. Due to the uncountability of the state space, we endow the set of all stationary strategies with the Young topology. For the expected discounted cost criteria, we show the existence of a stationary constrained discount Nash equilibrium via constructing an approximating sequence of the game models. For the expected average cost criteria, we first obtain the continuity of the expected average cost functions with respect to the stationary strategy profile. Then employing the vanishing discount approach we prove that any limit point of the stationary constrained discount Nash equilibrium is a constrained average Nash equilibrium.

In this paper we study partially observable discrete-time Markov decision processes with Borel state and action spaces. The optimality criterion under consideration is the risk-sensitive discounted cost criterion. The cost function is nonnegative and can be unbounded from above. The transition law is required to be semi-uniform Feller. We first construct a sequence of probability measures which are shown to be the joint conditional distribution of the unobservable state and accumulated cost given the observable history. Then we introduce an auxiliary risk-sensitive finite-horizon discounted optimization problem on the extended state space and obtain the properties of the risk-sensitive finite-horizon discounted optimal value function. Furthermore, via the risk-sensitive finite-horizon discounted cost optimality equation and an approximation method, we prove the existence of a deterministic optimal policy under weaker conditions than those in the literature. Finally, we use a stochastic partially observable control system to illustrate our results.

This paper studies partially observable discrete-time stochastic games under the risk probability criterion. The observable and unobservable state spaces are Borel spaces and the reward function is nonnegative. We introduce a sequence of probability measures for which the probabilistic interpretation is given. Moreover, we show that the partially observable zero-sum game problem can be solved via introducing a new auxiliary zero-sum game problem with extended state space. The extended state space consists of the observable state space and the joint distribution of the unobservable state and profit level. Furthermore, we establish the value iteration and Shapley equation for the auxiliary zero-sum game problem. Finally, we prove that there exists a saddle-point equilibrium and thus the value of the game exists.

In this paper we study discrete-time Markov decision processes with Borel state and action spaces under the risk-sensitive average cost criterion. The cost function can be unbounded. We introduce a new kernel and prove the quasi-compactness of the kernel from which the multiplicative Poisson equation is derived. Moreover, we develop a new approach to show the existence of a solution to the risk-sensitive average cost optimality equation and obtain the existence of an optimal deterministic stationary policy. Furthermore, we give two examples to illustrate our results.

This paper studies discrete-time nonzero-sum stochastic games under the risk-sensitive first passage discounted cost criterion. The state space is a countable set and the costs are allowed to be unbounded. Under the suitable optimality conditions, we prove that the risk-sensitive first passage discounted optimal value function of each player is a unique solution to the risk-sensitive first passage optimality equation via an approximation method. Moreover, by the risk-sensitive first passage discounted optimality equation, we show the existence of a randomized Markov Nash equilibrium. Finally, three examples are given to illustrate the results.

In this paper we study the risk-sensitive average optimality for discrete-time Markov decision processes with denumerable states and unbounded costs. We derive the multiplicative Poisson equation under the suitable ergodicity conditions via an approximation method. Moreover, we prove the existence of a unique solution to the risk-sensitive average cost optimality equation and give an equivalent characterization of the set of all optimal stationary policies. Finally, we present the policy iteration algorithm and show its convergence.

This paper studies the risk-sensitive first passage discounted cost criterion for continuous-time Markov decision processes with the Borel state and action spaces. The cost and transition rates are allowed to be unbounded. We introduce a new value iteration to establish the existence of a solution to the risk-sensitive first passage discounted cost optimality equation. Then applying the Feynman–Kac formula, we show that the risk-sensitive first passage discounted cost optimal value function is a unique solution to the risk-sensitive first passage discounted cost optimality equation. Moreover, we derive the existence of a deterministic Markov optimal policy in the class of randomized history-dependent policies. Finally, a cash flow model is given to illustrate the results.

In this paper we study the nonzero-sum stochastic games for continuous-time jump processes under the expected discounted payoff criterion. The state space and action spaces for players are Borel spaces and the reward and transition rates are allowed to be unbounded. Under certain reasonable conditions, we first introduce new auxiliary static games and obtain the corresponding properties of these static games. Then by an approximation approach, we show the existence of a stationary almost Markov Nash equilibrium in the class of randomized history-dependent strategy profiles. Moreover, we use two examples to illustrate our main results.

In this paper, we study discrete-time nonzero-sum stochastic games under the risk-sensitive average cost criterion. The state space is a denumerable set, the action spaces of players are Borel spaces, and the cost functions are unbounded. Under suitable conditions, we first introduce the risk-sensitive first passage payoff functions and obtain their properties. Then, we establish the existence of a solution to the risk-sensitive average cost optimality equation of each player for the case of unbounded cost functions and show the existence of a randomized stationary Nash equilibrium in the class of randomized history-dependent strategies. Finally, we use a controlled population system to illustrate the main results.

We consider nonzero-sum games for continuous-time jump processes with unbounded transition rates under expected average payoff criterion. The state and action spaces are Borel spaces and reward rates are unbounded. We introduce an approximating sequence of stochastic game models with extended state space, for which the uniform exponential ergodicity is obtained. Moreover, we prove the existence of a stationary almost Markov Nash equilibrium by introducing auxiliary static game models. Finally, a cash flow model is employed to illustrate the results.

In this paper we study the risk-sensitive average cost criterion for continuous-time Markov decision processes in the class of all randomized Markov policies. The state space is a denumerable set, and the cost and transition rates are allowed to be unbounded. Under the suitable conditions, we establish the optimality equation of the auxiliary risk-sensitive first passage optimization problem and obtain the properties of the corresponding optimal value function. Then by a technique of constructing the appropriate approximating sequences of the cost and transition rates and employing the results on the auxiliary optimization problem, we show the existence of a solution to the risk-sensitive average optimality inequality and develop a new approach called the risk-sensitive average optimality inequality approach to prove the existence of an optimal deterministic stationary policy. Furthermore, we give some sufficient conditions for the verification of the simultaneous Doeblin condition, use a controlled birth and death system to illustrate our conditions and provide an example for which the risk-sensitive average optimality strict inequality occurs.

In this paper we investigate nonzero-sum games for continuous-time jump processes with Borel state spaces. The optimality criterion to be considered is the expected average payoff criterion. The action spaces for the players are Borel spaces, and the reward and transition rates can be possibly unbounded. Under suitable conditions, we introduce the auxiliary static games and prove the existence of a stationary Nash equilibrium in the class of all randomized history-dependent strategy profiles via a technique of discounted approximation. Moreover, an example is given to illustrate the optimality conditions.

In this paper we study the mean–semivariance problem for continuous-time Markov decision processes with Borel state and action spaces and unbounded cost and transition rates. The optimality criterion is to minimize the semivariance of the discounted total cost over the set of all policies satisfying the constraint that the mean of the discounted total cost is equal to a given function. Under reasonable conditions, we show that the semivariance optimal value function is a solution to the optimality equation of the mean–semivariance criterion by an iteration approach. Moreover, we obtain the existence of mean–semivariance optimal policies from the optimality equation. Furthermore, we give a value iteration algorithm to compute approximately an optimal policy and the optimal value, and analyze the convergence of the algorithm.

This paper concerns the nonzero-sum games for continuous-time jump processes with unbounded transition rates under the risk-sensitive finite-horizon cost criterion. The state space is a countable set and the costs are allowed to be unbounded in the game model. Under the suitable optimality conditions, by introducing an appropriate topology for the set of all randomized Markov multi-strategies and employing the risk-sensitive finite-horizon optimality equations of the players, we prove the existence of a randomized Markov Nash equilibrium in the class of all randomized history-dependent multi-strategies.

In this paper we study nonzero-sum discrete-time stochastic games with an uncountable state space and Borel action spaces under the expected average payoff criterion. The reward functions can be possibly unbounded and the transition law is a convex combination of finitely many probability measures dependent on the state variable and dominated by some probability measure on the state space. We introduce several auxiliary static game models and obtain their properties. Moreover, by a technique of extending the space state, we introduce auxiliary stochastic game models and derive the uniform geometric ergodicity of Markov chains taking values in the extended state space. Furthermore, we show the existence of a stationary almost Markov Nash equilibrium via an approximation method. Finally, we use a resource extraction model to illustrate the main results.

In this paper we study the nonzero-sum constrained stochastic games for continuous-time jump processes with denumerable states and possibly unbounded transition rates. The optimality criterion under consideration is the expected average payoff criterion and the payoff functions of the players are allowed to be unbounded. Under the reasonable conditions, we introduce an approximating sequence of the auxiliary game models and show the existence of stationary constrained Nash equilibria for these approximating game models via employing the average occupation measures and constructing a suitable multifunction. Moreover, we obtain that any limit point of the stationary constrained Nash equilibria for the approximating sequence of the game models is a constrained Nash equilibrium for the original game model. Furthermore, we use a controlled birth and death system to illustrate our main results.

In this paper we study the zero-sum games for continuous-time Markov jump processes under the risk-sensitive finite-horizon cost criterion. The state space is a Borel space and the transition rates are allowed to be unbounded. Under the suitable conditions, we use a new value iteration approach to establish the existence of a solution to the risk-sensitive finite-horizon optimality equations of the players, obtain the existence of the value of the game and show the existence of saddle-point equilibria.

In this paper, we study the risk-sensitive average payoff criterion for the nonzero-sum discrete-time stochastic games with a denumerable state space. The risk-sensitivity coefficient can take positive values and negative values. Under the suitable conditions, we show the existence of a solution to the coupled equations by a technique of the discounted approximation, and obtain the existence of a stationary Nash equilibrium. Moreover, we present some verifiable sufficient conditions imposed on the primitive data of the model for the verification of our assumption and use an example to illustrate that our conditions are weaker than those in the existing literature.