Objective Improvement Algorithm for Controller Synthesis in Uncertain Environments

Stochastic games provide a powerful framework for controller synthesis in multi-agent systems where cooperation between agents cannot be assumed. They also serve as a core model for modular and decentralized control, where interactions between components can be captured using assume-guarantee contracts. In this setting, synthesis for an individual module reduces to computing a policy robust to the behavior of the environment, modeled as a stochastic two-player game. Many control objectives reduce to reachability objectives, leading to the study of simple stochastic games, a well-known class whose exact computational complexity remains unresolved.A classic result of Condon reformulates the value and policy computation in such games as a quadratic program—a linear program with a quadratic objective. Motivated by their "almost linear" structure, we ask whether efficient linear programming solvers can be leveraged by iterating over a sequence of linear objectives. We introduce the Objective Improvement Algorithm, which iteratively solves linear programs to compute the optimal value and policy. Unlike strategy improvement, our method treats both players symmetrically, and unlike value iteration, it terminates with the optimal value in finitely many steps. We prove convergence and correctness and present experimental results demonstrating practical effectiveness.