With the relentless increase in computing power and the ubiquitous availability of data in many industries, the fields of simulation optimization and artificial intelligence have emerged at the scientific and engineering forefront in their societal impact, manifested in the pervasiveness of technologies such as large language models, chatbots, digital twins, and agent-based systems. We examine cross-fertilization between simulation optimization and artificial intelligence, with a particular focus on reinforcement learning, highlighting research that has been mutually beneficial. After reviewing relevant methodologies from stochastic simulation and optimization, we provide an overview of foundational approaches of machine learning, and then present examples of the synergies between the fields, followed by real-world applications and case studies, including some futurist and forward-looking concepts.
In emerging fields such as machine learning, quantum computing, biomedical imaging, and robotics, data and decisions often exist in curved, non-Euclidean spaces due to physical constraints or underlying symmetries. Riemannian online optimization provides a new framework for handling learning tasks where data arrives sequentially in geometric spaces. This monograph offers a comprehensive overview of online learning over Riemannian manifolds.
This monograph deals with methods for stochastic or data-driven optimization. The overall goal in these methods is to minimize a certain parameter-dependent objective function that for any parameter value is an expectation of a noisy sample performance objective whose measurement can be made from a real system or a simulation device depending on the setting used. We present a class of model-free approaches based on stochastic approximation which involve random search procedures to efficiently make use of the noisy observations. The idea here is to simply estimate the minima of the expected objective via an incremental-update or recursive procedure and not to estimate the whole objective function itself. We provide both asymptotic as well as finite sample analyses of the procedures used for convex as well as non-convex objectives. We present algorithms that either estimate the gradient in gradient-based schemes or estimate both the gradient and the Hessian in Newton-type procedures using random direction approaches involving noisy function measurements. Hence the class of approaches that we study fall under the broad category of zeroth order optimization methods. We provide both asymptotic convergence guarantees in the general setup as well as asymptotic normality results for various algorithms. We also provide an introduction to stochastic recursive inclusions as well as their asymptotic convergence analysis. This is necessitated because many of these settings involve set-valued maps for any given parameter. We also present a couple of interesting applications of these methods in the domain of reinforcement learning. Five appendices at the end of this work quickly summarize the basic material. A large portion of this work is driven by our own contributions to this area.
This article describes a novel optimization solution framework, called alternating gradient descent (GD) and minimization (AltGDmin), that is useful for many problems for which alternating minimization (AltMin) is a popular solution. AltMin is a special case of the block coordinate descent algorithm that is useful for problems in which minimization w.r.t one subset of variables keeping the other fixed is closed form or otherwise reliably solved. Denote the two blocks/subsets of the optimization variables Z by Za, Zb, i.e., Z = Za, Zb. AltGDmin is often a faster solution than AltMin for any problem for which (i) the minimization over one set of variables, Zb, is much quicker than that over the other set, Za; and (ii) the cost function is differentiable w.r.t. Za. Often, the reason for one minimization to be quicker is that the problem is “decoupled" for Zb and each of the decoupled problems is quick to solve. This decoupling is also what makes AltGDmin communication-efficient for federated settings. Important examples where this assumption holds include (a) low rank column-wise compressive sensing (LRCS), low rank matrix completion (LRMC), (b) their outlier-corrupted extensions such as robust PCA, robust LRCS and robust LRMC; (c) phase retrieval and its sparse and low-rank model based extensions; (d) tensor extensions of many of these problems such as tensor LRCS and tensor completion; and (e) many partly discrete problems where GD does not apply – such as clustering, unlabeled sensing, and mixed linear regression. LRCS finds important applications in multi-task representation learning and few shot learning, federated sketching, and accelerated dynamic MRI. LRMC and robust PCA find important applications in recommender systems, computer vision and video analytics.
We provide a comprehensive survey of Integer Programming Games (IPGs), focusing on both simultaneous games and bilevel programs. These games are characterized by integral constraints within the players’ strategy sets. We start from the fundamental definitions of these games and various solution concepts associated with them, and derive the properties of the games and the solution concepts. For each of the two types of games – simultaneous and bilevel – we have one section dedicated to the analysis of the games and another section dedicated to the development and analyses of algorithms to solve them. The analyses sections present results on the computational complexity of the general game as well as various other restricted versions. These sections also discuss the structural properties of the games and the equilibrium concepts associated with them. The algorithm sections, in contrast, present some of the state-of-the-art algorithms developed to solve these games, either exactly, approximately or fast under fixed-parameter assumptions. These sections also contain proofs of the correctness of these algorithms and an assessment of their theoretical run times in the worst-case scenario.
