Extreme events – such as earthquakes and coronal mass ejections – are common in many chaotic dynamical systems, yet are difficult to characterize and predict due to the subtle instability mechanisms that drive them. In this work, we develop an interpretable technique that reveals the underlying mechanisms behind extreme events and uses them to build data-driven forecasts and intuitive event suppression controllers. In particular, we utilize the covariance balancing reduction using adjoint snapshots (CoBRAS) method to identify linear oblique projections that best capture the sensitivity of a quantity of interest and reconstruct the original state. Importantly, we bypass the need for cumbersome adjoint calculations, instead using backpropagation via modern automatically differentiable numerical frameworks. To accommodate spatially localized events, we also introduce a new variant of CoBRAS to obtain local sensitivity-balanced projections. We demonstrate the utility of this approach to characterize extreme events across a diverse set of challenging systems, including turbulent bursts of energy dissipation in the 2D Kolmogorov Flow, spontaneous synchronization in networks of coupled FitzHugh-Nagumo oscillators, and the localized formation of ocean rogue waves from a modified nonlinear Schrödinger equation. For each example, we show that our simple forecast models accurately predict extreme events and that the underlying mechanisms may be used to design control laws to prevent these events. Finally, we demonstrate that by learning a neural network surrogate model of the dynamics directly from data, we may extend this approach to experimental systems and systems that are not natively written in an automatically differentiable programming language.

Can a model be identified from one experiment? We study analytic systems that are linearly parameterized by a combination of prescribed dictionary terms, such as partial differential operators and dynamical systems. For a single input-response pair, recovery is possible exactly when the evaluated dictionary terms are linearly independent. We prove a sharp zero-one law: either no input uniquely determines the coefficients, or almost every random input sampled from a nondegenerate Gaussian measure does. This dichotomy reduces one-shot system identification to a question about degenerate inputs and provides an a posteriori certificate for any recovered model. Numerical examples recover dynamical systems, nonlinear partial differential equations, and structured matrix families from single trajectory data, while also detecting when an extra probe is necessary.

Symmetry is present throughout nature and continues to play an increasingly central role in physics and machine learning. Fundamental symmetries, such as Poincaré invariance, allow physical laws discovered in laboratories on Earth to be extrapolated to the farthest reaches of the universe. Symmetry is essential to achieving this extrapolatory power in machine learning applications. For example, translation invariance in image classification allows models with fewer parameters, such as convolutional neural networks, to be trained on smaller data sets and achieve state-of-the-art performance. In this paper, we provide a unifying theoretical and methodological framework for incorporating symmetry into machine learning models in three ways: 1. enforcing known symmetry when training a model; 2. discovering unknown symmetries of a given model or data set; and 3. promoting symmetry during training by learning a model that breaks symmetries within a user-specified group of candidates when there is sufficient evidence in the data. We show that these tasks can be cast within a common mathematical framework whose central object is the Lie derivative associated with fiber-linear Lie group actions on vector bundles. We extend and unify several existing results by showing that enforcing and discovering symmetry are linear-algebraic tasks that are dual with respect to the bilinear structure of the Lie derivative. We also propose a novel way to promote symmetry by introducing a class of convex regularization functions based on the Lie derivative and nuclear norm relaxation to penalize symmetry breaking during training of machine learning models. We explain how these ideas can be applied to a wide range of machine learning models including basis function regression, dynamical systems discovery, neural networks, and neural operators acting on fields.

Data-driven models for nonlinear dynamical systems based on approximating the underlying Koopman operator or generator have proven to be successful tools for forecasting, feature learning, state estimation, and control. It has become well known that the Koopman generators for control-affine systems also have affine dependence on the input, leading to convenient finite-dimensional bilinear approximations of the dynamics. Yet there are still two main obstacles that limit the scope of current approaches for approximating the Koopman generators of systems with actuation. First, the performance of existing methods depends heavily on the choice of basis functions over which the Koopman generator is to be approximated; and there is currently no universal way to choose them for systems that are not measure preserving. Secondly, if we do not observe the full state, then it becomes necessary to account for the dependence of the output time series on the sequence of supplied inputs when constructing observables to approximate Koopman operators. To address these issues, we write the dynamics of observables governed by the Koopman generator as a bilinear hidden Markov model, and determine the model parameters using the expectation-maximization (EM) algorithm. The E-step involves a standard Kalman filter and smoother, while the M-step resembles control-affine dynamic mode decomposition for the generator. We demonstrate the performance of this method on three examples, including recovery of a finite-dimensional Koopman-invariant subspace for an actuated system with a slow manifold; estimation of Koopman eigenfunctions for the unforced Duffing equation; and model-predictive control of a fluidic pinball system based only on noisy observations of lift and drag.

There is a mystery at the heart of operator learning: how can one recover a non-self-adjoint operator from data without probing the adjoint? Current practical approaches suggest that one can accurately recover an operator while only using data generated by the forward action of the operator without access to the adjoint. However, naively, it seems essential to sample the action of the adjoint. In this paper, we partially explain this mystery by proving that without querying the adjoint, one can approximate a family of non-self-adjoint infinite-dimensional compact operators via projection onto a Fourier basis. We then apply the result to recovering Green's functions of elliptic partial differential operators and derive an adjoint-free sample complexity bound. While existing theory justifies low sample complexity in operator learning, ours is the first adjoint-free analysis that attempts to close the gap between theory and practice.

The ability to measure differences in collected data is of fundamental importance for quantitative science and machine learning, motivating the establishment of metrics grounded in physical principles. In this study, we focus on the development of such metrics for viscoelastic fluid flows governed by a large class of linear and nonlinear stress models. To do this, we introduce energy-compatible families of kernel functions corresponding to a given viscoelastic stress model. Each kernel implicitly embeds flowfield snapshots into a Reproducing Kernel Hilbert Space (RKHS) in which distances and angles are computed and whose squared norm equals the total mechanical energy. Additionally, we present a solution to the preimage problem for these kernels, enabling accurate reconstruction of flowfields from their RKHS representations. Through numerical experiments on an unsteady viscoelastic lid-driven cavity flow, we demonstrate the utility of energy-compatible kernels for extracting energetically-dominant coherent structures in viscoelastic flows across a range of Reynolds and Weissenberg numbers. Specifically, the features extracted by Kernel Principal Component Analysis (KPCA) of flowfield snapshots using energy-compatible kernel functions yield reconstructions with superior accuracy in terms of mechanical energy compared to conventional methods such as ordinary Principal Component Analysis (PCA) with naively-defined state vectors or KPCA with ad-hoc choices of kernel functions. Our findings underscore the importance of principled choices of metrics in both scientific and machine learning investigations of complex fluid systems.

Data-driven reduced-order models often fail to make accurate forecasts of high -dimensional nonlinear dynamical systems that are sensitive along coordinates with low-variance be-cause such coordinates are often truncated, e.g., by proper orthogonal decomposition, kernel principal component analysis, and autoencoders. Such systems are encountered frequently in shear-dominated fluid flows where nonnormality plays a significant role in the growth of disturbances. In order to address these issues, we employ ideas from active subspaces to find low-dimensional systems of co-ordinates for model reduction that balance adjoint-based information about the system's sensitivity with the variance of states along trajectories. The resulting method, which we refer to as covariance balancing reduction using adjoint snapshots (CoBRAS), is analogous to balanced truncation with state and adjoint-based gradient covariance matrices replacing the system Gramians and obeying the same key transformation laws. Here, the extracted coordinates are associated with an oblique projection that can be used to construct Petrov-Galerkin reduced-order models. We provide an efficient snapshot-based computational method analogous to balanced proper orthogonal decompo-sition. This also leads to the observation that the reduced coordinates can be computed relying on inner products of state and gradient samples alone, allowing us to find rich nonlinear coordinates by replacing the inner product with a kernel function. In these coordinates, reduced-order models can be learned using regression. We demonstrate these techniques and compare to a variety of other methods on a simple, yet challenging three-dimensional system and a nonlinear axisymmetric jet flow simulation with 105 state variables.

Recently developed reduced-order modeling techniques aim to approximate nonlinear dynamical systems on low-dimensional manifolds learned from data. This is an effective approach for modeling dynamics in a post-transient regime where the effects of initial conditions and other disturbances have decayed. However, modeling transient dynamics near an underlying manifold, as needed for real-time control and forecasting applications, is complicated by the effects of fast dynamics and nonnormal sensitivity mechanisms. To begin to address these issues, we introduce a parametric class of nonlinear projections described by constrained autoencoder neural networks in which both the manifold and the projection fibers are learned from data. Our architecture uses invertible activation functions and biorthogonal weight matrices to ensure that the encoder is a left inverse of the decoder. We also introduce new dynamics-aware cost functions that promote learning of oblique projection fibers that account for fast dynamics and nonnormality. To demonstrate these methods and the specific challenges they address, we provide a detailed case study of a three-state model of vortex shedding in the wake of a bluff body immersed in a fluid, which has a two-dimensional slow manifold that can be computed analytically. In anticipation of future applications to high-dimensional systems, we also propose several techniques for constructing computationally efficient reduced-order models using our proposed nonlinear projection framework. This includes a novel sparsity-promoting penalty for the encoder that avoids detrimental weight matrix shrinkage via computation on the Grassmann manifold.

Reduced-order modeling techniques, including balanced truncation and $\mathcal{H}_2$-optimal model reduction, exploit the structure of linear dynamical systems to produce models that accurately capture the dynamics. For nonlinear systems operating far away from equilibria, on the other hand, current approaches seek low-dimensional representations of the state that often neglect low-energy features that have high dynamical significance. For instance, low-energy features are known to play an important role in fluid dynamics where they can be a driving mechanism for shear-layer instabilities. Neglecting these features leads to models with poor predictive accuracy despite being able to accurately encode and decode states. In order to improve predictive accuracy, we propose to optimize the reduced-order model to fit a collection of coarsely sampled trajectories from the original system. In particular, we optimize over the product of two Grassmann manifolds defining Petrov-Galerkin projections of the full-order governing equations. We compare our approach with existing methods including proper orthogonal decomposition, balanced truncation-based Petrov-Galerkin projection, quadratic-bilinear balanced truncation, and the quadratic-bilinear iterative rational Krylov algorithm. Our approach demonstrates significantly improved accuracy both on a nonlinear toy model and on an incompressible (nonlinear) axisymmetric jet flow with $10^5$ states.

Sensor placement and feature selection are critical steps in engineering, modeling, and data science that share a common mathematical theme: the selected measurements should enable solution of an inverse problem. Most real-world systems of interest are nonlinear, yet the majority of available techniques for feature selection and sensor placement rely on assumptions of linearity or simple statistical models. We show that when these assumptions are violated, standard techniques can lead to costly over-sensing without guaranteeing that the desired information can be recovered from the measurements. In order to remedy these problems, we introduce a novel datadriven approach for sensor placement and feature selection for a general type of nonlinear inverse problem based on the information contained in secant vectors between data points. Using the secant-based approach, we develop three efficient greedy algorithms that each provide different types of robust, nearminimal reconstruction guarantees. We demonstrate them on two problems where linear techniques consistently fail: sensor placement to reconstruct a fluid flow formed by a complicated shock-mixing layer interaction and selecting fundamental manifold learning coordinates on a torus.

A common way to represent a system's dynamics is to specify how the state evolves in time. An alternative viewpoint is to specify how functions of the state evolve in time. This evolution of functions is governed by a linear operator called the Koopman operator, whose spectral properties reveal intrinsic features of a system. For instance, its eigenfunctions determine coordinates in which the dynamics evolve linearly. This review discusses the theoretical foundations of Koopman operator methods, as well as numerical methods developed over the past two decades to approximate the Koopman operator from data, for systems both with and without actuation. We pay special attention to ergodic systems, for which especially effective numerical methods are available. For nonlinear systems with an affine control input, the Koopman formalism leads naturally to systems that are bilinear in the state and the input, and this structure can be leveraged for the design of controllers and estimators.

In recent years, the success of the Koopman operator in dynamical systems analysis has also fueled the development of Koopman operator-based control frameworks. In order to preserve the relatively low data requirements for an approximation via Dynamic Mode Decomposition, a quantization approach was recently proposed in [Peitz \& Klus, Automatica 106, 2019]. This way, control of nonlinear dynamical systems can be realized by means of switched systems techniques, using only a finite set of autonomous Koopman operator-based reduced models. These individual systems can be approximated very efficiently from data. The main idea is to transform a control system into a set of autonomous systems for which the optimal switching sequence has to be computed. In this article, we extend these results to continuous control inputs using relaxation. This way, we combine the advantages of the data efficiency of approximating a finite set of autonomous systems with continuous controls. We show that when using the Koopman generator, this relaxation --- realized by linear interpolation between two operators --- does not introduce any error for control affine systems. This allows us to control high-dimensional nonlinear systems using bilinear, low-dimensional surrogate models. The efficiency of the proposed approach is demonstrated using several examples with increasing complexity, from the Duffing oscillator to the chaotic fluidic pinball.

We propose a framework that elucidates the input-output characteristics of flows with complex dynamics arising from nonlinear interactions between different time scales. More specifically, we consider a periodically time-varying base flow, and perform a frequency-domain analysis of periodic perturbations about this base flow; the response of these perturbations is governed by the harmonic resolvent, which is a linear operator similar to the harmonic transfer function introduced by Wereley (1991). This approach makes it possible to explicitly capture the triadic interactions that are responsible for the energy transfer between different time scales in the flow. For instance, perturbations at frequency $\alpha$ are coupled with perturbations at frequency $\omega$ through the base flow at frequency $\omega-\alpha$. We draw a connection with resolvent analsyis, which is a special case of the harmonic resolvent when evaluated about a steady base flow. We show that the left and right singular vectors of the harmonic resolvent are the optimal response and forcing modes, which can be understood as full spatio-temporal signals that reveal space-time amplification characteristics of the flow. We illustrate the method on examples, including a three-dimensional system of ordinary differential equations and the flow over an airfoil at near-stall angle of attack.

Manifold learning techniques seek to discover structure-preserving mappings of high-dimensional data into low-dimensional spaces. While the new sets of coordinates specified by these mappings can closely parameterize the data, they are generally complicated nonlinear functions of the original variables. This makes them difficult to interpret physically. Furthermore, in data-driven model reduction applications the governing equations may have structure that is destroyed by nonlinear mapping into coordinates on an inertial manifold, creating a computational bottleneck for simulations. Instead, we propose to identify a small collection of the original variables which are capable of uniquely determining all others either locally via immersion or globally via embedding of the underlying manifold. When the data lies on a low-dimensional subspace the existing discrete empirical interpolation method (DEIM) accomplishes this with recent variants employing greedy algorithms based on pivoted QR (PQR) factorizations. However, low-dimensional manifolds coming from a variety of applications, particularly from advection-dominated PDEs, do not lie in or near any low-dimensional subspace. Our proposed approach extends DEIM to data lying near nonlinear manifolds by applying a similar pivoted QR procedure simultaneously on collections of patches making up locally linear approximations of the manifold, resulting in a novel simultaneously pivoted QR (SimPQR) algorithm. The immersion provided by SimPQR can be extended to an embedding by applying SimPQR a second time to a modified collection of vectors. The SimPQR method for computing these `nonlinear DEIM' (NLDEIM) coordinates is successfully applied to real-world data lying near an inertial manifold in a cylinder wake flow as well as data coming from a viscous Burgers equation with different initial conditions.

This paper describes a method for learning low-dimensional approximations of nonlinear dynamical systems, based on neural network approximations of the underlying Koopman operator. Extended Dynamic Mode Decomposition (EDMD) provides a useful data-driven approximation of the Koopman operator for analyzing dynamical systems. This paper addresses a fundamental problem associated with EDMD: a trade-off between representational capacity of the dictionary and overfitting due to insufficient data. A new neural network architecture combining an autoencoder with linear recurrent dynamics in the encoded state is used to learn a low-dimensional and highly informative Koopman-invariant subspace of observables. A method is also presented for balanced model reduction of overspecified EDMD systems in feature space. Nonlinear reconstruction using partially linear multikernel regression aims to improve reconstruction accuracy from the low-dimensional state when the data has complex but intrinsically low-dimensional structure. The techniques demonstrate the ability to identify Koopman eigenfunctions of the unforced Duffing equation, create accurate low-dimensional models of an unstable cylinder wake flow, and make short-time predictions of the chaotic Kuramoto-Sivashinsky equation.

Recently, scanning electron microscopy (SEM) has been used for digital image correlation (DIC), as micrographs can be acquired with high magnification, providing improved resolution to quantify strain heterogeneities. However, it has been shown by researchers that SEM images can be problematic due to inherent electromagnetic distortions that are not present in optical images. Drift, spatial distortions, and magnification uncertainties are the main issues that can seriously affect the accuracy of localized strain measurements. The present work focuses on long duration experiments, for which images are taken days or weeks apart. We have proposed a systematic procedure to reduce drift, correct spatial distortion, and account for magnification variations between pairs of acquired images. Additionally, SEM parameters are discussed and chosen to increase the signal-to-noise ratio and improve the accuracy of the DIC measurements. The spatial distortion correction increases the repeatability of the correlated values and the precision of the measurements. The implementation for this type of correction is done by applying the measured distortion gradient of a certified grid onto the DIC strain field. The magnification adjustment increases the reliability of the strain maps, ensuring the measurements are in agreement with the actual strain induced during the experiment. We have presented a systematic protocol for ex-situ DIC experiments within the SEM and some basic cross-check procedures that can be performed to evaluate the reliability of the reference grid and the precision of the final strain map.

A new design method for inward-turning streamline-traced inlets is presented. Resulting designs are intended for low-supersonic low-drag low-boom applications such as that required for NASA's proposed low-boom flight demonstration aircraft. A critical feature of these designs is the internal cowl lip angle that allows for little or no flow turning on the outer nacelle. Present methods using conical-flow Busemann parent flowfields have simply truncated, or otherwise modified, the stream-traced contours to include this internal cowl angle. Such modifications disrupt the parent flowfield, reducing inlet performance and flow uniformity. The method presented herein merges a conical flowfield that includes a leading shock with a truncated Busemann flowfield in a manner that minimizes unwanted interactions. A leading internal cowl angle is now inherent in the parent flowfield, and inlet contours traced from this flowfield retain its high performance and good flow uniformity. Computational fluid dynamics analysis of a candidate inlet design is presented that verifies the design technique, and it reveals a starting issue with the basic geometry. A minor modification to the cowl lip region is shown to eliminate this phenomenon, thereby allowing starting and smooth transition to subcritical operation as backpressure is increased. An inlet critical-point total pressure recovery of 96% is achieved based on computational fluid dynamics results for a Mach 1.7 freestream design. Correction for boundary-layer displacement thickness and sizing for a given engine airflow requirement are also discussed.