Machine learning models can effectively forecast dynamical systems from time-series data, but they typically require large amounts of past data, making forecasting particularly challenging for systems with limited history. To overcome this, we introduce Meta-learning for Tailored Forecasting using Related Time Series (METAFORS), which generalizes knowledge across systems to enable forecasting in data-limited scenarios. By learning from a library of models trained on longer time series from potentially related systems, METAFORS builds and initializes a model tailored to short time-series data from the system of interest. Using a reservoir computing implementation and testing on simulated chaotic systems, we demonstrate that METAFORS can reliably predict both short-term dynamics and long-term statistics without requiring contextual labels. We see this even when test and related systems exhibit substantially different behaviors, highlighting METAFORS' strengths in data-limited scenarios.

This paper explores the potential of a hybrid modeling approach that combines machine learning (ML) with conventional physics-based modeling for weather prediction beyond the medium range. It extends the work of Arcomano et al. (2022), which tested the approach for short- and medium-range weather prediction, and the work of Arcomano et al. (2023), which investigated its potential for climate modeling. The hybrid model used for the forecast experiments of the paper is based on the low-resolution, simplified parameterization atmospheric general circulation model SPEEDY. In addition to the hybridized prognostic variables of SPEEDY, the model has three purely ML-based prognostic variables: the 6h cumulative precipitation, the sea surface temperature, and the heat content of the top 300m deep layer of the ocean (a new addition compared to the model used in Arcomano et al., 2023). The model has skill in predicting the El Nino cycle and its global teleconnections with precipitation for 3-7 months depending on the season. The model captures equatorial variability of the precipitation associated with Kelvin and Rossby waves and MJO. Predictions of the precipitation in the equatorial region have skill for 15 days in the East Pacific and 11.5 days in the West Pacific. Though the model has low spatial resolution, for these tasks it has prediction skill comparable to what has been published for high-resolution, purely physics-based, conventional, operational forecast models.

Recent work has shown that machine learning (ML) models can skillfully forecast the dynamics of unknown chaotic systems. Short-term predictions of the state evolution and long-term predictions of the statistical patterns of the dynamics ("climate") can be produced by employing a feedback loop, whereby the model is trained to predict forward only one time step, then the model output is used as input for multiple time steps. In the absence of mitigating techniques, however, this feedback can result in artificially rapid error growth ("instability"). One established mitigating technique is to add noise to the ML model training input. Based on this technique, we formulate a new penalty term in the loss function for ML models with memory of past inputs that deterministically approximates the effect of many small, independent noise realizations added to the model input during training. We refer to this penalty and the resulting regularization as Linearized Multi-Noise Training (LMNT). We systematically examine the effect of LMNT, input noise, and other established regularization techniques in a case study using reservoir computing, a machine learning method using recurrent neural networks, to predict the spatiotemporal chaotic Kuramoto-Sivashinsky equation. We find that reservoir computers trained with noise or with LMNT produce climate predictions that appear to be indefinitely stable and have a climate very similar to the true system, while the short-term forecasts are substantially more accurate than those trained with other regularization techniques. Finally, we show the deterministic aspect of our LMNT regularization facilitates fast reservoir computer regularization hyperparameter tuning.

The ability of machine learning (ML) models to "extrapolate" to situations outside of the range spanned by their training data is crucial for predicting the long-term behavior of non-stationary dynamical systems (e.g., prediction of terrestrial climate change), since the future trajectories of such systems may (perhaps after crossing a tipping point) explore regions of state space which were not explored in past time-series measurements used as training data. We investigate the extent to which ML methods can yield useful results by extrapolation of such training data in the task of forecasting non-stationary dynamics, as well as conditions under which such methods fail. In general, we find that ML can be surprisingly effective even in situations that might appear to be extremely challenging, but do (as one would expect) fail when "too much" extrapolation is required. For the latter case, we show that good results can potentially be obtained by combining the ML approach with an available inaccurate conventional model based on scientific knowledge.

It is shown that a recently developed hybrid modeling approach that combines machine learning (ML) with an atmospheric global circulation model (AGCM) can serve as a basis for capturing atmospheric processes not captured by the AGCM. This power of the approach is illustrated by three examples from a decades‐long climate simulation experiment. The first example demonstrates that the hybrid model can produce sudden stratospheric warming, a dynamical process of nature not resolved by the low resolution AGCM component of the hybrid model. The second and third example show that introducing 6‐hr cumulative precipitation and sea surface temperature (SST) as ML‐based prognostic variables improves the precipitation climatology and leads to a realistic ENSO signal in the SST and atmospheric surface pressure.

Network link inference from measured time series data of the behavior of dynamically interacting network nodes is an important problem with wide-ranging applications, e.g., estimating synaptic connectivity among neurons from measurements of their calcium fluorescence. Network inference methods typically begin by using the measured time series to assign to any given ordered pair of nodes a numerical score reflecting the likelihood of a directed link between those two nodes. In typical cases, the measured time series data may be subject to limitations, including limited duration, low sampling rate, observational noise, and partial nodal state measurement. However, it is unknown how the performance of link inference techniques on such datasets depends on these experimental limitations of data acquisition. Here, we utilize both synthetic data generated from coupled chaotic systems as well as experimental data obtained from Caenorhabditis elegans neural activity to systematically assess the influence of data limitations on the character of scores reflecting the likelihood of a directed link between a given node pair. We do this for three network inference techniques: Granger causality, transfer entropy, and, a machine learning-based method. Furthermore, we assess the ability of appropriate surrogate data to determine statistical confidence levels associated with the results of link-inference techniques.

Wave energy distribution within enclosures with irregular boundaries is a common phenomenon in many branches of electromagnetics. If the wavelength of the injected wave is small compared with the structure size, the scattering properties of the enclosure will be extremely sensitive to small changes in geometry or wave frequency. In this case, statistical models are sought. The random coupling model (RCM) is one such model that has been explored through experiments and theory. Previous studies were conducted by injecting waves into high Q cavities in a nearly omnidirectional manner. In this article, a directed beam approach is taken, and relatively low Q cavities are considered. The goal is to determine when the so-called “random plane wave hypothesis,” a fundamental basis of the RCM formulation, breaks down. Results show that injecting such directed beams leads to large deviations in the wave statistics for single realizations of the enclosure geometry. The expected statistics are restored to some degree when multiple realizations are considered.

This paper describes an implementation of the combined hybrid‐parallel prediction (CHyPP) approach of Wikner et al. (2020), https://doi.org/10.1063/5.0005541 on a low‐resolution atmospheric global circulation model (AGCM). The CHyPP approach combines a physics‐based numerical model of a dynamical system (e.g., the atmosphere) with a computationally efficient type of machine learning (ML) called reservoir computing to construct a hybrid model. This hybrid atmospheric model produces more accurate forecasts of most atmospheric state variables than the host AGCM for the first 7–8 forecast days, and for even longer times for the temperature and humidity near the earth's surface. It also produces more accurate forecasts than a model based only on ML, or a model that combines linear regression, rather than ML, with the AGCM. The potential of the CHyPP approach for climate research is demonstrated by a 10‐year long hybrid model simulation of the atmospheric general circulation, which shows that the hybrid model can simulate the general circulation with substantially smaller systematic errors and more realistic variability than the host AGCM.

Recent work has shown that machine learning (ML) models can be trained to accurately forecast the dynamics of unknown chaotic dynamical systems. Short-term predictions of the state evolution and long-term predictions of the statistical patterns of the dynamics (``climate'') can be produced by employing a feedback loop, whereby the model is trained to predict forward one time step, then the model output is used as input for multiple time steps. In the absence of mitigating techniques, however, this technique can result in artificially rapid error growth. In this article, we systematically examine the technique of adding noise to the ML model input during training to promote stability and improve prediction accuracy. Furthermore, we introduce Linearized Multi-Noise Training (LMNT), a regularization technique that deterministically approximates the effect of many small, independent noise realizations added to the model input during training. Our case study uses reservoir computing, a machine-learning method using recurrent neural networks, to predict the spatiotemporal chaotic Kuramoto-Sivashinsky equation. We find that reservoir computers trained with noise or with LMNT produce climate predictions that appear to be indefinitely stable and have a climate very similar to the true system, while reservoir computers trained without regularization are unstable. Compared with other regularization techniques that yield stability in some cases, we find that both short-term and climate predictions from reservoir computers trained with noise or with LMNT are substantially more accurate. Finally, we show that the deterministic aspect of our LMNT regularization facilitates fast hyperparameter tuning when compared to training with noise.

Machine learning (ML) has found widespread application over a broad range of important tasks. To enhance ML performance, researchers have investigated computational architectures whose physical implementations promise compactness, high-speed execution, physical robustness, and low-energy cost. Here, we experimentally demonstrate an approach that uses the high sensitivity of reverberant short-wavelength waves for physical realization and enhancement of computational power of a type of ML known as reservoir computing (RC). The potential computation power of RC systems increases with their effective size. We here exploit the intrinsic property of short-wavelength reverberant wave sensitivity to perturbations to expand the effective size of the RC system by means of spatial and spectral perturbations. Working in the microwave regime, this scheme is tested experimentally on different ML tasks. Our results indicate the general applicability of reverberant wave based implementations of RC and of our effective reservoir size expansion techniques.

Forecasting the dynamics of large, complex, sparse networks from previous time series data is important in a wide range of contexts. Here we present a machine learning scheme for this task using a parallel architecture that mimics the topology of the network of interest. We demonstrate the utility and scalability of our method implemented using reservoir computing on a chaotic network of oscillators. Two levels of prior knowledge are considered: (i) the network links are known, and (ii) the network links are unknown and inferred via a data-driven approach to approximately optimize prediction.

Benjamin W. Frazier, 2, 3, ∗ Thomas M. Antonsen, Jr., 2, 4 Steven M. Anlage, 4, 5 and Edward Ott 2, 4 Institute for Research in Electronics and Applied Physics, University of Maryland, College Park, MD 20742, USA Department of Electrical and Computer Engineering, University of Maryland, College Park, MD 20742, USA Johns Hopkins University Applied Physics Laboratory, Laurel, MD 20723, USA Department of Physics, University of Maryland, College Park, MD 20742, USA Quantum Materials Center, University of Maryland, College Park, MD 20742, USA (Dated: August 17, 2021)

We consider the problem of data-assisted forecasting of chaotic dynamical systems when the available data are in the form of noisy partial measurements of the past and present state of the dynamical system. Recently, there have been several promising data-driven approaches to forecasting of chaotic dynamical systems using machine learning. Particularly promising among these are hybrid approaches that combine machine learning with a knowledge-based model, where a machine-learning technique is used to correct the imperfections in the knowledge-based model. Such imperfections may be due to incomplete understanding and/or limited resolution of the physical processes in the underlying dynamical system, e.g., the atmosphere or the ocean. Previously proposed data-driven forecasting approaches tend to require, for training, measurements of all the variables that are intended to be forecast. We describe a way to relax this assumption by combining data assimilation with machine learning. We demonstrate this technique using the Ensemble Transform Kalman Filter to assimilate synthetic data for the three-variable Lorenz 1963 system and for the Kuramoto-Sivashinsky system, simulating a model error in each case by a misspecified parameter value. We show that by using partial measurements of the state of the dynamical system, we can train a machine-learning model to improve predictions made by an imperfect knowledge-based model.

We propose a novel hybrid modeling approach that combines a physics-based, numerical model of the Earth system with a machine learning model. We also envision to integrate the hybrid model training with a data assimilation cycle, so that new observations update both the state estimate used as initial conditions and the machine learning model parameters.

Electromagnetic environments are becoming increasingly complex and congested, creating a growing challenge for systems that rely on electromagnetic waves for communication, sensing, or imaging. The use of intelligent, reconfigurable metasurfaces provides a potential means for achieving a radio environment that is capable of directing propagating waves to optimize wireless channels on-demand, ensuring reliable operation and protecting sensitive electronic components. Here we introduce wavefront shaping, a technique that combines a deep learning network with a binary programmable metasurface to shape waves in complex electromagnetic environments and to drive the system towards a desired scattering response. We applied this technique to accurately determine metasurface configurations based on measured system scattering responses in a chaotic microwave cavity. The state of the metasurface that realizes desired electromagnetic wave field distribution properties was successfully determined even in cases previously unseen by the deep learning algorithm. Our work represents an important step towards realizing intelligent reconfigurable metasurfaces for smart radio environments that can ensure both the integrity of electronic systems and optimum performance of wireless networks.

We develop and test machine learning techniques for successfully using past state time series data and knowledge of a time-dependent system parameter to predict the evolution of the “climate” associated with the long-term behavior of a non-stationary dynamical system, where the non-stationary dynamical system is itself unknown. By the term climate, we mean the statistical properties of orbits rather than their precise trajectories in time. By the term non-stationary, we refer to systems that are, themselves, varying with time. We show that our methods perform well on test systems predicting both continuous gradual climate evolution as well as relatively sudden climate changes (which we refer to as “regime transitions”). We consider not only noiseless (i.e., deterministic) non-stationary dynamical systems, but also climate prediction for non-stationary dynamical systems subject to stochastic forcing (i.e., dynamical noise), and we develop a method for handling this latter case. The main conclusion of this paper is that machine learning has great promise as a new and highly effective approach to accomplishing data driven prediction of non-stationary systems.

Benjamin W. Frazier, 2, 3, ∗ Thomas M. Antonsen, Jr., 2, 4 Steven M. Anlage, 4, 5 and Edward Ott 2, 4 Institute for Research in Electronics and Applied Physics, University of Maryland, College Park, MD 20742, USA Department of Electrical and Computer Engineering, University of Maryland, College Park, MD 20742, USA Johns Hopkins University Applied Physics Laboratory, Laurel, MD 20723, USA Department of Physics, University of Maryland, College Park, MD 20742, USA Quantum Materials Center, University of Maryland, College Park, MD 20742, USA (Dated: July 13, 2021)