With the advent of the Internet of Things, it is increasingly common to have large networks of sensors, where each sensor may collect different types of data, has limited local computing resources and the ability to transmit data to a central cloud. Detecting events that trigger changes in sensor data properties is a key concern. However, minimizing sensor-to-cloud communication might be necessary due either to privacy constraints or limited battery resources. To detect changes within such a network, we introduce a new method, mixFOCuS, which can detect changes in mixed-type data, where data from different sensors follow different, possibly non-Gaussian, distributions. This methods builds on the FOCuS algorithms, which are recent improvements of the classic approach of Page (1954), extending the approach to streaming data setting for distributed sensor networks. Our method does not require assuming known pre- and post-change parameters, yet is still efficient in both computation and communication, and suitable for detecting changes in real-time. We show the trade-off between reduced transmission frequency and detection power. Simulation results indicate improved power for mixed-type data and better performance than the existing works on Gaussian data.

Understanding historical datasets, such as the England and Wales infant mortality data, for local government districts can provide valuable insights into our changing society. Such analyses can prove challenging in practice, due to frequent changes in the boundaries of local government districts for which records are collected. One solution adopted in the literature to overcome such practical challenges is to pre-process data using areal interpolation to render the units consistent over the time period of focus. However, such methods are prone to errors. In this paper we introduce a novel changepoint method to detect instances where interpolation performs poorly. We demonstrate the utility of our method on original data, and also demonstrate how correcting interpolation errors can affect the clustering of the infant mortality curves.

Detecting anomalies in time series data is a challenging task with broad relevance in many applications. Existing methods work effectively only under idealized conditions, typically focusing on point anomalies or assuming a constant baseline. Our approach overcomes these limitations by detecting both collective and point anomalies, while allowing for polynomial trends and seasonal patterns. We establish statistical theory demonstrating that our method accurately decomposes the time series into anomaly, trend, seasonality, and remainder components. We further show that the approach provides a consistent estimate of the number of anomalies and their locations. Simulation studies confirm its strong detection performance with finite samples, and an application to energy price data illustrates its practical utility.

The rapid advancements of scalable methodologies have opened new avenues for analyzing complex spatio-temporal data, which is crucial in understanding dynamic environmental phenomena. This paper introduces a likelihood-based methodology for detecting abrupt changes in time in spatio-temporal processes, a field where traditional time series methods fall short. Unlike recent approaches, we do not make the unrealistic assumption that data is independent across changepoints. Instead, we use a recently proposed family of covariance models that allows nonstationarity in time, and we propose a Markov approximation to reduce the computational burden of calculating likelihoods under this model. We apply our method to two years of daily wind speed data from various synoptic weather stations in Ireland, identifying a significant changepoint on July 24, 2021, which aligns with a major shift in weather patterns. This application not only demonstrates the method’s utility in handling spatio-temporal datasets but also showcases its potential in broader environmental and climatic studies, offering a scalable solution for analyzing changing patterns in spatial data over time.

Online changepoint detection aims to detect anomalies and changes in real-time in high-frequency data streams, sometimes with limited available computational resources. This is an important task that is rooted in many real-world applications, including and not limited to cybersecurity, medicine and astrophysics. While fast and efficient online algorithms have been recently introduced, these rely on parametric assumptions which are often violated in practical applications. Motivated by data streams from the telecommunications sector, we build a flexible nonparametric approach to detect a change in the distribution of a sequence. Our procedure, NP-FOCuS, builds a sequential likelihood ratio test for a change in a set of points of the empirical cumulative density function of our data. This is achieved by keeping track of the number of observations above or below those points. Thanks to functional pruning ideas, NP-FOCuS has a computational cost that is log-linear in the number of observations and is suitable for high-frequency data streams. In terms of detection power, NP-FOCuS is seen to outperform current nonparametric online changepoint techniques in a variety of settings. We demonstrate the utility of the procedure on both simulated and real data.

Motivated by an example arising from digital networks, we propose a novel approach for detecting the emergence of anomalies in functional data. In contrast to classical functional data approaches, which detect anomalies in completely observed curves, the proposed approach seeks to identify anomalies sequentially as each point on the curve is received. The new method, the Functional Anomaly Sequential Test (FAST), captures the common profile of the curves using Principal Differential Analysis and uses a form of CUSUM test to monitor a new functional observation as it emerges. Various theoretical properties of the procedure are derived. The performance of FAST is then assessed on both simulated and telecommunications data.

We describe how a novel online change-point detection algorithm, called Poisson-FOCuS, can be used to optimally detect gamma-ray bursts within the computational constraints imposed by miniaturized satellites such as the upcoming HERMES-Pathfinder constellation. Poisson-FOCuS enables testing for gamma-ray burst onset at all intervals in a count time series, across all timescales and offsets, in real time and at a fraction of the computational cost of conventional strategies. We validate an implementation with automatic background assessment through exponential smoothing, using archival data from Fermi-GBM. Through simulations of lightcurves modeled after real short and long gamma-ray bursts, we demonstrate that the same implementation has higher detection power than algorithms designed to emulate the logic of Fermi-GBM and Compton-BATSE, reaching the performance of a brute-force benchmark with oracle information on the true background rate, when not hindered by automatic background assessment. Finally, using simulated data with different lengths and means, we show that Poisson-FOCuS can analyze data twice as fast as a similarly implemented benchmark emulator for the historic Fermi-GBM on-board trigger algorithms.

Vector autoregressive (VAR) models are popularly adopted for modelling high-dimensional time series, and their piecewise extensions allow for structural changes in the data. In VAR modelling, the number of parameters grow quadratically with the dimensionality which necessitates the sparsity assumption in high dimensions. However, it is debatable whether such an assumption is adequate for handling datasets exhibiting strong serial and cross-sectional correlations. We propose a piecewise stationary time series model that simultaneously allows for strong correlations as well as structural changes, where pervasive serial and cross-sectional correlations are accounted for by a time-varying factor structure, and any remaining idiosyncratic dependence between the variables is handled by a piecewise stationary VAR model. We propose an accompanying two-stage data segmentation methodology which fully addresses the challenges arising from the latency of the component processes. Its consistency in estimating both the total number and the locations of the change points in the latent components, is established under conditions considerably more general than those in the existing literature. We demonstrate the competitive performance of the proposed methodology on simulated datasets and an application to US blue chip stocks data.

One of the contemporary challenges in anomaly detection is the ability to detect, and differentiate between, both point and collective anomalies within a data sequence or time series. The anomaly package has been developed to provide users with a choice of anomaly detection methods and, in particular, provides an implementation of the recently proposed Collective And Point Anomaly family of anomaly detection algorithms. This article describes the methods implemented whilst also highlighting their application to simulated data as well as real data examples contained in the package.

Abstract Accurate forecasting of the UK gross value added (GVA) is fundamental for measuring the growth of the UK economy. A common nonstationarity in GVA data, such as the ABML series, is its increase in variance over time due to inflation. Transformed or inflation-adjusted series can still be challenging for classical stationarity-assuming forecasters. We adopt a different approach that works directly with the GVA series by advancing recent forecasting methods for locally stationary time series. Our approach results in more accurate and reliable forecasts, and continues to work well even when the ABML series becomes highly variable during the COVID pandemic.

Accurate forecasting of the U.K. gross value added (GVA) is fundamental for measuring the growth of the U.K. economy. A common nonstationarity in GVA data, such as the ABML series, is its increase in variance over time due to inflation. Transformed or inflation-adjusted series can still be challenging for classical stationarity-assuming forecasters. We adopt a different approach that works directly with the GVA series by advancing recent forecasting methods for locally stationary time series. Our approach results in more accurate and reliable forecasts, and continues to work well even when the ABML series becomes highly variable during the COVID pandemic.

We consider the challenge of efficiently detecting changes within a network of sensors, where we also need to minimise communication between sensors and the cloud. We propose an online, communication-efficient method to detect such changes. The procedure works by performing likelihood ratio tests at each time point, and two thresholds are chosen to filter unimportant test statistics and make decisions based on the aggregated test statistics respectively. We provide asymptotic theory concerning consistency and the asymptotic distribution if there are no changes. Simulation results suggest that our method can achieve similar performance to the idealised setting, where we have no constraints on communication between sensors, but substantially reduce the transmission costs.

Online changepoint detection algorithms that are based on (generalised) likelihood-ratio tests have been shown to have excellent statistical properties. However, a simple online implementation is computationally infeasible as, at time T, it involves considering O(T) possible locations for the change. Recently, the FOCuS algorithm has been introduced for detecting changes in mean in Gaussian data that decreases the per-iteration cost to O(log T) . This is possible by using pruning ideas, which reduce the set of changepoint locations that need to be considered at time T to approximately log T . We show that if one wishes to perform the likelihood ratio test for a different one-parameter exponential family model, then exactly the same pruning rule can be used, and again one need only consider approximately log T locations at iteration T. Furthermore, we show how we can adaptively perform the maximisation step of the algorithm so that we need only maximise the test statistic over a small subset of these possible locations. Empirical results show that the resulting online algorithm, which can detect changes under a wide range of models, has a constant-per-iteration cost on average.

Different environmental variables are often monitored using different sampling rates; examples include half-hourly weather station measurements, daily CO2 data, and six-day satellite data. Further when researchers want to combine the data into a single analysis this often requires data aggregation or down-scaling. When one is seeking to identify changes within multivariate data, the aggregation and/or down-scaling processes obscure the changes we seek. In this article, we propose a novel changepoint detection algorithm which can analyze multiple time series for co-occurring changepoints with potentially different sampling rates, without requiring preprocessing to a standard sampling scale. We demonstrate the algorithm on synthetic data before providing an example identifying simultaneous changes in multiple variables at a location on the Greenland ice sheet using synthetic aperture radar and weather station data.

Motivated by a telecommunications application where there are few computational constraints, a novel nonparametric algorithm, NUNC, is introduced to perform an online detection for changes in the distribution of data. Two variants are considered: the first, NUNC Local, detects changes within a sliding window. Conversely, NUNC Global, compares the current window of data to all of the historic information seen so far and makes use of an efficient update step so that this historic information does not need to be stored. To explore the properties of both algorithms, both real and simulated datasets are analysed. Furthermore, a theoretical result for the choice of test threshold to control the false alarm rate is presented, a result that could be applied in other binary segmentation change detection settings.

Many modern applications of online changepoint detection require the ability to process high-frequency observations, sometimes with limited available computational resources. Online algorithms for detecting a change in mean often involve using a moving window, or specifying the expected size of change. Such choices affect which changes the algorithms have most power to detect. We introduce an algorithm, Functional Online CuSUM (FOCuS), which is equivalent to running these earlier methods simultaneously for all sizes of window, or all possible values for the size of change. Our theoretical results give tight bounds on the expected computational cost per iteration of FOCuS, with this being logarithmic in the number of observations. We show how FOCuS can be applied to a number of different change in mean scenarios, and demonstrate its practical utility through its state-of-the art performance at detecting anomalous behaviour in computer server data.

This paper proposes a new method to detect changepoints in the location and scale of univariate data sequences. The proposed method assumes that the data belong to the location‐scale family of distributions and estimate the associated densities nonparametrically. Specifically, the approach does not require knowledge of the functional form of the distribution of the data sequence. As such, the approach can detect changepoints in many distributions. We also propose a new method to detect changes in the location of multivariate sequences, using the marginals and a copula to capture the dependence between variables without the influence of marginal distributions. The performance of the proposed semiparametric approach is contrasted against both other competing nonparametric and Gaussian methods, via simulation studies, as well as applications arising from health and finance.

There is increasing interest in detecting collective anomalies: potentially short periods of time where the features of data change before reverting back to normal behaviour. We propose a new method for detecting a collective anomaly in VAR models. Our focus is on situations where the change in the VAR coefficient matrix at an anomaly is sparse, i.e. a small number of entries of the VAR coefficient matrix change. To tackle this problem, we propose a test statistic for a local segment that is built on the lasso estimator of the change in model parameters. This enables us to detect a sparse change more efficiently and our lasso-based approach becomes especially advantageous when the anomalous interval is short. We show that the new procedure controls Type 1 error and has asymptotic power tending to one. The practicality of our approach is demonstrated through simulations and two data examples, involving New York taxi trip data and EEG data.

Gamma-ray bursts are flashes of light from distant exploding stars. Cube satellites that monitor photons across different energy bands are used to detect these bursts. There is a need for computationally efficient algorithms, able to run using the limited computational resource onboard a cube satellite, that can detect when gamma-ray bursts occur. Current algorithms are based on monitoring photon counts across a grid of different sizes of time window. We propose a new algorithm, which extends the recently developed FOCuS algorithm for online change detection to Poisson data. Our algorithm is mathematically equivalent to searching over all possible window sizes, but at half the computational cost of the current grid-based methods. We demonstrate the additional power of our approach using simulations and data drawn from the Fermi gamma-ray burst catalogue.

A common approach to detect multiple changepoints is to min-imise a measure of data fit plus a penalty that is linear in the number of changepoints. This paper shows that the general finite sample behaviour of such a method can be related to its behaviour when analysing data with either none or one changepoint. This property results in simpler conditions for verifying whether the method will consistently estimate the number and locations of the changepoints. We apply and demonstrate the useful-ness of these simple conditions for a range of changepoint problems. Our new results include a weaker requirement on the choice of penalty to have consistency in a change-in-slope model; and the first results for the accuracy of recently-proposed methods for detecting spikes.