Data augmentation is becoming increasingly important across various areas of time series analysis, including forecasting, classification, and anomaly detection. We introduce the Latent Generative Temporal Augmentation (L-GTA) model, a generative approach based on a Variational Autoencoder with a Bi-LSTM backbone and temporal self-attention. The model learns a latent representation for each timestep and applies controlled perturbations such as jittering, magnitude warping, or drift. We define an equivariance objective to further encourage consistency between latent space and data space transformations. As a result, the augmented samples show predictable and interpretable transformation signatures. We evaluate L-GTA on several real-world datasets against SOTA generative methods, including TimeGAN, TimeVAE, and Diffusion-TS, as well as direct transformation approaches. Across experiments on downstream forecasting, distribution fidelity, and controllability of transformation intensity, L-GTA consistently outperforms competing approaches. In downstream forecasting, it reduces prediction error by up to 26

This paper studies the application of ensembles composed of multi-output models for multi-step ahead forecasting problems. Dynamic ensembles have been commonly used for forecasting. However, these are typically designed for one-step-ahead tasks. On the other hand, the literature regarding the application of dynamic ensembles for multi-step ahead forecasting is scarce. Moreover, it is not clear how the combination rule is applied across the forecasting horizon. We carried out extensive experiments to analyze the application of dynamic ensembles for multi-step forecasting. We resorted to a case study with 3568 time series and an ensemble of 30 multi-output models. We discovered that dynamic ensembles based on arbitrating and windowing present the best performance according to average rank. Moreover, as the horizon increases, most approaches struggle to outperform a static ensemble that assigns equal weights to all models. The experiments are publicly available in a repository.

The importance of time series forecasting drives continuous research and the development of new approaches to tackle this problem. Typically, these methods are introduced through empirical studies that frequently claim superior accuracy for the proposed approaches. Nevertheless, concerns are rising about the reliability and generalizability of these results due to limitations in experimental setups. This paper addresses a critical limitation: the number and representativeness of the datasets used. We investigate the impact of dataset selection bias, particularly the practice of cherry-picking datasets, on the performance evaluation of forecasting methods. Through empirical analysis with a diverse set of benchmark datasets, our findings reveal that cherry-picking datasets can significantly distort the perceived performance of methods, often exaggerating their effectiveness. Furthermore, our results demonstrate that by selectively choosing just four datasets — what most studies report — 46% of methods could be deemed best in class, and 77% could rank within the top three. Additionally, recent deep learning-based approaches show high sensitivity to dataset selection, whereas classical methods exhibit greater robustness. Finally, our results indicate that, when empirically validating forecasting algorithms on a subset of the benchmarks, increasing the number of datasets tested from 3 to 6 reduces the risk of incorrectly identifying an algorithm as the best one by approximately 40%. Our study highlights the critical need for comprehensive evaluation frameworks that more accurately reflect real-world scenarios. Adopting such frameworks will ensure the development of robust and reliable forecasting methods.

The microbial rare biosphere, composed of low-abundance microorganisms in a community, lacks a standardized delineation method for its definition. Currently, most studies rely on arbitrary thresholds to define the microbial rare biosphere (e.g., 0.1% relative abundance per sample), hampering comparisons across studies. To address this challenge, we present ulrb (Unsupervised Learning based Definition of the Rare Biosphere), available as an R package. ulrb uses unsupervised machine learning to optimally classify taxa into abundance categories (e.g., rare, intermediate, or abundant) within microbial communities. We show that ulrb is more consistent than threshold-based approaches and can be applied to data derived from common microbial ecology protocols and non-microbial studies. ulrb can be used to identify different types of rarity and is statistically valid for the analysis of various dataset sizes. In conclusion, ulrb discerns rare from abundant organisms in a user-independent manner, finding applicability in selected ecological datasets.

Low-abundance microorganisms, often referred to as the “rare biosphere”, play a crucial role in ecosystem resistance and resilience, but remain challenging to study. One of the main difficulties lies in the lack of an appropriate definition of rare taxa. Most studies use relative abundance thresholds (e.g., 0.1 % relative abundance, per sample) to discern rare from abundant taxa within a microbial community. This is inappropriate because such thresholds are arbitrary and lack biological meaning. To solve this problem, we have proposed the utilization of unsupervised machine learning, through the ulrb (“Unsupervised Learning Definition of the Microbial Rare Biosphere”) algorithm, implemented as an R package (v0.1.8). This algorithm applies the partition around medoids (pam) algorithm to cluster taxa based on their abundance, in a community, for any number of samples. Based on the clusters, ulrb automatically classifies taxa into “rare”, “undetermined” or “abundant”, by default. Ulrb includes functions for all analytical steps necessary to define the rare biosphere. Specifically, we include four groups of functions: 1) process data of the user into the correct format for the ulrb algorithm; 2) cluster taxa into abundance classifications; 3) helper functions to evaluate detailed statistics of the clustering steps; and 4) visualization functions, focused on rank abundance curves and Silhouette scores, for assessment of clustering quality. In addition, ulrb allows the user to change the number of classifications obtained and includes options for detailed reporting. In this article, we describe the ulrb R package architecture, coding organization, and strategy. Furthermore, we use a 16S rRNA gene amplicon sequencing dataset from the Arctic Ocean to provide illustrative examples, with code, on how to use and explore ulrb capabilities. By explaining the architecture and implementation of ulrb, this study allows independent groups to integrate an abundance classification step in their data analysis protocols, instead of relying on taxa labeled by inconsistent or manual strategies.

A subgroup discovery-based method has recently been proposed to understand the behavior of models in the (original) feature space. The subgroups identified represent areas of feature space where the model obtains better or worse predictive performance when compared to the average test performance. For instance, in the marketing domain, the approach extracts subgroups such as: in groups of customers with higher income and who are younger, the random forest achieves higher accuracy than on average. Here, we propose a complementary method, Meta Subspace Analysis (MSA), MSA uses metalearning to analyze these subgroups in the metafeature space. We use association rules to relate metafeatures of the feature space represented by the subgroups to the improvement or degradation of the performance of models. For instance, in the same domain, the approach extracts rules such as: when the class entropy decreases and the mutual information increases in the subgroup data, the random forest achieves lower accuracy. While the subgroups in the original feature space are useful for the end user and the data scientist developing the corresponding model, the meta-level rules provide a domain-independent perspective on the behavior of the model that is suitable for the same data scientist but also for ML researchers, to understand the behavior of algorithms. We illustrate the approach with the results of two well-known algorithms, naive Bayes and random forest, on the Adult dataset. The results confirm some expected behavior of algorithms. However, and most interestingly, some unexpected behaviors are also obtained, requiring additional investigation. In general, the empirical study demonstrates the usefulness of the approach to obtain additional knowledge about the behavior of models.

IntroductionHypoxia is defined as a critically low-oxygen condition of water, which, if prolonged, can be harmful to fish and many other aquatic species. In the context of ocean salmon fish farming, early detection of hypoxia events is critical for farm managers to mitigate these events to reduce fish stress, however in complex natural systems accurate forecasting tools are limited. The goal of this research is to use a machine learning approach to forecast oxygen concentration and predict hypoxia events in marine net-pen salmon farms.MethodsThe developed model is based on gradient boosting and works in two stages. First, we apply auto-regression to build a forecasting model that predicts oxygen concentration levels within a cage. We take a global forecasting approach by building a model using the historical data provided by sensors at several marine fish farms located in eastern Canada. Then, the forecasts are transformed into binary probabilities that indicate the likelihood of a low-oxygen event. We leverage the cumulative distribution function to compute these probabilities.Results and discussionWe tested our model in a case study that included several cages across 14 fish farms. The experiments suggest that the model can detect future hypoxic events with a commercially acceptable false alarm rate. The resulting probabilistic predictions and oxygen concentration forecasts can help salmon farmers to prioritize resources, and reduce harm to crops.

We introduce the Robustness of Hierarchically Organized Time Series (RHiOTS) framework, designed to assess the robustness of hierarchical time series forecasting models and algorithms on real-world datasets. Hierarchical time series, where lower-level forecasts must sum to upper-level ones, are prevalent in various contexts, such as retail sales across countries. Current empirical evaluations of forecasting methods are often limited to a small set of benchmark datasets, offering a narrow view of algorithm behavior. RHiOTS addresses this gap by systematically altering existing datasets and modifying the characteristics of individual series and their interrelations. It uses a set of parameterizable transformations to simulate those changes in the data distribution. Additionally, RHiOTS incorporates an innovative visualization component, turning complex, multidimensional robustness evaluation results into intuitive, easily interpretable visuals. This approach allows an in-depth analysis of algorithm and model behavior under diverse conditions. We illustrate the use of RHiOTS by analyzing the predictive performance of several algorithms. Our findings show that traditional statistical methods are more robust than state-of-the-art deep learning algorithms, except when the transformation effect is highly disruptive. Furthermore, we found no significant differences in the robustness of the algorithms when applying specific reconciliation methods, such as MinT. RHiOTS provides researchers with a comprehensive tool for understanding the nuanced behavior of forecasting algorithms, offering a more reliable basis for selecting the most appropriate method for a given problem.

The aquaculture industry faces significant challenges related to sustainability, productivity, and fish welfare. Key issues include managing environmental conditions, disease, pests, and data integration from various sensors and monitoring systems. The BigFish project aims to address these challenges through advanced analytics and machine learning, focusing on three case studies in Atlantic salmon farms: predicting oxygen levels, reducing sea lice infestations, and improving data interaction and visualization. Predictive models for oxygen levels and sea lice infestation, as well as natural language interfaces for data visualization, demonstrate the potential for improved decision-making and management practices in aquaculture. Early results indicate the effectiveness of these approaches, highlighting the importance of data-driven solutions in enhancing industry sustainability and productivity.

BackgroundIsolating the effects of deterministic variables (e.g., physicochemical conditions) on soil microbial communities from those of neutral processes (e.g., dispersal) remains a major challenge in microbial ecology. In this study, we disturbed soil microbial communities of two McMurdo Dry Valleys of Antarctica exhibiting distinct microbial biogeographic patterns, both devoid of aboveground biota and different in macro- and micro-physicochemical conditions. We modified the availability of water, nitrogen, carbon, copper ions, and sodium chloride salts in a laboratory-based experiment and monitored the microbial communities for up to two months. Our aim was to mimic a likely scenario in the near future, in which similar selective pressures will be applied to both valleys. We hypothesized that, given their unique microbial communities, the two valleys would select for different microbial populations when subjected to the same disturbances.ResultsThe two soil microbial communities, subjected to the same disturbances, did not respond similarly as reflected in both 16S rRNA genes and transcripts. Turnover of the two microbial communities showed a contrasting response to the same environmental disturbances and revealed different potentials for adaptation to change. These results suggest that the heterogeneity between these microbial communities, reflected in their strong biogeographic patterns, was maintained even when subjected to the same selective pressure and that the 'rare biosphere', at least in these samples, were deeply divergent and did not act as a reservoir for microbiota that enabled convergent responses to change in environmental conditions.ConclusionsOur findings strongly support the occurrence of endemic microbial communities that show a structural resilience to environmental disturbances, spanning a wide range of physicochemical conditions. In the highly arid and nutrient-limited environment of the Dry Valleys, these results provide direct evidence of microbial biogeographic patterns that can shape the communities' response in the face of future environmental changes.

Multi-step prediction is a key challenge in univariate forecasting. However, forecasting accuracy decreases as predictions are made further into the future. This is caused by the decreasing predictability and the error propagation along the horizon. In this paper, we propose a novel method called Forecasted Trajectory Neighbors (FTN) for multi-step forecasting with univariate time series. FTN is a meta-learning strategy that can be integrated with any state-of-the-art multi-step forecasting approach. It works by using training observations to correct the errors made during multiple predictions. This is accomplished by retrieving the nearest neighbors of the multi-step forecasts and averaging these for prediction. The motivation is to introduce, in a lightweight manner, a conditional dependent constraint across the forecasting horizons. Such a constraint, not always taken into account by most strategies, can be considered as a sort of regularization element. We carried out extensive experiments using 7795 time series from different application domains. We found that our method improves the performance of several state-of-the-art multi-step forecasting methods. An implementation of the proposed method is publicly available online, and the experiments are reproducible.

Machine learning algorithms have shown several advantages compared to humans, namely in terms of the scale of data that can be analysed, delivering high speed and precision. However, it is not always possible to understand how algorithms work. As a result of the complexity of some algorithms, users started to feel the need to ask for explanations, boosting the relevance of Explainable Artificial Intelligence. This field aims to explain and interpret models with the use of specific analytical methods that usually analyse how their predicted values and/or errors behave. While prediction analysis is widely studied, performance analysis has limitations for regression models. This paper proposes a rule‐based approach, Error Distribution Rules (EDRs), to uncover atypical error regions, while considering multivariate feature interactions without size restrictions. Extracting EDRs is a form of subgroup mining. EDRs are model agnostic and a drill‐down technique to evaluate regression models, which consider multivariate interactions between predictors. EDRs uncover regions of the input space with deviating performance providing an interpretable description of these regions. They can be regarded as a complementary tool to the standard reporting of the expected average predictive performance. Moreover, by providing interpretable descriptions of these specific regions, EDRs allow end users to understand the dangers of using regression tools for some specific cases that fall on these regions, thaṯ is, they improve the accountability of models. The performance of several models from different problems was studied, showing that our proposal allows the analysis of many situations and direct model comparison. In order to facilitate the examination of rules, two visualization tools based on boxplots and density plots were implemented. A network visualization tool is also provided to rapidly check interactions of every feature condition. An additional tool is provided by using a grid of boxplots, where comparison between quartiles of every distribution with a reference is performed. Based on this comparison, an extrapolation of counterfactual examples to regression was also implemented. A set of examples is described, including a setting where regression models performance is compared in detail using EDRs. Specifically, the error difference between two models in a dataset is studied by deriving rules highlighting regions of the input space where model performance difference is unexpected. The application of visual tools is illustrated using EDRs examples derived from public available datasets. Also, case studies illustrating the specialization of subgroups, identification of counter factual subgroups and detecting unanticipated complex models are presented. This paper extends the state of the art by providing a method to derive explanations for model performance instead of explanations for model predictions.

Evaluating predictive models is a crucial task in predictive analytics. This process is especially challenging with time series data where the observations show temporal dependencies. Several studies have analysed how different performance estimation methods compare with each other for approximating the true loss incurred by a given forecasting model. However, these studies do not address how the estimators behave for model selection: the ability to select the best solution among a set of alternatives. We address this issue and compare a set of estimation methods for model selection in time series forecasting tasks. We attempt to answer two main questions: (i) how often is the best possible model selected by the estimators; and (ii) what is the performance loss when it does not. We empirically found that the accuracy of the estimators for selecting the best solution is low, and the overall forecasting performance loss associated with the model selection process ranges from 1.2% to 2.3%. We also discovered that some factors, such as the sample size, are important in the relative performance of the estimators.

The early detection of anomalous events in time series data is essential in many domains of application. In this paper we deal with critical health events, which represent a significant cause of mortality in intensive care units of hospitals. The timely prediction of these events is crucial for mitigating their consequences and improving healthcare. One of the most common approaches to tackle early anomaly detection problems is through standard classification methods. In this paper we propose a novel method that uses a layered learning architecture to address these tasks. One key contribution of our work is the idea of pre-conditional events, which denote arbitrary but computable relaxed versions of the event of interest. We leverage this idea to break the original problem into two hierarchical layers, which we hypothesize are easier to solve. The results suggest that the proposed approach leads to a better performance relative to state of the art approaches for critical health episode prediction.

The Ocean Aware project, led by Innovasea and funded through Canada's Ocean Supercluster, is developing a fish passage observation platform to monitor fish without the use of traditional tags. This will provide an alternative to standard tracking technology, such as acoustic telemetry fish tracking, which are often not appropriate for tracking at-risk fish species protected by legislation. Rather, the observation platform uses a combination of sensors including acoustic devices, visual and active sonar, and optical cameras. This will enable more in-depth scientific research and better support regulatory monitoring of at-risk fish species in fish passages or marine energy sites. Analysis of this data will require a robust and accurate method to automatically detect fish, count fish, and classify them by species in real-time using both sonar and optical cameras. To meet this need, we developed and tested an automated real-time deep learning framework combining state of the art convolutional neural networks and Kalman filters. First, we showed that an adaptation of the widely used YOLO machine learning model can accurately detect and classify eight species of fish from a public high resolution DIDSON imaging sonar dataset captured from the Ocqueoc River in Michigan, USA. Although there has been extensive research in the literature identifying particular fish such as eel vs. non-eel and seal vs. fish, to our knowledge this is the first successful application of deep learning for classifying multiple fish species with high resolution imaging sonar. Second, we integrated the Norfair object tracking framework to track and count fish using a public video dataset captured by optical cameras from the Wells Dam fish ladder on the Columbia River in Washington State, USA. Our results demonstrate that deep learning models can indeed be used to detect, classify species, and track fish using both high resolution imaging sonar and underwater video from a fish ladder. This work is a first step toward developing a fully implemented system which can accurately detect, classify and generate insights about fish in a wide variety of fish passage environments and conditions with data collected from multiple types of sensors.

Concept drift detection is a crucial task in data stream evolving environments. Most of state of the art approaches designed to tackle this problem monitor the loss of predictive models. However, this approach falls short in many real-world scenarios, where the true labels are not readily available to compute the loss. In this context, there is increasing attention to approaches that perform concept drift detection in an unsupervised manner, i.e., without access to the true labels after the model is deployed. We propose a novel approach to unsupervised concept drift detection based on a student-teacher learning paradigm. Essentially, we create an auxiliary model (student) to mimic the primary model's behaviour (teacher). At run-time, our approach is to use the teacher for predicting new instances and monitoring the mimicking loss of the student for concept drift detection. In a set of experiments using 19 data streams, we show that the proposed approach can detect concept drift and present a competitive behaviour relative to the state of the art approaches.

Significant wave height forecasting is a key problem in ocean data analytics. Predicting the significant wave height is crucial for estimating the energy production from waves. Moreover, the timely prediction of large waves is important to ensure the safety of maritime operations, e.g. passage of vessels. We frame the task of predicting extreme values of significant wave height as an exceedance probability forecasting problem. Accordingly, we aim at estimating the probability that the significant wave height will exceed a predefined threshold. This task is usually solved using a probabilistic binary classification model. Instead, we propose a novel approach based on a forecasting model. The method leverages the forecasts for the upcoming observations to estimate the exceedance probability according to the cumulative distribution function. We carried out experiments using data from a buoy placed in the coast of Halifax, Canada. The results suggest that the proposed methodology is better than state-of-the-art approaches for exceedance probability forecasting.

In recent years the number of vessel accidents has increased with the growing number of vessels in the Ocean.A substantial number of these accidents are due to human errors.However, a vessel could avoid some accidents if it knows future locations or trajectories of surrounding vessels.The future location (or trajectory) prediction of vessels is also valuable for maritime traffic management and planning.Existing approaches for predicting the future location are not vessel-specific, but the behavioural pattern of each type of vessel is different.On the other hand, clustering-based approaches are computationally expensive due to computing the similarity measures among vessel trajectories.In this paper, we propose a clustering-based framework that consists of three modules: dataset preparation, route clustering, and location prediction model.The main novelty of this proposal is that the prediction models are built using only trajectories of vessels of the same type, which we hypothesise will increase their accuracy.The other main contribution of our work is the evaluation of different strategies to improve computational efficiency, which can be critical for real-time vessel location prediction.We have used a density-based hierarchical clustering algorithm, HDBSCAN, where two fast implementations of trajectory similarity measures (Hausdorff Distance and Dynamic Time Warping) are evaluated for clustering route patterns.Also, the effect of using a trajectory compression algorithm (Ramer-Douglas-Peucker) to expedite the clustering process is analyzed.The performance of our approach is evaluated using real-world AIS data.We have compared our proposal against two baseline models.Evaluation results confirm the validity of the proposed approach for prediction of the future location of vessels.