Logistic regression is a ubiquitous method for probabilistic classification. However, the effectiveness of logistic regression depends upon careful and relatively computationally expensive tuning, especially for the regularisation hyperparameter, and especially in the context of high-dimensional data. We present a prevalidated ridge regression model that closely matches logistic regression in terms of classification error and log-loss, particularly for high-dimensional data, while being significantly more computationally efficient and having effectively no hyperparameters beyond regularisation. We scale the coefficients of the model so as to minimise log-loss for a set of prevalidated predictions derived from the estimated leave-one-out cross-validation error. This exploits quantities already computed in the course of fitting the ridge regression model in order to find the scaling parameter with nominal additional computational expense.

Bayesian network classifiers (BNCs) possess a number of properties desirable for a modern classifier: They are easily interpretable, highly scalable, and offer adaptable complexity. However, traditional methods for learning BNCs have historically underperformed when compared to leading classification methods such as random forests. Recent parameter smoothing techniques using hierarchical Dirichlet processes (HDPs) have enabled BNCs to achieve performance competitive with random forests on categorical data, but these techniques are relatively inflexible, and require a complicated, specialized sampling process. In this paper, we introduce a novel method for parameter estimation that uses a log-linear regression to approximate the behaviour of HDPs. As a linear model, our method is remarkably flexible and simple to interpret, and can leverage the vast literature on learning linear models. Our experiments show that our method can outperform HDP smoothing while being orders of magnitude faster, remaining competitive with random forests on categorical data.

Enzyme turnover numbers (Kcat) are fundamental kinetic constants that quantify enzymatic efficiency. Systematic studies of Kcat are essential for characterizing the mechanisms underlying proteomic composition and cellular metabolism. However, experimental measurements of Kcat remain limited and prone to noise. To address this, we present KcatNet, a geometric deep learning model designed for high-throughput prediction of Kcat in metabolic enzymes across all organisms, leveraging paired enzyme sequence and substrate representations. KcatNet consistently outperforms existing predictors, particularly for enzymes with high catalytic efficiency, and demonstrates strong generalization to enzymes that are dissimilar to those in the training set. The model generates interpretable importance scores for enzyme residues involved in catalysis, thereby enabling a quantitative assessment of their contributions to catalytic activity. Furthermore, KcatNet uncovers structural mechanisms and interaction patterns within enzyme-substrate complexes, providing insights into architectural principles that are inaccessible with existing methods. We applied KcatNet to genome-scale Kcat prediction across diverse yeast species, improving proteome allocation predictions by integrating its outputs into metabolic models. Experimental validation further confirmed the model's ability to identify enzyme mutants with enhanced activity. By bridging the gap between sequence, structure, and function, KcatNet establishes a robust foundation for advancing understanding of molecular-level mechanisms and accelerating enzyme engineering efforts. ### Competing Interest Statement The authors have declared no competing interest.

Large language models (LLMs) are a form of artificial intelligence system encapsulating vast knowledge in the form of natural language. These systems are adept at numerous complex tasks including creative writing, storytelling, translation, question-answering, summarization and computer code generation. Although LLMs have seen initial applications in natural sciences, their potential for driving scientific discovery remains largely unexplored. In this work, we introduce LLM4SD, a framework designed to harness LLMs for driving scientific discovery in molecular property prediction by synthesizing knowledge from literature and inferring knowledge from scientific data. LLMs synthesize knowledge by extracting established information from scientific literature, such as molecular weight being key to predicting solubility. For inference, LLMs identify patterns in molecular data, particularly in Simplified Molecular Input Line Entry System-encoded structures, such as halogen-containing molecules being more likely to cross the blood–brain barrier. This information is presented as interpretable knowledge, enabling the transformation of molecules into feature vectors. By using these features with interpretable models such as random forest, LLM4SD can outperform the current state of the art across a range of benchmark tasks for predicting molecular properties. We foresee it providing interpretable and potentially new insights, aiding scientific discovery in molecular property prediction. Zheng et al. developed LLM4SD, a framework using large language models to predict molecular properties. The method leverages the ability of large language models to synthesize knowledge from literature and to reason about scientific data with domain expertise.

Synthetic anomaly injection is a recent and promising approach for time series anomaly detection (TSAD), but existing methods rely on ad hoc, hand-crafted strategies applied to raw time series that fail to capture diverse and complex anomalous patterns, particularly in multivariate settings. We propose a synthetic anomaly generation method named Generator for Instantiating Anomalies in Time Series (GenIAS), which generates realistic and diverse anomalies via a novel learnable perturbation in the latent space of a variational autoencoder. This enables abnormal patterns to be injected across different temporal segments at varying scales based on variational reparameterization. To generate anomalies that align with normal patterns while remaining distinguishable, we introduce a learning strategy that jointly learns the perturbation scale and compact latent representations via a tunable prior, which improves the distinguishability of generated anomalies, as supported by our theoretical analysis. Extensive experiments show that GenIAS produces more diverse and realistic anomalies, and that detection models trained with these anomalies outperform 17 baseline methods on 9 popular TSAD benchmarks.

Time series classification (TSC) is a challenging task due to the diversity of types of features that may be relevant for different classification tasks, including trends, variance, frequency, magnitude, and various patterns. To address this challenge, several alternative classes of approach have been developed. While kernel, neural network, and hybrid approaches perform well overall, some specialized approaches are better suited for specific tasks. In this paper, we propose a new similarity-based classifier, Proximity Forest version 2.0 (PF 2.0), which outperforms previous state-of-the-art similarity-based classifiers across the UCR benchmark and outperforms other state-of-the-art methods on specific datasets in the benchmark that are best addressed by similarity-base methods. PF 2.0 incorporates three recent advances in time series similarity measures — (1) computationally efficient early abandoning and pruning to speedup elastic similarity computations; (2) a new elastic similarity measure, Amerced Dynamic Time Warping (ADTW); and (3) cost function tuning. It rationalizes the set of similarity measures employed, reducing the eight base measures of the original PF to four and using the first derivative transform with all similarity measures, rather than a limited subset. It also incorporates HYDRA, a dictionary-based transform. We have re-implemented PF 1.0 and implemented PF 2.0 framework in Java, making the PF framework more efficient.

Small additive ensembles of symbolic rules offer interpretable prediction models. Traditionally, these ensembles use rule conditions based on conjunctions of simple threshold propositions $x \geq t$ on a single input variable $x$ and threshold $t$, resulting geometrically in axis-parallel polytopes as decision regions. While this form ensures a high degree of interpretability for individual rules and can be learned efficiently using the gradient boosting approach, it relies on having access to a curated set of expressive and ideally independent input features so that a small ensemble of axis-parallel regions can describe the target variable well. Absent such features, reaching sufficient accuracy requires increasing the number and complexity of individual rules, which diminishes the interpretability of the model. Here, we extend classical rule ensembles by introducing logical propositions with learnable sparse linear transformations of input variables, i.e., propositions of the form $\mathbf{x}^\mathrm{T}\mathbf{w} \geq t$, where $\mathbf{w}$ is a learnable sparse weight vector, enabling decision regions as general polytopes with oblique faces. We propose a learning method using sequential greedy optimization based on an iteratively reweighted formulation of logistic regression. Experimental results demonstrate that the proposed method efficiently constructs rule ensembles with the same test risk as state-of-the-art methods while significantly reducing model complexity across ten benchmark datasets.

Bayesian network classifiers provide a feasible solution to tabular data classification, with a number of merits like high time and memory efficiency, and great explainability. However, due to the parameter explosion and data sparsity issues, Bayesian network classifiers are restricted to low-order feature dependency modeling, making them struggle in extrapolating the occurrence probabilities of complex real-world data. In this paper, we propose a novel paradigm to design high-order Bayesian network classifiers, by learning distributional representations for feature values, as what has been done in word embedding and graph representation learning. The learned distributional representations are encoded with the semantic relatedness between different features through their observed co-occurrence patterns in training data, which then serve as a hallmark to extrapolate the occurrence probabilities of new test samples. As a classifier design realization, we remake the K-dependence Bayesian classifier (KDB) by extending it into a neural version, i.e., NeuralKDB, where a novel neural network architecture is designed to learn distributional representations of feature values and parameterize the conditional probabilities between interdependent features. A stochastic gradient descent based algorithm is designed to train the NeuralKDB model efficiently. Extensive classification experiments on 60 UCI datasets demonstrate that the proposed NeuralKDB classifier excels in capturing high-order feature dependencies and significantly outperforms the conventional Bayesian network classifiers, as well as other competitive classifiers, including two neural network based classifiers without distributional representation learning.

One main challenge in time series anomaly detection (TSAD) is the lack of labelled data in many real- life scenarios. Most of the existing anomaly detection methods focus on learning the normal behaviour of unlabelled time series in an unsupervised manner. The normal boundary is often defined tightly, resulting in slight deviations being classified as anomalies, consequently leading to a high false positive rate and a limited ability to generalise normal patterns. To address this, we introduce a novel end-to-end self-supervised ContrAstive Representation Learning approach for time series Anomaly detection (CARLA). While existing contrastive learning methods assume that augmented time series windows are positive samples and temporally distant windows are negative samples, we argue that these assumptions are limited as augmentation of time series can transform them to negative samples, and a temporally distant window can represent a positive sample. Existing approaches to contrastive learning for time series have directly copied methods developed for image analysis. We argue that these methods do not transfer well. Instead, our contrastive approach leverages existing generic knowledge about time series anomalies and injects various types of anomalies as negative samples. Therefore, CARLA not only learns normal behaviour but also learns deviations indicating anomalies. It creates similar representations for temporally close windows and distinct ones for anomalies. Additionally, it leverages the information about representations' neighbours through a self-supervised approach to classify windows based on their nearest/furthest neighbours to further enhance the performance of anomaly detection. In extensive tests on seven major real-world TSAD datasets, CARLA shows superior performance (F1 and AU- PR) over state-of-the-art self-supervised, semi-supervised, and unsupervised TSAD methods for univariate time series and multivariate time series. Our research highlights the immense potential of contrastive representation learning in advancing the TSAD field, thus paving the way for novel applications and in-depth exploration.

Time series anomaly detection has applications in a wide range of research fields and applications, including manufacturing and healthcare. The presence of anomalies can indicate novel or unexpected events, such as production faults, system defects, or heart fluttering, and is therefore of particular interest. The large size and complex patterns of time series have led researchers to develop specialised deep learning models for detecting anomalous patterns. This survey focuses on providing structured and comprehensive state-of-the-art time series anomaly detection models through the use of deep learning. It providing a taxonomy based on the factors that divide anomaly detection models into different categories. Aside from describing the basic anomaly detection technique for each category, the advantages and limitations are also discussed. Furthermore, this study includes examples of deep anomaly detection in time series across various application domains in recent years. It finally summarises open issues in research and challenges faced while adopting deep anomaly detection models.

Primary malignant bone tumors are the third highest cause of cancer-related mortality among patients under the age of 20. X-ray scan is the primary tool for detecting bone tumors. However, due to the varying morphologies of bone tumors, it is challenging for radiologists to make a definitive diagnosis based on radiographs. With the recent advancement in deep learning algorithms, there is a surge of interest in computer-aided diagnosis of primary bone tumors. Nonetheless, the development in this field has been hindered by the lack of publicly available X-ray datasets for bone tumors. To tackle this challenge, we established the Bone Tumor X-ray Radiograph dataset (termed BTXRD) in collaboration with multiple medical institutes and hospitals. The BTXRD dataset comprises 3,746 bone images (1,879 normal and 1,867 tumor), with clinical information and global labels available for each image, and distinct mask and annotated bounding box for each tumor instance. This publicly available dataset can support the development and evaluation of deep learning algorithms for the diagnosis of primary bone tumors.

We introduce MONSTER-the MONash Scalable Time Series Evaluation Repository-a collection of large datasets for time series classification. The field of time series classification has benefitted from common benchmarks set by the UCR and UEA time series classification repositories. However, the datasets in these benchmarks are small, with median sizes of 217 and 255 examples, respectively. In consequence they favour a narrow subspace of models that are optimised to achieve low classification error on a wide variety of smaller datasets, that is, models that minimise variance, and give little weight to computational issues such as scalability. Our hope is to diversify the field by introducing benchmarks using larger datasets. We believe that there is enormous potential for new progress in the field by engaging with the theoretical and practical challenges of learning effectively from larger quantities of data.

Time series classification (TSC) is a challenging task due to the diversity of types of feature that may be relevant for different classification tasks, including trends, variance, frequency, magnitude, and various patterns. To address this challenge, several alternative classes of approach have been developed, including similarity-based, features and intervals, shapelets, dictionary, kernel, neural network, and hybrid approaches. While kernel, neural network, and hybrid approaches perform well overall, some specialized approaches are better suited for specific tasks. In this paper, we propose a new similarity-based classifier, Proximity Forest version 2.0 (PF 2.0), which outperforms previous state-of-the-art similarity-based classifiers across the UCR benchmark and outperforms state-of-the-art kernel, neural network, and hybrid methods on specific datasets in the benchmark that are best addressed by similarity-base methods. PF 2.0 incorporates three recent advances in time series similarity measures -- (1) computationally efficient early abandoning and pruning to speedup elastic similarity computations; (2) a new elastic similarity measure, Amerced Dynamic Time Warping (ADTW); and (3) cost function tuning. It rationalizes the set of similarity measures employed, reducing the eight base measures of the original PF to three and using the first derivative transform with all similarity measures, rather than a limited subset. We have implemented both PF 1.0 and PF 2.0 in a single C++ framework, making the PF framework more efficient.

In time series anomaly detection (TSAD), the scarcity of labeled data poses a challenge to the development of accurate models. Unsupervised domain adaptation (UDA) offers a solution by leveraging labeled data from a related domain to detect anomalies in an unlabeled target domain. However, existing UDA methods assume consistent anomalous classes across domains. To address this limitation, we propose a novel Domain Adaptation Contrastive learning model for Anomaly Detection in multivariate time series (DACAD), combining UDA with contrastive learning. DACAD utilizes an anomaly injection mechanism that enhances generalization across unseen anomalous classes, improving adaptability and robustness. Additionally, our model employs supervised contrastive loss for the source domain and self-supervised contrastive triplet loss for the target domain, ensuring comprehensive feature representation learning and domain-invariant feature extraction. Finally, an effective Center-based Entropy Classifier (CEC) accurately learns normal boundaries in the source domain. Extensive evaluations on multiple real-world datasets and a synthetic dataset highlight DACAD's superior performance in transferring knowledge across domains and mitigating the challenge of limited labeled data in TSAD.

The accurate identification of catalytic residues contributes to our understanding of enzyme functions in biological processes and pathways. The increasing number of protein sequences necessitates computational tools for the automated prediction of catalytic residues in enzymes. Here, we introduce SCREEN, a graph neural network for the high-throughput prediction of catalytic residues via the integration of enzyme functional and structural information. SCREEN constructs residue representations based on spatial arrangements and incorporates enzyme function priors into such representations through contrastive learning. We demonstrate that SCREEN (1) consistently outperforms currently-available predictors; (2) provides accurate results when applied to inferred enzyme structures; and (3) generalizes well to enzymes dissimilar from those in the training set. We also show that the putative catalytic residues predicted by SCREEN mimic key structural and biophysical characteristics of native catalytic residues. Moreover, using experimental datasets, we show that SCREEN's predictions can be used to distinguish residues with a high mutation tolerance from those likely to cause functional loss when mutated, indicating that this tool might be used to infer disease-associated mutations. SCREEN is publicly available at https://github.com/BioColLab/SCREEN and https://ngdc.cncb.ac.cn/biocode/tool/7580.

Data science approaches have been increasingly implemented in healthcare big data to evaluate the safety and effectiveness of drugs. Association discovery is a data mining approach that finds potentially associated elements in high-dimensional data. We present a novel implementation of the association discovery approach in longitudinal healthcare data to identify drug safety and drug repurposing signals from positive and inverse associations between drug use and clinical outcomes, respectively. We used the 10% sample data from the Australian Pharmaceutical Benefits Scheme (2014-2024), which comprises prescription claims records. Using the Magnum Opus association discovery tool, we identified associations between a wide range of drugs and three common chronic medical conditions (i.e., coronary artery disease, type 2 diabetes, epilepsy). Cases with the conditions were identified using the supply of indicator drug(s) as a proxy for the conditions and matched to controls not supplied the indicator drug(s). Drug use was defined using Anatomical Therapeutic Chemical (ATC) codes supplied during a one-year lookback period before the supply of the indicator drug(s). We also evaluated combinations of up to four drugs and identified associations at the ATC level of drug class(es). In this study, we reproduced several known adverse drug events and protective drug effects, while some associations were attributable to confounding, mutual indications, or reverse causation. The remaining associations may represent previously uncharacterized drug safety and drug repurposing signals, necessitating further validation. We also discussed methodological differences between this association discovery approach and other similar data science approaches.

OBJECTIVE:The healthcare landscape is experiencing a transformation with the integration of Artificial Intelligence (AI) into traditional analytic workflows. However, its integration faces challenges resulting in a crisis of generalisability. Key obstacles include; 1) Insufficient consideration of local contextual factors, such as institution-specific data formats, practices, and protocols, which can lead to variability in clinical practices across different institutions. 2) ad-hoc data preparation and design of machine learning strategies. 3) manual subjective adjustment of design parameters resulting in sub-optimal performance. 4) EHR specific challenges regarding data biases affecting the model outcomes and unique intermittent temporal nature of the data necessitating specialised handling 5) lack of cross-institutional data validations. METHODS:To address these challenges, EHR-ML, provides an easy to use structured framework for designing optimum machine learning applications in a data-driven manner. The framework supports ingestion of local institutional electronic health records (EHRs) and process standardisation. The study design and parameter optimisation is done in a fully data-driven evidence-based approach. It seamlessly integrating with existing quality control tools. To handle the unique characteristics of the EHR data, it offers customisable ensemble models. It enables the acquisition of EHR data from diverse systems and harmonise them into common formats following international standards. RESULTS:The effectiveness of the EHR-ML is demonstrated through a series of case studies. These studies highlight its capability to develop high-performance models in a fully automated manner, consistently surpassing the performance of traditional methodologies. Furthermore, they exhibited strong generalisability across diverse healthcare settings. DISCUSSION AND CONCLUSION:EHR-ML enhances the clinical relevance and accuracy of predictive models by incorporating local context into machine learning applications. Additionally, by providing an user-friendly fully-automated framework, it facilitates rapid hypothesis testing aimed to generate localised biomedical knowledge.

Time series classification and regression are rapidly evolving fields that find areas of application in all domains of machine learning and data science. This hands on tutorial will provide an accessible overview of the recent research in these fields, using code examples to introduce the process of implementing and evaluating an estimator. We will show how to easily reproduce published results and how to compare a new algorithm to state-of-the-art. Finally, we will work through real world examples from the field of Electroencephalogram (EEG) classification and regression. EEG machine learning tasks arise in medicine, brain-computer interface research and psychology. We use these problems to how to compare algorithms on problems from a single domain and how to deal with data with different characteristics, such as missing values, unequal length and high dimensionality. The latest advances in the fields of time series classification and regression are all available through the aeon toolkit, an open source, scikit-learn compatible framework for time series machine learning which we use to provide our code examples.

Time series are the primary data type used to record dynamic system measurements and generated in great volume by both physical sensors and online processes (virtual sensors). Time series analytics is therefore crucial to unlocking the wealth of information implicit in available data. With the recent advancements in graph neural networks (GNNs), there has been a surge in GNN-based approaches for time series analysis. These approaches can explicitly model inter-temporal and inter-variable relationships, which traditional and other deep neural network-based methods struggle to do. In this survey, we provide a comprehensive review of graph neural networks for time series analysis (GNN4TS), encompassing four fundamental dimensions: forecasting, classification, anomaly detection, and imputation. Our aim is to guide designers and practitioners to understand, build applications, and advance research of GNN4TS. At first, we provide a comprehensive task-oriented taxonomy of GNN4TS. Then, we present and discuss representative research works and introduce mainstream applications of GNN4TS. A comprehensive discussion of potential future research directions completes the survey. This survey, for the first time, brings together a vast array of knowledge on GNN-based time series research, highlighting foundations, practical applications, and opportunities of graph neural networks for time series analysis.

Time Series Classification and Extrinsic Regression are important and challenging machine learning tasks. Deep learning has revolutionized natural language processing and computer vision and holds great promise in other fields such as time series analysis where the relevant features must often be abstracted from the raw data but are not known a priori. This paper surveys the current state of the art in the fast-moving field of deep learning for time series classification and extrinsic regression. We review different network architectures and training methods used for these tasks and discuss the challenges and opportunities when applying deep learning to time series data. We also summarize two critical applications of time series classification and extrinsic regression, human activity recognition and satellite earth observation.