Despite their success, Deep Learning (DL) models are susceptible to spurious correlations. These are accidental correlations between non-predictive features of the training dataset and the target label, leading to models that do not generalise well to samples without such spurious correlations. We propose to address this failure mode by extending and improving an existing method for spurious correlation robustness — UnLearning from Experience (ULE). ULE trains two models in parallel; a biased model is encouraged to learn the spurious correlations, while a separate debiased model is trained to solve the same task while avoiding learning the same behaviour as the first, thus avoiding learning the spurious correlations. A weakness of ULE is that it treats every training example identically, regardless of whether or not it contains a spurious correlation. Therefore, ULE does not make optimal use of the training data, leading to worse-than-necessary performance. To address this weakness, we introduce Weighted UnLearning from Experience (wULE), which modifies the original ULE loss function to reweight the components depending on whether a training sample likely contains spurious correlations. Hence, we explore two methods for identifying training samples likely to contain spurious correlations. We test our approach using several well-known spurious correlation robustness benchmarks — Waterbirds, CelebA, UrbanCars and MultiNLI. Our novel wULE method outperforms the original ULE in all cases and achieves Worst-Group Accuracy (WGA) results comparable to the best methods in the spurious correlation robustness literature.
We study parameter inference in simulation-based stochastic models where the analytical form of the likelihood is unknown. The main difficulty is that score evaluation as a ratio of noisy Monte Carlo estimators induces bias and instability, which we overcome with a ratio-free nested multi-time-scale (NMTS) stochastic approximation (SA) method that simultaneously tracks the score and drives the parameter update. We provide a comprehensive theoretical analysis of the proposed NMTS algorithm for solving likelihood-free inference problems, including strong convergence, asymptotic normality, and convergence rates. We show that our algorithm can eliminate the original asymptotic bias O (√(1/N) ) and accelerate the convergence rate from O (β _k+√(1/N) ) to O (β _k/α _k+√(α _k/N) ) , where N is the fixed batch size, α _k and β _k are decreasing step sizes with α _k , β _k , β _k/α _k→ 0 . With proper choice of α _k and β _k , our convergence rates can match the optimal rate in the multi-time-scale SA literature. Numerical experiments demonstrate that our algorithm can improve the estimation accuracy by one to two orders of magnitude at the same computational cost, making it efficient for parameter estimation in stochastic systems.
ML is playing an increasingly crucial role in estimating causal effects of treatments on outcomes from observational data. Many ML methods (‘causal estimators’) have been proposed for this task. All of these methods, as with any ML approach, require extensive hyperparameter tuning. For non-causal predictive tasks, there is a consensus on the choice of tuning metrics (e.g. mean squared error), making it simple to compare models. However, for causal inference tasks, such a consensus is yet to be reached, making a any comparison of causal models difficult. On top of that, there is no ideal metric on which to tune causal estimators, so one must rely on proxies. Furthermore, the fact that model selection in causal inference involves multiple components (causal estimator, ML regressor, hyperparameters, metric), complicates the issue even further. In order to evaluate the importance of each component, we perform an extensive empirical study on their combination. Our experimental setup involves many commonly used causal estimators, regressors (‘base learners’ henceforth) and metrics applied to four well-known causal inference benchmark datasets. Our results show that hyperparameter tuning increased the probability of reaching state-of-the-art performance in average ( 65%→ 81% ) and individualised ( 50%→ 57% ) effect estimation with only commonly used estimators. We also show that the performance of standard metrics can be inconsistent across different scenarios. Our findings highlight the need for further research to establish whether metrics uniformly capable of state-of-the-art performance in causal model evaluation can be found.
Synthetic tabular data generation is increasingly important for privacy-preserving machine learning in healthcare, finance, and other regulated industries. Many current methods embed categorical variables into continuous spaces, which can blur their discrete structure and introduce interpolations between categorical states. Here, we propose flow matching on product manifolds, where each variable type resides on its natural geometric space: continuous variables in Euclidean space, categorical variables on probability simplexes with Fisher-Rao geometry, and ordinal variables on order-preserving arcs of unit circles. Our framework employs coordinate-wise vector field decomposition for manifold-specific processing and geometry-aware midpoint aggregation to capture inter-type dependencies with preserved geometric consistency. We evaluate the method on 12 benchmark datasets that range from small clinical records to educational datasets with over 101 million samples. Results show measurable improvements in generation quality, particularly on categorical-heavy and large-scale datasets. Geometric constraints also exhibit empirical resistance to memorization without explicit noise injection, as measured by distance to the closest record and membership inference metrics. The coordinate-wise structure enables efficient parallel computation that scales well to large datasets.
Singling out the most informative features able to discriminate between normal and anomalous data is a formidable problem dealing with the enormous search space represented by the power set of the whole feature set. These features indeed encode the explanation of the deviating behavior associated with selected observations and represent the hidden knowledge of interest for the analyst in many applicative contexts, namely cybersecurity, bioinformatics, and green by AI, to cite a few. In this work, we present RLAX, a reinforcement learning-based algorithm detecting explanations from collections of high-dimensional data sets. The algorithm receives in input the unlabelled data collection of normal instances, a small set of labelled inlier examples, and a small set of anomalies to be explained. Separation between normal samples and anomalies is measured by means of a distance-based outlier score. The designed learning strategy thus induces a ranking among features that allows isolating those maximizing separation accuracy. Beyond an add-only formulation, we investigate a subspace refinement strategy that augments the action space with remove and stop actions, enabling non-monotone refinement and adaptive early termination. Experimental results highlight that the size of the explanation returned consists of a small fraction of the whole feature set, and that the associated accuracy and subspace margin are of remarkable quality. Additional analyses of the refinement strategy further clarify its impact on exploration efficiency and explanation quality.
