Canonical correlation analysis (CCA) is a widely used multivariate analysis technique for explaining the relation between two sets of variables. It achieves this goal by finding linear combinations of the variables with maximal correlation. Recently, under the assumption that leading canonical directions are sparse, various penalized CCA procedures have been proposed for high dimensional data applications. However, all these procedures have the inconvenience of not preserving the sparsity among the retained leading canonical directions. To address this issue, two new sparse CCA methods are proposed in this paper. The first method obtained by diagonal thresholding of two square matrices derived from the cross-covariance matrix of the two sets of variables where each matrix characterizes one set of variables. A model selection criterion is used to select the number of variables to retain from each matrix diagonal. The second method is derived within an adaptive alternating penalized least squares framework where the l1/2-norm is used as a penalty promoting block sparsity. Compared to existing sparse CCA methods, the proposed methods have the advantage of preserving the sparsity across the retained canonical loading vectors. Their performance are illustrated in an extended experimental study which shows the superior performance of the proposed methods.

Domain Generalization (DG) aims to learn representations robust to distribution shifts. Recent geometric alignment methods, such as CPCANet, extract domain-invariant structures through batch-wise Common Principal Component Analysis (CPCA). However, CPCANet suffers from rank-deficient covariance estimation due to the small-sample-size issue in mini-batch training. To address this limitation, we propose Projection Pursuit CPCANet (PP-CPCANet), a covariance-free framework that learns a global orthogonal basis on the Stiefel manifold and jointly optimizes it with network parameters via the Cayley transform. We further introduce a symmetry-breaking detached-median PP dispersion objective to extract common principal components (CPCs) with dense and robust optimization signals. Experiments on four DG benchmarks show that PP-CPCANet achieves SOTA performance while maintaining stable training.

This paper proposes a novel robust Fisher Discriminant Analysis (FDA) for discriminative subspace learning in the presence of outliers. The proposed approach is motivated by the maximum-likelihood perspective on FDA and its connection to Kullback-Leibler (KL) divergence minimization. Within the probabilistic FDA framework, we develop a robust model by adopting the $\alpha $ -divergence as a flexible alternative to the KL divergence. The resulting method induces an adaptive redescending weighting scheme, in which each observation is weighted according to its statistical compatibility with the model, where the robustness level continuously controlled by $\alpha $ . As $\alpha $ decreases from 1, the influence of outliers is progressively suppressed, while classical FDA is recovered at $\alpha = 1$ . Combined with a two-fold iterative optimization procedure, the proposed method mitigates contamination at both the class-modeling stage and the projection-learning stage. We further provide theoretical analysis of the robustness mechanism, convergence, and computational complexity analysis to support the effectiveness and efficiency of the proposed method. Extensive experiments on synthetic data and multiple public image datasets under diverse contamination settings demonstrate that the proposed method consistently outperforms representative robust FDA variants and related approaches.

Domain Generalization (DG) aims to learn representations that remain robust under out-of-distribution (OOD) shifts and generalize effectively to unseen target domains. While recent invariant learning strategies and architectural advances have achieved strong performance, explicitly discovering a structured domain-invariant subspace through second-order statistics remains underexplored. In this work, we propose CPCANet, a novel framework grounded in Common Principal Component Analysis (CPCA), which unrolls the iterative Flury-Gautschi (FG) algorithm into fully differentiable neural layers. This approach integrates the statistical properties of CPCA into an end-to-end trainable framework, enforcing the discovery of a shared subspace across diverse domains while preserving interpretability. Experiments on four standard DG benchmarks demonstrate that CPCANet achieves state-of-the-art (SOTA) performance in zero-shot transfer. Moreover, CPCANet is architecture-agnostic and requires no dataset-specific tuning, providing a simple and efficient approach to learning robust representations under distribution shift. Code is available at https://github.com/wish44165/CPCANet.

In this paper, we address the problem of robust direction -of-arrival (DOA) estimation in unknown spatially correlated noise fields using sensor arrays composed of subarrays in sparse configurations. In such arrays, the noise covariance matrix has a block-diagonal structure. The proposed robust DOA estimation method is derived from a parametric distribution divergence, cr-divergence. Our approach can be viewed as an extension of existing Maximum Likelihood (ML) iterative procedures. The degree of robustness is controlled by the parameter a: as a 1, the proposed method converges to the traditional ML approach, while for a < 1, our method effectively mitigates the impact of potential outliers. Moreover, simulation studies show that our robust DOA estimation not only handles two types of outliers better than the ML method, but also exhibits high breakdown point properties.

Small object detection (SOD) in optical images and videos is a challenging problem that even state-of-the-art generic object detection methods fail to accurately localize and identify such objects. Typically, small objects appear in real-world due to large camera-object distance. Because small objects occupy only a small area in the input image (e.g., less than 10%), the information extracted from such a small area is not always rich enough to support decision making. Multidisciplinary strategies are being developed by researchers working at the interface of deep learning and computer vision to enhance the performance of SOD deep learning based methods. In this paper, we provide a comprehensive review of over 160 research papers published between 2017 and 2022 in order to survey this growing subject. This paper summarizes the existing literature and provide a taxonomy that illustrates the broad picture of current research. We investigate how to improve the performance of small object detection in maritime environments, where increasing performance is critical. By establishing a connection between generic and maritime SOD research, future directions have been identified. In addition, the popular datasets that have been used for SOD for generic and maritime applications are discussed, and also well-known evaluation metrics for the state-of-the-art methods on some of the datasets are provided.

This study examined the statistical underpinnings of dynamic functional connectivity in mental disorders, using resting-state fMRI signals. Notably, there has been an absence of research demonstrating the non-stationarity of the empirical probability distribution of functional connectivity. This gap has prompted debate on the existence of dynamic functional connectivity, leading skeptics to question its relevance and the reliability of research findings. Our aim was to fill this gap by conducting a comprehensive empirical distribution analysis of functional connectivity, using Pearson's correlation as a measure. We conducted our analysis on a set of preprocessed resting-state fMRI samples obtained from 186 subjects selected from the UCLA Consortium for Neuropsychiatric Phenomics dataset. Departing from conventional methods that aggregated signals over voxels within a region of interest, our approach leveraged individual voxel signals. Specifically, our approach offered a precise characterization of the empirical probability distribution of resting-state fMRI signals by evaluating the temporal variations and non-stationarity in dynamic functional connectivity, as measured by Pearson's correlation. Our study investigated functional connectivity patterns across 49 regions of interest, comparing healthy control subjects with patients diagnosed with ADHD, bipolar disorder, and schizophrenia. Our analysis revealed that (1) the empirical distribution of the correlation coefficient exhibited non-stationarity, (2) the beta distribution was an accurate approximation of the exact correlation coefficient distribution, and (3) the empirical distribution of means derived from the fitted beta distributions, unraveled distinctive dynamic functional connectivity patterns with potential as biomarkers associated with different mental disorders. A key contribution of our study was the presentation of the first comprehensive empirical distribution analysis of dynamic functional connectivity, thus providing compelling evidence for its existence. Overall, our study presented an innovative statistical approach that advances our understanding of the dynamic nature of functional connectivity patterns derived from resting-state fMRI. Our examination of the empirical distribution of dynamic functional connectivity provided solid evidence supporting its existence. The distinctive dynamic functional connectivity patterns we identified across various mental disorders hold promise as potential biomarkers for further development.

In this paper, novel robust principal component analysis (RPCA) methods are proposed to exploit the local structure of datasets. The proposed methods are derived by minimizing the α -divergence between the sample distribution and the Gaussian density model. The α- divergence is used in different frameworks to represent variants of RPCA approaches including orthogonal, non-orthogonal, and sparse methods. We show that the classical PCA is a special case of our proposed methods where the α- divergence is reduced to the Kullback-Leibler (KL) divergence. It is shown in simulations that the proposed approaches recover the underlying principal components (PCs) by down-weighting the importance of structured and unstructured outliers. Furthermore, using simulated data, it is shown that the proposed methods can be applied to fMRI signal recovery and Foreground-Background (FB) separation in video analysis. Results on real world problems of FB separation as well as image reconstruction are also provided.

Canonical correlation analysis (CCA) is a widely used mutivariate statistical technique for exploring the relationship between two multivariable datasets. It extracts existing relationship information by finding pairs of linear combinations from the two sets of variables with maximum correlation. In some applications however, the observed datasets may be contaminated by outliers and the standard CCA methods are sensitive to the presence of outliers in the datasets. In this paper, a robust CCA (RCCA) algorithm is presented. It is obtained using the interpretation of CCA as a latent variable model with two Gaussian random vectors and a robust loss function derived from the alpha-divergence as an alternative to maximum likelihood. Compared to existing robust CCA approaches, the proposed loss has the advantage of belonging to class of redescending M-estimators, guaranteeing inference stability for large deviation from the Gaussian nominal noise model. Experimental results on simulated and real datasets show that the proposed RCCA outperforms some existing robust and standard CCA methods.

Convex and penalized robust regression methods often suffer from a persistent bias induced by large outliers, limiting their effectiveness in adversarial or heavy-tailed settings. In this work, we study a smooth redescending non-convex M-estimator, specifically the Welsch estimator, and show that it can eliminate this bias whenever it is statistically identifiable. We focus on high-dimensional linear regression under adversarial contamination, where a fraction of samples may be corrupted by an adversary with full knowledge of the data and underlying model. A central technical contribution of this paper is a practical algorithm that provably finds a statistically valid solution to this non-convex problem. We show that the Welsch objective remains locally convex within a well-characterized basin of attraction, and our algorithm is guaranteed to converge into this region and recover the desired estimator. We establish three main guarantees: (a) non-asymptotic minimax-optimal deviation bounds under contamination, (b) improved unbiasedness in the presence of large outliers, and (c) asymptotic normality, yielding statistical efficiency as the sample size grows. Finally, we support our theoretical findings with comprehensive experiments on synthetic and real datasets, demonstrating the estimator's superior robustness, efficiency, and effectiveness in mitigating outlier-induced bias relative to state-of-the-art robust regression methods.

Given a set of p symmetric (real) matrices, the Orthogonal Joint Diagonalization (OJD) problem consists of finding an orthonormal basis in which the representation of each of these p matrices is as close as possible to a diagonal matrix. We argue that when the matrices are of large dimension, then the natural generalization of this problem is to seek an orthonormal basis of a certain subspace that is a near eigenspace for all the matrices in the set. We refer to this as the problem of “partial joint diagonalization of matrices.” The approach proposed first finds this approximate common near eigenspace and then proceeds to a joint diagonalization of the restrictions of the input matrices in this subspace. A few solution methods for this problem are proposed and illustrations of its potential applications are provided.

Download This Paper Open PDF in Browser Add Paper to My Library Share: Permalink Using these links will ensure access to this page indefinitely Copy URL Copy DOI

We studied the problem of robust subspace tracking (RST) in contaminated environments. Leveraging the fast approximated power iteration and α-divergence, a novel robust algorithm called αFAPI was developed for tracking the underlying principal subspace of streaming data over time. αFAPI is fast and it outperforms many RST methods while only having a low complexity linear to the data dimension. Some experiments were conducted to illustrate the performance of αFAPI.

Camera pose estimation has long relied on geometry-based approaches and sparse 2D-3D keypoint correspondences. With the advent of deep learning methods, the estimation of camera pose parameters (i.e., the six parameters that describe position and rotation) has decreased from tens of meters to a few centimeters in median error for indoor applications. For outdoor applications, errors can be quite large and highly dependent on the levels of variations in occlusion, contrast, brightness, repetitive structures, or blur introduced by camera motion. To address these limitations, we introduce, BPose, a Bayesian Convolutional deep network capable of not only automatically estimating the camera’s pose parameters from a single RGB image but also providing a measure of uncertainty in the parameter estimation. Reported experiments on outdoor and indoor datasets demonstrate that B-Pose outperforms SOTA techniques and generalizes better to unseen RGB images. A strong correlation is shown between the prediction error and the model’s uncertainty, indicating that the prediction is almost always incorrect whenever the model’s uncertainty is high.

In this paper, we generalize the well-known Expectation Maximization (EM) algorithm using the α−divergence for Gaussian Mixture Model (GMM). This approach is used in robust subspace detection when the number of parameters is kept small to avoid overfitting and large estimation variances. The level of robustness can be tuned by the parameter α. When α → 1, our method is equivalent to the standard EM approach and for α < 1 the method is robust against potential outliers. Simulation results show that the method outperforms the standard EM when it comes to mismatches between noise models and their realizations. In addition, we use the proposed method to detect active brain areas using collected functional Magnetic Resonance Imaging (fMRI) data during task-related experiments.

In this article, the problem of estimating and tracking a subspace signal in the presence of non-Gaussian noise is addressed. In contrast to non-robust methods such as PAST, NIC, and NP3, which are based on restrictive noise models, we use the popular $\epsilon -$ contamination noise model employed in robust statistics with Gaussian density as the nominal model to estimate the subspace that the target signal lives in. Adopting a new robust measure borrowed from the information geometry, i.e., the parametric family of $\alpha -$ divergence, two subspace tracking methods are proposed. The first one tolerates deviations over the entire observed vector signal whereas the second method is robust to sparsely distributed deviations in entries of the observed signal vector and efficiently uses the information in clean entries to perform the subspace estimation task. Both variants are implemented recursively, and are fit for adaptive real time processing. Simulation results reveal the superiority of the proposed methods in some metrics such as subspace estimation performance (SEP) and orthonormal error (OE) in several synthetic and real world problems.

Dictionary learning methods have been extensively used in different types of image and signal processing tasks. In a number of applications, the collected data/signal may have a multi-subspace structure and be perturbed with outliers. These motivate the use of robust and block-sparse signal representations. In this paper, a new algorithm for learning a block-structured dictionary in the presence of outliers is proposed. It is based on & alpha;-divergence and has the advantage of tolerating the presence of outliers. A block coordinate descent approach is adopted to obtain simple closed-form solutions for both the sparse coding and dictionary update stages. Finally, experimental results illustrating the superiority of the proposed method over some state-of-the-art dictionary learning methods, are provided.& COPY; 2023 Published by Elsevier B.V.

Objective: In this paper, we aim to address the problem of subspace detection in the presence of locally-correlated complex Gaussian noise and interference. For applications like brain activity detection using functional magnetic resonance imaging (fMRI) data where the noise is possibly locally correlated, using the sample covariance estimator is not a suitable choice due to significant dependency of its accuracy on the number of observations. Methods: In this study, we take advantage of an assumed banded structure in the covariance matrix to model the local dependence in the noise and propose a new covariance estimation approach. In particular, we use the idea of factorizing the joint likelihood function into a few conditional likelihood terms and maximizing each term independently of the others. This process leads to an explicit estimator for banded covariance matrices which requires fewer observations to achieve the same accuracy as the sample covariance. This estimate is then fed into an adaptive matched filter, two-step Rao and two-step Wald tests for detection. Results: Simulation results reveal the superiority of the proposed methods over well known classical detectors. Finally, the proposed methods are applied to functional magnetic resonance imaging (fMRI) data to localize neural activities in the brain. Conclusion: The proposed method can offer better activation maps in terms of accuracy and spatial smoothness. Significance: The proposed methods can be seen as alternatives for standard detection approaches which are not perfectly aligned with the properties of fMRI data.

Robust variants of Wald, Rao and likelihood ratio (LR) tests for the detection of a signal subspace in a signal interference subspace corrupted by contaminated Gaussian noise are proposed in this paper. They are derived using the α-divergence, and the trade-off between the robustness and the power (the probability of detection) of the tests is adjustable using a single hyperparameter α. It is shown that when α→ 1, these tests are equivalent to their well known classical counterparts. For example the robust LR test coincides with the LR test or the matched subspace detector (MSD). Asymptotic results are provided to support the proposed tests and robustness to outliers is obtained using values of α <; 1. Numerical experiments illustrating the performance of these tests on simulated, real functional magnetic resonance imaging (fMRI), hyperspectral and synthetic aperture radar (SAR) data are also presented.

This article addresses the robust estimation of the output layer linear parameters in a radial basis function network (RBFN). A prominent method used to estimate the output layer parameters in an RBFN with the predetermined hidden layer parameters is the least-squares estimation, which is the maximum-likelihood (ML) solution in the specific case of the Gaussian noise. We highlight the connection between the ML estimation and minimizing the Kullback-Leibler (KL) divergence between the actual noise distribution and the assumed Gaussian noise. Based on this connection, a method is proposed using a variant of a generalized KL divergence, which is known to be more robust to outliers in the pattern recognition and machine-learning problems. The proposed approach produces a surrogate-likelihood function, which is robust in the sense that it is adaptive to a broader class of noise distributions. Several signal processing experiments are conducted using artificially generated and real-world data. It is shown that in all cases, the proposed adaptive learning algorithm outperforms the standard approaches in terms of mean-squared error (MSE). Using the relative increase in the MSE for different noise conditions, we compare the robustness of our proposed algorithm with the existing methods for robust RBFN training and show that our method results in overall improvement in terms of absolute MSE values and consistency.