Cardinality-constrained portfolio optimization aims at determining the investment weights on given assets using the historical data. This problem typically requires three constraints, namely, capital budget, long-only, and sparsity. The sparsity restraint allows investment managers to select a small number of stocks from the given assets. Most existing approaches exploit the penalty technique to handle the sparsity constraint. Therefore, they require tweaking the associated regularization parameter to obtain the desired cardinality level, which is timeconsuming. This paper formulates the sparse portfolio design as a cardinality-constrained nonconvex optimization problem, where the sparsity constraint is modeled as a bounded 80 0norm. The projected gradient descent (PGD) method is then utilized to deal with the resultant problem. Different from existing algorithms, the suggested approach, called 80-PGD, 0-PGD, can explicitly control the cardinality level. In addition, its convergence is established. Specifically, the 80-PGD 0-PGD guarantees that the objective function value converges, and the variable sequences converges to a local minimum. To remedy the weaknesses of gradient descent, the momentum technique is exploited to enhance the performance of the 80-PGD, 0-PGD, yielding 80-PMGD. 0-PMGD. Numerical results on four real-world datasets, viz. NASDAQ 100, S&P 500, Russell 1000, and Russell 2000 exhibit the superiority of the 80-PGD 0-PGD and 80-PMGD 0-PMGD over existing algorithms in terms of mean return and Sharpe ratio.

The effectiveness of an image classification system depends on the following two key components: 1) the feature learning module and 2) the classification module. A well-designed loss function can not only enhance the classification ability of the latter but also improve the feature extraction capabilities of the former. This article devises a novel hypersphere loss function, which enhances the intraclass compactness and interclass separability of feature vectors given by the feature learning module. Furthermore, a new generalized class center is introduced into the loss function to handle the inevitable variability in samples (such as illumination, background, blurriness, low resolution, etc.) within the same class. Then, an alternative learning strategy is employed to optimize trainable parameters and class centers. Specifically, we first fix the trainable parameters of the deep learning model and calculate class centers using the exponentially weighted moving average method. Subsequently, we fix the generalized class centers and update the model's trainable parameters using mini-batch stochastic gradient descent. The proposed algorithm is evaluated on a range of typical tasks, including standard image classification, face verification, object detection, and retail product checkout. The results demonstrate that our proposed algorithm outperforms several state-of-the-art approaches.

In practical scenarios, the non-line-of-sight errors or impulsive noise in dense urban areas pose great challenges to efficient and robust distributed multiple-input multiple-output radar localization. To achieve robustness against outliers, this paper proposes integrating data selection and localization into a unified framework based on the l(0)-norm optimization and the idea of hard weighting. To be specific, the proposed model introduces a binary auxiliary variable to indicate the outlier-contaminated and unpolluted bistatic range measurements with 0 and 1, respectively. Considering the sparse nature of outliers, we impose sparsity constraint on the binary variable. Then, the robust localization task is cast in the form of mixed integer programming with an l(0)-norm constraint. To address the resultant problem, the alternating minimization algorithm is adopted as the solver, where the target location and the binary variable are updated in an alternating manner. In particular, both subtasks have closed-form solutions with low computational complexity. Through numerical experiments, the efficiency and accuracy of the proposed algorithm is verified in comparison to several competing methods.

One-bit compressed sensing (1-bit CS) inherits the merits of traditional CS and further reduces the cost and burden on the hardware device via employing the 1-bit analog-to-digital converter. When the measurements do not involve sign flips caused by additive noise, most contemporary algorithms can attain excellent signal restoration. However, their recovery performance might significantly degrade if there is even a small portion of sign flips. In order to increase the estimation accuracy in noisy scenarios, we devise a new signal model for 1-bit CS to attain robustness against sign flips. Then, we give a double-sparsity optimization formulation of the restoration problem. Subsequently, we combine proximal alternating minimization and projected gradient descent to tackle the problem. Different from existing robust methodologies, our approach, referred to as robust one-bit CS (ROCS), does not require the number of sign flips. Furthermore, we analyze the convergence behavior of ROCS and show that the objective value and variable sequences converge. Numerical results using synthetic data demonstrate that ROCS is superior to the competing methods in terms of reconstruction error in noisy environments. ROCS is also applied to direction-of-arrival estimation and outperforms state-of-the-art approaches.

Most of the existing localization frameworks are established under the Gaussian noise assumption and thus provide unsatisfactory accuracy in the presence of outliers. This work considers the robust and efficient target localization with multiple-input multiple-output radar by adopting the idea of outlier separation and the l(0)-norm. Specifically, we model the outliers with an auxiliary variable and impose sparsity constraint on it. The localization task is then formulated in the form of l(0)-norm constrained optimization. In doing so, we integrate outlier detection and target localization into a single problem. An alternating optimization (AO)-based solver is developed for the resultant optimization problem. In detail, the AO-based algorithm consists of two steps, which updates the target location and the auxiliary variable alternately. In particular, both subtasks have closed-form solutions with low-computational complexity. Numerical results on both synthetic and real data verify the efficiency and accuracy of the proposed algorithm in comparison with four competing methods.

Inspired by sparse learning, the Markowitz mean-variance model with a sparse regularization term is popularly used in sparse portfolio optimization. However, in penalty-based portfolio optimization algorithms, the cardinality level of the resultant portfolio relies on the choice of the regularization parameter. This brief formulates the mean-variance model as a cardinality ( $\ell _{0}$ -norm) constrained nonconvex optimization problem, in which we can explicitly specify the number of assets in the portfolio. We then use the alternating direction method of multipliers (ADMMs) concept to develop an algorithm to solve the constrained nonconvex problem. Unlike some existing algorithms, the proposed algorithm can explicitly control the portfolio cardinality. In addition, the dynamic behavior of the proposed algorithm is derived. Numerical results on four real-world datasets demonstrate the superiority of our approach over several state-of-the-art algorithms.

This short communication addresses the problem of elliptic localization with outlier measurements. Outliers are prevalent in various location-enabled applications, and can significantly compromise the positioning performance if not adequately handled. Instead of following the common trend of using M-estimation or adjusting the conventional least squares formulation by integrating extra error variables, we take a different path. Specifically, we explore the worst-case robust approximation criterion to bolster resistance of the elliptic location estimator against outliers. From a geometric standpoint, our method boils down to pinpointing the Chebyshev center of a feasible set, which is defined by the available bistatic ranges with bounded measurement errors. For a practical approach to the associated min-max problem, we convert it into the convex optimization framework of semidefinite programming (SDP). Numerical simulations confirm that our SDP-based technique can outperform a number of existing elliptic localization schemes in terms of positioning accuracy in Gaussian mixture noise.

Deep generative models play a crucial role across various domains, with Generative Adversarial Networks (GANs) leading the advancements in deep generative techniques. However, GANs face challenges like unstable training and mode collapse, impacting image generation quality. In this paper, we introduce an optimization algorithm for GAN training, GEGAN, which leverages a gradient-directed evolutionary strategy. GEGAN integrates gradient and evolutionary algorithms and employs knowledge distillation to guide individual mutation, promoting mutual learning among individuals. Additionally, an accept-reject strategy is utilized in selection to prevent ineffective updates. Experimental results across multiple datasets demonstrate that GEGAN outperforms baseline models by capturing sample distribution features more efficiently, generating diverse samples, and mitigating mode collapse, showcasing its effectiveness from an evolutionary perspective.

Tensor completion (TC) refers to restoring the missing entries in a given tensor by making use of the low-rank structure. Most existing algorithms have excellent performance in Gaussian noise or impulsive noise scenarios. Generally speaking, the Frobenius-norm-based methods achieve excellent performance in additive Gaussian noise, while their recovery severely degrades in impulsive noise. Although the algorithms using the lp -norm ( ) or its variants can attain high restoration accuracy in the presence of gross errors, they are inferior to the Frobenius-norm-based methods when the noise is Gaussian-distributed. Therefore, an approach that is able to perform well in both Gaussian noise and impulsive noise is desired. In this work, we use a capped Frobenius norm to restrain outliers, which corresponds to a form of the truncated least-squares loss function. The upper bound of our capped Frobenius norm is automatically updated using normalized median absolute deviation during iterations. Therefore, it achieves better performance than the lp -norm with outlier-contaminated observations and attains comparable accuracy to the Frobenius norm without tuning parameter in Gaussian noise. We then adopt the half-quadratic theory to convert the nonconvex problem into a tractable multivariable problem, that is, convex optimization with respect to (w.r.t.) each individual variable. To address the resultant task, we exploit the proximal block coordinate descent (PBCD) method and then establish the convergence of the suggested algorithm. Specifically, the objective function value is guaranteed to be convergent while the variable sequence has a subsequence converging to a critical point. Experimental results based on real-world images and videos exhibit the superiority of the devised approach over several state-of-the-art algorithms in terms of recovery performance. MATLAB code is available at https://github.com/Li-X-P/Code-of-Robust-Tensor-Completion.

This short communication considers mitigating the negative effects of possibly unreliable path delay measurements acquired in non-line-of-sight (NLOS) environments on the positioning performance, a problem deserving further investigation within the expanding research area of elliptic localization. We present CASTELO, a Convex Approximation based Solution To Elliptic Localization with Outliers, to achieve such a goal. Our proposal corresponds to a mixed semidefinite (SD)/second-order cone (SOC) programming formulation derived from an error-mitigated nonlinear least squares (LS) location estimator, presenting itself as a remedy for the neglect of positivity of NLOS biases suffered by the majority of currently fashionable outlier-handling approaches. In terms of analytical discussions, we provide rationales supporting the incorporation of the SOC constraints, which serve to tighten the problem obtained after SD relaxation, and conduct a complexity analysis for the ultimate mixed SD/SOC programming formulation. Simulations are carried out to confirm the strong ability of CASTELO to attain reliable elliptic localization in the presence of NLOS outliers.

This contribution studies passive elliptic positioning (PEP) with unknown transmitter locations, a localization technique having great potential applicability ranging from underwater wireless sensor networks to intelligent transportation systems. Specifically, we aim to address the challenge of employing PEP in complex real-world environments where outliers may exist, by using the concept of robust statistics. To achieve such a goal, we replace the ℓ2 loss in the traditional nonlinear least squares formulation by a differentiable cost function that possesses outlier-resistance. The neurodynamic approach of Lagrange programming neural network is then adopted to solve the resultant nonconvex statistically robustified PEP problem in a computationally efficient manner. Simulations and acoustic positioning experiments demonstrate the performance superiority of our proposal over its competitors.

Direction-of-arrival (DOA) estimation is a frequent need in the field of array signal processing. While many conventional algorithms achieve excellent performance in Gaussian noise, they are vulnerable to impulsive noise. Although several approaches have been proposed for robust DOA estimation against gross errors, their disadvantages might limit the applicability in practice. For instance, the maximum likelihood (ML) estimation-based algorithms involve high computational complexity, and $\ell _{p}$-multiple signal classification with $p\in (1,2)$ requires tweaking $p$ for handling different noises. In this work, we devise a capped Frobenius norm (CFN) for complex-valued data inspired by the truncated least squares loss function. Since the cap threshold is the boundary to differentiate the normal and outlier-contaminated entries, we propose a normalized median absolute deviation-based strategy for its automatic determination. In doing so, the accurate estimation is achieved in both Gaussian and impulsive noise. As the CFN is nonconvex and nonsmooth, we exploit the half-quadratic theory to simplify the resultant problem into a tractable optimization, which is then handled by alternating convex optimization with computationally efficient closed-form solution. Furthermore, its convergence behaviors are analyzed, i.e., the objective function value is convergent, and there exists a subsequence in the variable sequence converging to a critical point. Simulation results exhibit its superior performance over several state-of-the-art algorithms in terms of estimation accuracy and resolution capability. MATLAB code is available at https://github.com/Li-X-P/Code-Robust-DOA-Estimator.

From the feature representation's point of view, the feature learning module of a convolutional neural network (CNN) is to transform an input pattern into a feature vector. This feature vector is then multiplied with a number of output weight vectors to produce softmax scores. The common training objective in CNNs is based on the softmax loss, which ignores the intra-class compactness. This brief proposes a constrained center loss (CCL)-based algorithm to extract robust features. The training objective of a CNN consists of two terms, softmax loss and CCL. The aim of the softmax loss is to push the feature vectors from different classes apart. Meanwhile, the CCL aims at clustering the feature vectors such that the feature vectors from the same classes are close together. Instead of using stochastic gradient descent (SGD) algorithms to learn all the connection weights and the cluster centers at the same time. Our CCL-based algorithm is based on the alternative learning strategy. We first fix the connection weights of the CNN and update the cluster centers based on an analytical formula, which can be implemented based on the minibatch concept. We then fix the cluster centers and update the connection weights for a number of SGD minibatch iterations. We also propose a simplified CCL (SCCL) algorithm. Experiments are performed on six commonly used benchmark datasets. The results demonstrate that the two proposed algorithms outperform several state-of-the-art approaches.

Matrix completion (MC) aims at recovering missing entries, given an incomplete matrix. Existing algorithms for MC are mainly designed for noiseless or Gaussian noise scenarios and, thus, they are not robust to impulsive noise. For outlier resistance, entry-wise $\ell _{p}$ -norm with $0 < p < 2$ and M-estimation are two popular approaches. Yet the optimum selection of $p$ for the entrywise $\ell _{p}$ -norm-based methods is still an open problem. Besides, M-estimation is limited by a breakdown point, that is, the largest proportion of outliers. In this article, we adopt entrywise $\ell _{0}$ -norm, namely, the number of nonzero entries in a matrix, to separate anomalies from the observed matrix. Prior to separation, the Laplacian kernel is exploited for outlier detection, which provides a strategy to automatically update the entrywise $\ell _{0}$ -norm penalty parameter. The resultant multivariable optimization problem is addressed by block coordinate descent (BCD), yielding $\ell _{0}$ -BCD and $\ell _{0}$ -BCD-F. The former detects and separates outliers, as well as its convergence is guaranteed. In contrast, the latter attempts to treat outlier-contaminated elements as missing entries, which leads to higher computational efficiency. Making use of majorization–minimization (MM), we further propose $\ell _{0}$ -BCD-MM and $\ell _{0}$ -BCD-MM-F for robust non-negative MC where the nonnegativity constraint is handled by a closed-form update. Experimental results of image inpainting and hyperspectral image recovery demonstrate that the suggested algorithms outperform several state-of-the-art methods in terms of recovery accuracy and computational efficiency.

This letter focuses on the robust low-rank matrix recovery (RLRMR) in the presence of gross sparse outliers. Instead of using $\ell _{1}$ -norm to reduce or suppress the influence of anomalies, we aim to eliminate their impact. To this end, we model the RLRMR as a mixed integer programming (MIP) problem based on the $\ell _{0}$ -norm. Then, a block coordinate descent (BCD) algorithm is developed to iteratively solve the resultant MIP. At each iteration, the proposed approach first utilizes the $\ell _{0}$ -norm optimization theory to assign binary weights to all entries of the residual between the known and estimated matrices. With these binary weights, the optimization over the bilinear term is reduced to a weighted extension of the Frobenius norm. As a result, the optimization problem is decomposed into a group of row-wise and column-wise subproblems with closed-form solutions. Additionally, the convergence of the proposed algorithm is studied. Simulation results demonstrate that the proposed method is superior to five state-of-the-art RLRMR algorithms.

Sparse index tracking, as one of the passive investment strategies, is to track a benchmark financial index via constructing a portfolio with a few assets in a market index. It can be considered as parameter learning in an adaptive system, in which we periodically update the selected assets and their investment percentages based on the sliding window approach. However, many existing algorithms for sparse index tracking cannot explicitly and directly control the number of assets or the tracking error. This article formulates sparse index tracking as two constrained optimization problems and then proposes two algorithms, namely, nonnegative orthogonal matching pursuit with projected gradient descent (NNOMP-PGD) and alternating direction method of multipliers for $\ell _{0}$ -norm (ADMM- $\ell _{0}$ ). The NNOMP-PGD aims at minimizing the tracking error subject to the number of selected assets less than or equal to a predefined number. With the NNOMP-PGD, investors can directly and explicitly control the number of selected assets. The ADMM- $\ell _{0}$ aims at minimizing the number of selected assets subject to the tracking error that is upper bounded by a preset threshold. It can directly and explicitly control the tracking error. The convergence of the two proposed algorithms is also presented. With our algorithms, investors can explicitly and directly control the number of selected assets or the tracking error of the resultant portfolio. In addition, numerical experiments demonstrate that the proposed algorithms outperform the existing approaches.

Nonstationary fuzzy inference systems (NFIS) are able to tackle uncertainties and avoid the difficulty of type-reduction operation. Combining NFIS and neural network, a first-order sparse TSK nonstationary fuzzy neural network (SNFNN-1) is proposed in this paper to improve the interpretability/translatability of neural networks and the self-learning ability of fuzzy rules/sets. The whole architecture of SNFNN-1 can be considered as an integrated model of multiple sub-networks with a variation in center, variation in width or variation in noise. Thus, it is able to model both “intraexpert” and “interexpert” variability. There are two techniques adopted in this network: the Mean Shift-based fuzzy partition and the Group Lasso-based rule selection, which can adaptively generate a suitable number of clusters and select important fuzzy rules, respectively. Quantitative experiments on six UCI datasets demonstrate the effectiveness and robustness of the proposed model.

Given a set of 2-dimensional (2-D) scattering points, which are usually obtained from the edge detection process, the aim of ellipse fitting is to construct an elliptic equation that best fits the collected observations. However, some of the scattering points may contain outliers due to imperfect edge detection. To address this issue, we devise a robust real-time ellipse fitting approach based on two kinds of analog neural network, Lagrange programming neural network (LPNN) and locally competitive algorithm (LCA). First, to alleviate the influence of these outliers, the fitting task is formulated as a nonsmooth constrained optimization problem in which the objective function is either an l1-norm or l0-norm term. It is because compared with the l2-norm in some traditional ellipse fitting models, the lp-norm with p<2 is less sensitive to outliers. Then, to calculate a real-time solution of this optimization problem, LPNN is applied. As the LPNN model cannot handle the non-differentiable term in its objective, the concept of LCA is introduced and combined with the LPNN framework. Simulation and experimental results show that the proposed ellipse fitting approach is superior to several state-of-the-art algorithms.

This paper focuses on target localization in a widely distributed multiple-input-multiple-output (MIMO) radar system. In this system, range measurements, which include the sum of distances between transmitter and target and the distances from the target to receivers, are used. We can obtain an accurate estimated position of the target by minimizing the measurement errors. In order to make our model come closer to reality, we introduce two kinds of noises, namely, Gaussian noise and outliers. When we evaluate a target localization algorithm, its localization accuracy and computational complexity are two main criteria. To improve the positioning accuracy, the original problem is formulated as solving a non-smooth constrained optimization problem in which the objective function is either l1-norm or l0-norm term. To achieve a real-time solution, the Lagrange programming neural network (LPNN) is utilized to solve this problem. However, it is well known that LPNN requires twice-differentiable objective function and constraints. Obviously, the l1-norm or l0-norm term in the objective function does not satisfy this requirement. To address this non-smooth optimization problem, this paper proposes two modifications based on the LPNN framework. In the first method, a differentiable proximate l1-norm function is introduced. While in the second method, locally competitive algorithm is utilized. Simulation and experimental results demonstrate that the performance of the proposed algorithms outperforms several existing schemes.

In feature representation learning, robust features are expected to have intra-class compactness and inter-class separability. The traditional softmax loss concept ignores the intra-class compactness. Hence the discriminative power of deep features is weakened. This paper proposes a constrained center loss (CCL) to enable CNNs to extract robust features. Unlike the general center loss (CL) concept, class centers are analytically updated from the deep features in our formulation. In addition, we propose to use the entire training set to approximate class centers. By doing so, class centers can better capture the global information of feature space. To improve training efficiency, an alternative algorithm is proposed to optimize the joint supervision of softmax loss and CCL. Experiments are performed on four benchmark datasets. The results demonstrate that the proposed scheme outperforms several existing architectures.