Class-imbalanced and high-dimensional data pose significant challenges in machine learning, often leading to learning models that underperform on minority classes and struggle with the curse of dimensionality, respectively, and these two issues are inherently interconnected. This paper simultaneously addresses both class imbalance and high dimensionality through a multi-task multi-objective feature selection method. To be specific, the proposed method establishes two related yet distinct feature selection problem formulations for a given task, each focusing on different aspects of overall and minority classification performance. By selecting important features from the original imbalanced data set, the proposed method avoids the need to artificially equalize class distribution during training. Additionally, the cross-task genetic transfer operator enables the reuse of valuable feature subsets evolved from separate searches, thereby improving both minority class accuracy and overall model performance. Experimental results on 17 class-imbalanced data sets show that the proposed method achieves better results in terms of overall classification performance and minority class accuracy than the compared state-of-the-art methods, particularly on the high-dimensional data sets.

Mathematical equations describe fundamental laws across various disciplines, yet discovering concise and effective mathematical expressions from data remains a challenging task. Traditional symbolic regression methods often overlook domain-specific prior knowledge that scientists rely on, while large language model (LLM)-driven symbolic regression approaches can effectively leverage it. However, existing LLM-driven symbolic regression methods typically require substantial computational resources to generate equations while still suffering from low efficiency in producing high-quality expressions due to the lack of carefully designed prompts. To address this issue, we propose LLM-Guided Genetic Programming for Symbolic Regression (LLMGP-SR), a novel prompt-guided evolutionary search algorithm. LLMGP-SR integrates LLMs into the initialization, crossover, and mutation operations of genetic programming, achieving an organic integration of semantic generation and structural evolution of expressions. By leveraging an adaptive prompt strategy, LLMGP-SR constructs carefully designed prompts to guide LLMs in generating effective expressions. Experimental results demonstrate that LLMGP-SR significantly outperforms traditional genetic programming in symbolic regression problems across six standard benchmarks. The source code is available at https://github.com/chaoguo02/LLMGP-SR .

Minimizing the classification error rate and the number of selected features are the two major objectives of feature selection, and they are often in conflict with each other, which is a multiobjective problem. Evolutionary algorithms (EAs) have been widely used for multiobjective feature selection problems. Preselection in EAs is used to improve the sampling quality by selecting only potentially promising candidate solutions for fitness evaluations. However, traditional preselection methods struggle to effectively handle feature selection due to its large-scale combinatorial nature and intricate feature interactions. To alleviate this issue, this article proposes a filter-based performance predictor to preselect feature subsets for subsequent classification fitness evaluations. It uses multiple filter measures to estimate the classification performance of a feature subset, which can explore complex feature interactions and is also insensitive to the dimensionality. Additionally, a correlation coefficient is used to measure the compatibility between the learned performance predictor and the classification performance. Based on the degree of compatibility, a preselection method that considers both the predicted classification performance and the feature subset diversity is proposed, which can preselect promising solutions from multiple candidate solutions and thus improve the feature subset search efficiency. The proposed method is verified experimentally on a total of 18 classification datasets spanning various domains, and the results reveal that it can find feature subsets with better classification performance and converge faster to competitive results compared to state-of-the-art methods.

In many real-world scenarios, features could arrive in groups over time, and the total size of the feature space is often unknown. Streaming feature selection is a commonly-used approach to addressing such dynamic scenarios, where newly arriving features must be assessed for both their relevance and redundancy with previously selected features. To effectively solve such a task, this paper proposes an evolutionary streaming feature selection method via incremental feature clustering that comprises three stages. First, the online irrelevant feature filtering stage eliminates irrelevant streaming features to reduce the noise effect and shrink the search space. Second, the incremental redundant feature clustering stage groups mutually redundant features into clusters, adaptively creating or merging feature clusters to adapt to the dynamically-changing feature sets. Finally, the interactive feature subset search stage identifies representative features within each cluster to form the best feature subset. Experimental results on 18 real-world datasets demonstrate that the proposed method has better classification performance than the compared state-of-the-art methods.

In the big data era, the large volume and high dimensionality of data challenge many data mining algorithms. Since many classification algorithms are sensitive to data distribution, removing redundant or noisy features and instances can greatly improve their performance. Feature selection and instance selection are two major data reduction techniques that are inherently interconnected. This article proposes an auxiliary-optimization-assisted constrained multiobjective optimization method that concurrently tackles feature and instance selection, with two key constraints: forcing the obtained subsets to have better classification performance than that of using all features and instances under the given classification algorithm, and bounding the worst-class error. A simple but effective initialization method is designed to sample initial solutions relatively uniformly across different regions of the objective space, to provide a diverse and high-quality set of initial feature and instance subsets. An auxiliary-optimization-based search approach is proposed to fully utilize useful infeasible solutions, which can further reduce the number of selected features and instances without compromising classification performance. The proposed method is compared with a number of promising methods on 20 real-world classification datasets, and the experimental results show that it is generally better than those methods. Additionally, the proposed method offers a significant advantage whereby the majority of its solutions exhibit superior or comparable classification performance compared to using all original features and instances, while selecting no more than 30% of the features and 50% of the training instances across most datasets.

Neural architecture search (NAS) has attracted considerable interest due to its capability of automatically designing effective network architectures tailored to specific tasks. However, most existing NAS methods require extensive computational resources for architecture evaluation. Meanwhile, the increasing demand for deploying deep neural networks on mobile devices highlights the importance of reducing the model size. To overcome the aforementioned challenges, this paper proposes an evolutionary multi-objective NAS algorithm with efficient blocks using training-free ZiCo-Block evaluation, called EZB-NAS. We design two computationally efficient blocks and construct a hierarchical and variable-length search space to discover lightweight architectures. In addition, we improve the ZiCo proxy to reduce the structural bias in block evaluation by averaging the ZiCo scores across all layers within a block. The developed zero-cost proxy, named ZiCo-Block, is integrated into an evolutionary computation approach for lightweight architecture design. With improved genetic operators, we simultaneously optimize the ZiCo-Block score and the number of parameters to discover highly accurate and lightweight architectures. Experimental results on the CIFAR datasets show that EZB-NAS achieves competitive performance in terms of accuracy, model size, and computational cost, compared to efficient NAS algorithms. The source code is available at https://github.com/wangyule0/EZB-NAS .

Long-tailed image classification faces challenges of data imbalance and poor feature representation for tail classes, leading to biased predictions favoring head classes. While most existing methods rely on deep neural networks (DNNs), they typically require large amounts of training data and lack interpretability. Genetic Programming (GP) has shown promise in few-shot learning but has seldom been investigated in long-tailed image classification, primarily due to its limited ability to handle class imbalance and its tendency for fitness functions to be biased toward head classes. To fill this gap, this paper proposes a multi-expert GP method for long-tailed image classification.We develop three objective functions, each serving as an expert: 1) a longtailed expert focusing on head-class performance; 2) a balanced-class expert that promotes equal class representation; and 3) an inverse long-tailed expert emphasizing tail classes. This triexpert framework enables GP to jointly optimize complementary objectives and learn robust feature representations for both head and tail classes. To further improve classification performance, the evolved GP individuals from the final population are used to train base learners, and their outputs are integrated via a voting-based ensemble model. Experimental results demonstrate that the proposed method outperforms state-of-the-art GP and DNN approaches without pretraining across seven long-tailed image classification datasets.

In expensive optimization problems, surrogate models are expected to simultaneously achieve high prediction accuracy and reliable uncertainty quantification to support sampling decisions under limited evaluation budgets. Neural networks (NNs), owing to their strong capability in modeling complex system behaviors, are considered promising surrogate models. However, they lack the ability to directly quantify predictive uncertainty. Considering that Gaussian processes (GPs) possess inherent advantages in uncertainty estimation, this paper proposes integrating GPs into NNs, which can also be interpreted as a Gaussian model built upon NN features. Specifically, a novel nonstationary modeling approach is developed by combining NNs with GPs. In the proposed method, the hidden-layer features of NNs are employed as basis functions of nonstationary GPs (NSGPs), enabling the mean function to adaptively vary with decision variables, while the residual component is modeled by a stationary GP (SGP). Theoretical analysis demonstrates that the proposed method achieves a higher optimal likelihood value and stronger trend representation capability. Experimental results further show that the proposed approach yields superior predictive accuracy and optimization performance across different dimensional settings, validating its effectiveness in surrogate-assisted evolutionary optimization for solving expensive optimization problems.

Classification tasks often encounter imbalanced datasets, where skewed class distributions bias models toward the majority class, resulting in poor performance for the minority class. This issue becomes even more challenging in multi-class imbalanced datasets. Existing methods for addressing class imbalance often prioritize improving the classification performance of the minority class at the expense of the majority class. To tackle this issue, this paper proposes a skill-diverse expert learning strategy, which performs multi-objective evolutionary sampling from imbalanced data to obtain representative high-quality instance subsets. Three expert objective functions, acting as experts simulating different class distributions, are designed to evaluate the quality of the instance subsets. A constraint is proposed for each expert objective function to ensure that the instance subsets achieve better classification performance than using the full training set. Finally, an ensemble strategy is used to combine classifiers trained on diverse subsets of instances for prediction. Compared to state-of-the-art data-level and ensemble learning-based methods, the experimental results show that the proposed method delivers the best overall classification performance across 22 imbalanced datasets.

In surrogate-assisted evolutionary algorithms (SAEAs), uncertainty measures the confidence level of the predicted fitness and plays an important role in selecting candidate points for expensive evaluation, as evaluating those with a large degree of uncertainty can enhance the exploration ability of algorithms and significantly improve model reliability. Gaussian process (GP) gains great popularity with a predictive variance to estimate uncertainty. However, the performance of GP-assisted SAEAs is still limited with weakened predictions and underutilized uncertainty. In this paper, we propose a novel SAEA that leverages a boosting ensemble with GP as the base learner to enhance optimization performance. First, GP is leveraged to boost a weak learner that captures global trends, refining its prediction in guiding the search. Then, the uncertainty provided by GP is integrated into boosting ensemble model to estimate its overall ensemble uncertainty, enhancing the model reliability and promoting the exploration capabilities. Moreover, a mechanism study is performed through standardized residual and a distance-based metric to have deep insights into search behaviors, demonstrating that the migrated uncertainty basically covers the statistical boundary of Gaussian distribution and contributes to exploitation and exploration trade-off. Experimental results across multiple benchmark problems and an antenna design problem demonstrate that the proposed method significantly enhanced GP-assisted SAEAs in statistical results of the best fitness value and convergence profile without increasing too much running time, particularly in high-dimensional problems.

High-dimensional and small-sample datasets pose challenges for data analysis due to the “curse of dimensionality”. Traditional feature selection methods often struggle to capture complex feature interactions and cope with limited training samples. To address this issue, this paper proposes a large language model (LLM)-assisted feature selection method that combines both data-driven and text-guided approaches. Specifically, the method leverages LLM prior knowledge to perform statistical reasoning based on sample values and semantic associations through descriptive context. Unlike conventional LLM-assisted feature selection methods that usually select top-ranked features, the proposed method not only evaluates the relevance of features to the target variable but also incorporates redundancy analysis among features, ensuring the selection of relevant and non-redundant features. Experiments across 20 small-sample datasets from various domains, covering both regression and classification tasks, demonstrate the effectiveness and generalizability of the proposed method.

High-dimensional and imbalanced data often pose significant challenges, including the risk of biased learning models that disproportionately favor the majority class and the adverse effects of the curse of dimensionality. To address these two challenges simultaneously, this paper proposes a multiform multi-objective feature selection method tailored for high-dimensional imbalanced classification tasks. The proposed method reformulates the target problem by prioritizing minority class performance, rather than focusing solely on overall classification accuracy. Through performing an evolutionary search on the alternative formulation, valuable features identified through the auxiliary task can be transferred back to the target task, steering the feature subset search towards both enhanced overall classification performance and minority class performance. The proposed method is a general framework and could be integrated into three mainstream multi-objective optimization frameworks to tackle imbalanced classification tasks. Experimental results on 12 real-world datasets show that the three instantiations of the proposed method enhance both overall and minority class performance when compared to their respective baselines.

This paper proposes a customized algorithm named MOEA/D-AEPM for direction finding array (DFA) design. The aim of DFA design is to search a set of optimal spacing between array elements that minimizes the direction finding error (DFE), the correlation peak ratio (CPR), and the active S-parameter, resulting in an expensive multiobjective optimization problem (EMOP). Different from traditional EMOPs with expensive objectives, an active element pattern (AEP) from electromagnetic simulation is added with Gaussian noise to simulate the received signals. To avoid the influence of noise level on DFE and CPR, an AEP surrogate model (AEPM) is built to reduce cost. The experimental results demonstrate that MOEA/D-AEPM achieved reasonably good results within a limited cost, together with effective AEPM prediction, which can also provide a deep insight into radiation knowledge for antenna experts.

One of the most promising approaches of handling constrained optimization problems (COPs) is to adopt multi-objective methods, which can provide a trade-off between the objective and constraints.However, the multi-objective-based constraint-handling techniques take preference over infeasible solutions, some promising feasible solutions cannot survive during the course of search because they are dominated ones.Furthermore, some nondominated infeasible solutions with worse objective values should not be reserved in that they are far from the feasible optimal solution.To address these two problems, this paper proposes a Pareto front transformation model which transforms a part of potential feasible solutions into nondominated ones.Meanwhile, combined with the dynamic multi-objective technique, certain nondominated infeasible solutions with worse objective values will be rejected.In this way, the search can towards the global optimum from both the feasible and infeasible sides of the search space.The proposed Pareto front transformation model is integrated into a multi-objective-based constrained evolutionary algorithm (CEA).The new designed algorithm is named PT-MOEA, and it is compared with seven peer multi-objective-based CEAs and five state-of-the-art CEAs on solving IEEE CEC 2006 and IEEE CEC 2010 test suites, respectively.Experimental results demonstrate the competitiveness of the proposed method in comparison with its competitors for solving COPs.

The computing power network links distributed and heterogeneous computing resources via the network, to enable efficient configuration and utilization of computing power. However, scheduling computing resources within this network presents several challenges, such as resource heterogeneity, vast search spaces, uncertainty, high constraints, and real-time requirements. To simulate the real-world computing power network scheduling problem, this paper integrates cloud servers, fog servers, and edge servers into a unified computing power network, considering their respective GPU, CPU, and bandwidth resources. We introduce a Dual-Tree Genetic Programming (DTGP) approach that simultaneously optimizes two critical decisions—routing and sequencing—to automatically evolve computing power network scheduling heuristics for real-time decision-making. Additionally, to improve the performance of DTGP, we propose new terminal sets tailored to fit within these two GP trees. Experimental results demonstrate that the proposed method significantly outperforms existing state-of-the-art methods in six test scenarios, achieving up to 40% reduction in completion time.

Minimizing the number of selected features and maximizing the classification performance are two main objectives in feature selection, which can be formulated as a biobjective optimization problem. Due to the complex interactions between features, a solution (i.e., feature subset) with poor objective values does not mean that all the features it selects are useless, as some of them combined with other complementary features can greatly improve the classification performance. Thus, it is necessary to consider not only the performance of feature subsets in the objective space, but also their differences in the search space, to explore more promising feature combinations. To this end, this paper proposes a tri-objective method for bi-objective feature selection in classification, which solves a bi-objective feature selection problem as a triobjective problem by considering the diversity (differences) between feature subsets in the search space as the third objective. The selection based on the converted triobjective method can maintain a balance between minimizing the number of selected features, maximizing the classification performance, and exploring more promising feature subsets. Furthermore, a novel initialization strategy and an offspring reproduction operator are proposed to promote the diversity of feature subsets in the objective space and improve the search ability, respectively. The proposed algorithm is compared with five multi-objective-based feature selection methods, six typical feature selection methods, and two peer methods with diversity as a helper objective. Experimental results on 20 real-world classification datasets suggest that the proposed method outperforms the compared methods in most scenarios.

Sparse learning-based feature selection is an emerging topic, acclaimed for its potential in delivering promising performance and interpretability. Nevertheless, the task of determining a suitable regularization parameter to strike a balance between the loss function and regularization is a challenging endeavor, where existing methods encounter great difficulties. Moreover, the ranking mechanism in most sparse learning-based feature selection methods requires a predefined number of selected features, which is usually dataset-dependent and not known in advance. It is of great importance to automatically balance the loss function and sparse regularization and determine the appropriate number of selected features. To this end, this paper proposes formulating the sparse learning-based feature selection problem as a bi-objective optimization problem, which takes the loss term and the l(2,0)-norm regularization as two objectives, to automatically identify the optimal number of selected features and obtain a set of trade-off solutions between the loss term and the number of selected features. To solve such a non-convex problem, a novel solution representation, an initialization strategy, and an environmental selection operator are proposed. Compared with seven feature selection methods, extensive experiments on 16 practical classification datasets demonstrate that the proposed method attains highly competitive classification accuracy with a small number of selected features, and the features selected by the proposed method have low redundancy.

Reducing the number of selected features and improving the classification performance are two major objectives in feature selection, which can be viewed as a multiobjective optimization problem. Multiobjective feature selection in classification has its unique characteristics, such as it has a strong preference for the classification performance over the number of selected features. Besides, solution duplication often appears in both the search and the objective spaces, which degenerates the diversity and results in the premature convergence of the population. To deal with the above issues, in this article, during the evolutionary training process, a multiobjective feature selection problem is reformulated and solved as a constrained multiobjective optimization problem, which adds a constraint on the classification performance for each solution (e.g., feature subset) according to the distribution of nondominated solutions, with the aim of selecting promising feature subsets that contain more informative and strongly relevant features, which are beneficial to improve the classification performance. Furthermore, based on the distribution of feature subsets in the objective space and their similarity in the search space, a duplication analysis and handling method is proposed to enhance the diversity of the population. Experimental results demonstrate that the proposed method outperforms six state-of-the-art algorithms and is computationally efficient on 18 classification datasets.

Maximizing the classification accuracy and minimizing the number of selected features are two primary objectives in feature selection (FS), which is inherently a multiobjective task. Multiobjective FS (MOFS) enables us to gain various insights from complex data in addition to dimensionality reduction and improved accuracy, which has attracted increasing attention from researchers and practitioners. Over the past two decades, significant advancements in MOFS in classification have been achieved in both the methodologies and applications, but have not been well summarized and discussed. To fill this gap, this article presents a broad survey on existing research on MOFS in classification, focusing on up-to-date approaches, applications, current challenges, and future directions. To be specific, we categorize MOFS in classification on the basis of different criteria, and provide detailed descriptions of representative methods in each category. Additionally, we summarize a list of successful real-world applications of MOFS from different domains, to exemplify their significant practical value and demonstrate their abilities in providing a set of tradeoff feature subsets to meet different requirements of decision makers. We also discuss key challenges and shed lights on emerging directions for future developments of MOFS.

Constrained multiobjective optimization problems (CMOPs) pose great difficulties to the existing multiobjective evolutionary algorithms (MOEAs), in terms of constraint handling and the tradeoffs between diversity and convergence. The constraints divide the search space into feasible and infeasible regions. A key to solving CMOPs is how to effectively utilize the information of both feasible and infeasible solutions during the optimization process. In this article, we propose a multiform optimization framework to solve a CMOP task together with an auxiliary CMOP task in a multitask setting. The proposed framework is designed to conduct a search in different sizes of feasible space that is derived from the original CMOP task. The derived feasible space is easier to search and can provide a useful inductive bias to the search process of the original CMOP task, by leveraging the transferable knowledge shared between them, thereby helping the search to toward the Pareto optimal solutions from both the infeasible and feasible regions of the search space. The proposed framework is instantiated in three kinds of MOEAs: 1) dominance-based; 2) decomposition-based; and 3) indicator-based algorithms. Experiments on four sets of benchmark test problems demonstrate the superiority of the proposed method over four representative constraint-handling techniques. In addition, the comparison against five state-of-the-art-constrained MOEAs demonstrates that the proposed approach outperforms these contender algorithms. Finally, the proposed method is successfully applied to solve a real-world antenna array synthesis problem.