Real-world multi-objective optimization problems usually have multiple constraints. To solve constrained multi-objective optimization problems (CMOPs), researchers have proposed various constrained multi-objective evolutionary algorithms (CMOEAs). These CMOEAs explicitly or implicitly assume that there are large infeasible regions in the objective space between the initial population and the constrained Pareto front (CPF), and that the CPF is close to the unconstrained Pareto front (UPF). Thus, some special techniques, such as the push-and-pull strategy and the coevolution strategy, are used to traverse large infeasible regions to find the UPF. However, it is unclear whether these characteristics also exist in real-world CMOPs, and the state-of-the-art algorithms performing well on artificial CMOPs also work well on real-world CMOPs. This paper first compares the performance of fifteen CMOEAs on artificial and real-world CMOPs. It is observed that performance comparison results on real-world CMOPs are clearly different from those on artificial CMOPs. This observation shows the importance of using real-world CMOPs in benchmarking studies. Next, the comparison results are analyzed by using the difference between artificial and real-world CMOPs. It is observed that most of the examined real-world CMOPs do not have large infeasible regions near the CPFs, and the CPFs of some real-world CMOPs are not close to the UPFs. It is also observed that feasible solutions of some real-world CMOPs are difficult to obtain, and NSGA-II performs better than recently-proposed CMOEAs on these difficult real-world problems. This observation suggests the necessity of strong search ability to find feasible solutions for real-world applications.
Neural networks are commonly regarded as black boxes, necessitating post-hoc Explainable AI (XAI) methods. However, different feature attribution techniques applied to the same model and data often yield divergent explanations, undermining the reliability of any single method. This work proposes a model-centric approach that moves beyond evaluating individual XAI methods, focusing instead on optimizing neural network architectures toward cross-method consensus via surrogate-based multi-objective hyperparameter tuning. Cross-method XAI consensus, defined as the degree to which diverse attribution techniques yield convergent explanations for a given architecture, is formalized as a novel optimization objective. Consensus is operationalized by aggregating local attributions into global feature importance profiles, standardizing them, and quantifying convergence via the Euclidean distance. This score is integrated as a second objective alongside predictive loss into a Kriging-based Bayesian optimization framework using desirability functions. Empirical evaluation across 14 diverse datasets confirms that explicitly tuning for explainability is feasible. The framework reveals a prevalent performance-explainability trade-off in most in-distribution settings, while analysis of tuning trajectories and selected architectures reveals exploitable structure in the hyperparameter-to-consensus landscape. These findings introduce cross-method XAI consensus as a practical, model-level criterion for neural architecture selection and provide a quantitative basis for analyzing the performance-explainability trade-off.
This study introduces sPPO-CR, a policy-gradient deep reinforcement learning (DRL) algorithm for multi-objective (MO) and multi-condition multi-objective (MCMO) optimization. The method builds on a simplified single-step variant of Proximal Policy Optimization (PPO) and incorporates a Chebyshev-based reproduction mechanism with a log-likelihood rejection filter to safely integrate synthetic off-policy samples (sPPO-CR). sPPO-CR is benchmarked against sTD3-CR, a single-step variant of TD3, and three evolutionary baselines (NSGA-II, NSGA-III, and MOEA/D) across mathematical benchmarks and an airfoil shape optimization task. Results show that while evolutionary algorithms remain effective in standard MO settings, DRL methods, particularly sPPO-CR, offer improved performance in certain higher-dimensional and condition-dependent problems through effective generalization across both the design and condition spaces. Moreover, sPPO-CR achieves similar or superior performance to sTD3-CR while requiring an order of magnitude fewer network updates. Overall, the findings demonstrate that DRL-based optimizers provide a scalable and computationally efficient alternative for solving complex multi-condition multi-objective problems.
Training a neural network is an optimisation task that is commonly solved using gradient-based techniques, such as stochastic gradient descent or Adam. There are, however, scenarios in which black-box optimisation approaches, such as evolutionary algorithms, may be appropriate for optimising neural network weights. In this study, we view neural network weight optimisation as a black-box problem and analyse the search space using exploratory landscape analysis (ELA). The purpose is to understand what makes neural network search spaces different from other bound-constrained, continuous-valued problems commonly used to benchmark evolutionary algorithms. We perform an instance space analysis on a combined set of neural network training instances and instances from the widely used Black-Box Optimisation Benchmarking (BBOB) problem suite. The analysis included instances with 41, 261, and 481 decision variables, corresponding with 1-, 3-, and 5-layer networks. We show that neural-network training instances and BBOB instances occupy distinct regions of the ELA feature space. While the landscapes of BBOB instances typically exhibit strong global structure and large differences between local and global optima, neural network instances exhibit ill-conditioned landscapes with very weak global structure. Surprisingly, these instances were easier for black-box algorithms to find acceptable solutions than BBOB instances, likely due to the abundance of high-quality solutions. Another key and unexpected finding was that the features of the neural network error landscapes were more strongly influenced by the choice of activation function and the network size than by the underlying machine learning task, in this case, the regression function to be fitted.
Optimization algorithms contain parameters that greatly influence their behavior. Finding the right settings for parameters through automated algorithm configuration has become a critical component of designing competitive algorithms. While traditional offline configurators tackle this problem by finding one configuration that works well for a set of instances, instance-specific algorithm configuration utilizes features of the instances to provide configurations that are tailored to each instance to maximize performance. We propose the first instance-specific algorithm configurator based on deep reinforcement learning that can be used in general algorithm configuration settings. Our method is able to handle large, mixed, discrete and continuous search spaces and only requires a small number of instances for training. Not only does it select an individual configuration for every instance, it also selects configurations from a much broader range. We show that our configurator provides improvements over the state-of-the-art instance-specific configurators ISAC++ and Hydra on a wide range of problem domains.
