Large Language Models (LLMs) and Evolutionary Computation (EC) are increasingly being combined to support automated optimization, algorithm design, and adaptive decision-making. This survey reviews the bidirectional interaction between these two paradigms and examines how their complementary strengths can be leveraged in hybrid intelligent systems. First, we analyze how EC can enhance LLM-based systems through prompt optimization, hyperparameter tuning, and architecture search. Second, we review how LLMs can improve EC by supporting metaheuristic design, surrogate reasoning, adaptive operator control, and heuristic generation. We further discuss emerging co-adaptive frameworks in which LLMs and EC interact through iterative feedback loops. Beyond summarizing recent developments, the survey provides a structured perspective on interaction mechanisms, application patterns, and methodological challenges, including computational cost, reproducibility, interpretability, benchmarking, and generalization. The paper concludes by outlining open research questions and future directions for developing more robust, transparent, and scalable LLM-EC systems.

Large language models have enabled automated algorithm design (AAD) by generating optimization algorithms directly from natural-language prompts. While evolutionary frameworks such as LLaMEA demonstrate strong exploratory capabilities across the algorithm design space, their search dynamics are entirely driven by fitness feedback, leaving substantial information about the generated code unused. We propose a mechanism for guiding AAD using feedback constructed from graph-theoretic and complexity features extracted from the abstract syntax trees of the generated algorithms, based on a surrogate model learned over an archive of evaluated solutions. Using explainable AI techniques, we identify features that substantially affect performance and translate them into natural-language mutation instructions that steer subsequent LLM-based code generation without restricting expressivity. We propose LLaMEA-SAGE, which integrates this feature-driven guidance into LLaMEA, and evaluate it across several benchmarks. We show that the proposed structured guidance achieves the same performance faster than vanilla LLaMEA in a small controlled experiment. In a larger-scale experiment using the MA-BBOB suite from the GECCO-MA-BBOB competition, our guided approach achieves superior performance compared to state-of-the-art AAD methods. These results demonstrate that signals derived from code can effectively bias LLM-driven algorithm evolution, bridging the gap between code structure and human-understandable performance feedback in automated algorithm design.

Automated algorithm design is entering a new phase: Large Language Models can now generate full optimisation (meta)heuristics, explore vast design spaces and adapt through iterative feedback. Yet this rapid progress is largely performance-driven and opaque. Current LLM-based approaches rarely reveal why a generated algorithm works, which components matter or how design choices relate to underlying problem structures. This paper argues that the next breakthrough will come not from more automation, but from coupling automation with understanding from systematic benchmarking. We outline a vision for explainable automated algorithm design, built on three pillars: (i) LLM-driven discovery of algorithmic variants, (ii) explainable benchmarking that attributes performance to components and hyperparameters and (iii) problem-class descriptors that connect algorithm behaviour to landscape structure. Together, these elements form a closed knowledge loop in which discovery, explanation and generalisation reinforce each other. We argue that this integration will shift the field from blind search to interpretable, class-specific algorithm design, accelerating progress while producing reusable scientific insight into when and why optimisation strategies succeed.

Finding optimal solutions of conflicting objectives is a daily matter in many industrial applications, with multi-objective optimization trying to find the best solutions to them. The advent of quantum computing has led to researchers wondering if the promised exponential advantage can be obtained for these problems by variational quantum multi-objective optimization (QMOO) algorithm. Here, we improve it by introducing a Pareto Archiving and dominated solutions substitution, clearly improving in hyper-volume convergence at additional quantum and classical cost. We propose the use of RMNK-landscapes as a unifying testbed for benchmarking QMOO, as it is common in classical multi-objective field. By devising a generic classical-to-quantum mapping of these landscapes, we perform a numerical hyperparameter tuning of QMOO, significantly enhancing its performance. Finally, we compare QMOO against well-known classical solvers for multi-objective tasks, NSGA-II/III, showing comparable results in small instances. Our results demonstrate that QMOO, when carefully tuned for the task at hand, might be advantageous on harder problems than its classical counter-parts.

While contemporary Evolution Strategies handle integer optimization problems effectively, their adaptation mechanism is grounded in ℓ_2-based Gaussian models, which are not native to the integer lattice. In contrast, the ℓ_1-norm provides the natural measure of displacement on ℤ^n, with the double geometric distribution as its canonical mutation operator. In this work, we derive a fully ℓ_1-native step-size adaptation mechanism from first principles and propose an Integer Natural Evolution Strategy. We show that the DG distribution belongs to the exponential family, and that its sufficient statistic |z| yields a natural-gradient signal for dispersion adaptation. By accumulating this signal via an evolution path, we obtain a fading-memory online estimator of the natural gradient, following Ollivier (2018). This establishes that DG-based step-size adaptation arises directly from the statistical structure of the mutation distribution, rather than as a discrete analog of continuous ES mechanisms. Empirical results on integer quadratic benchmarks show that INES learns meaningful coordinate-wise step-sizes and is competitive with integer-handling CMA-ES baselines. Its advantages are most visible in high-dimensional Ellipsoidal problems and in robust convergence at larger dimensions.

Optimization modeling translates real decision-making problems into mathematical optimization models and solver-executable implementations. Although language models are increasingly used to generate optimization formulations and solver code, existing benchmarks are almost entirely text-only. This omits many optimization-modeling tasks that arise in operational practice, where requirements are described in text but instance information is conveyed through visual artifacts such as tables, graphs, maps, schedules, and dashboards. We introduce multimodal optimization modeling, a benchmark setting in which models must construct both a mathematical formulation and executable solver code from a text-and-visual problem specification. To evaluate this setting, we develop a solver-grounded framework that generates structured optimization instances, verifies each with an exact solver, and builds both the model-facing inputs and hidden reference files from the same verified source. We instantiate the framework as MM-OptBench, a benchmark of 780 solver-verified instances spanning 6 optimization families, 26 subcategories, and 3 structural difficulty levels. We evaluate 9 multimodal large language models (MLLMs), including 6 frontier general-purpose models and 3 math-specialized models, with aggregate, family-level, difficulty-level, and failure-mode analyses. The results show that the task remains far from solved: the best two models reach 52.1

In real-world applications, there is a fundamental problem: the data most critical to predict interesting events, anomalies, and high-stakes outliers are the rarest, while less interesting data is abundant. Although deep learning is deployed specifically for these difficult prediction tasks, data-driven models inevitably fail in underrepresented areas. This discrepancy between the empirical data- and the desired evaluation distribution is equivalent to a target distribution shift. The research field, termed Deep Imbalanced Regression (DIR), has emerged explicitly to address this challenge, which is particularly acute for continuous targets where most conventional classification-based methods are ill-suited. In this paper, we present the first comprehensive review of the DIR landscape, organized around a novel two-axis taxonomy that disentangles challenges along a Data Axis (target distribution shift, continuity, and density) and a Deep-Learning Axis (shared capacity, biased updates, and manifold distortion), where the latter captures a cascading failure mechanism through which deep models systematically neglect underrepresented targets. Within this framework, we systematically categorize and analyze 19 state-of-the-art methods spanning architectural, algorithm-level, and representation learning approaches, and empirically re-evaluate twelve of them with publicly available implementations under controlled, identical conditions. To stress-test generalization across the full target range, we introduce three novel targeted evaluation protocols, Balanced Extrapolation, Bimodal Interpolation, and Blind-Spot Isolation, that expose failure modes hidden by standard benchmarks (https://github.com/noah-puetz/deconstructing_deep_imbalanced_regression). Our study underscores the significant impact of imbalance on regression accuracy, offering a conceptual framework and practical benchmarks to catalyze further development of systems capable of capturing the rare as reliably as the common.

Benchmarking has driven scientific progress in Evolutionary Computation, yet current practices fall short of real-world needs. Widely used synthetic suites such as BBOB and CEC isolate algorithmic phenomena but poorly reflect the structure, constraints and information limitations of continuous and mixed-integer optimisation problems in practice. This disconnect leads to the misuse of benchmarking suites for competitions, automated algorithm selection and industrial decision-making, despite these suites being designed for different purposes. We identify key gaps in current benchmarking practices and tooling, including limited availability of real-world-inspired problems, missing high-level features and challenges in multi-objective and noisy settings. We propose a vision centred on curated real-world-inspired benchmarks, practitioner-accessible feature spaces and community-maintained performance databases. Real progress requires coordinated effort: A living benchmarking ecosystem that evolves with real-world insights and supports both scientific understanding and industrial use.

Deep reinforcement learning (DRL) has shown great promise in addressing multi-objective combinatorial optimization problems (MOCOPs). Nevertheless, the robustness of these learning-based solvers has remained insufficiently explored, especially across diverse and complex problem distributions. In this paper, we propose a unified robustness-oriented framework for preference-conditioned DRL solvers for MOCOPs. Within this framework, we develop a preference-based adversarial attack to generate hard instances that expose solver weaknesses, and quantify the attack impact by the resulting degradation on Pareto-front quality. We further introduce a defense strategy that integrates hardness-aware preference selection into adversarial training to reduce overfitting to restricted preference regions and improve out-of-distribution performance. The experimental results on multi-objective traveling salesman problem (MOTSP), multi-objective capacitated vehicle routing problem (MOCVRP), and multi-objective knapsack problem (MOKP) verify that our attack method successfully learns hard instances for different solvers. Furthermore, our defense method significantly strengthens the robustness and generalizability of neural solvers, delivering superior performance on hard or out-of-distribution instances.

Condition-based maintenance (CBM) offers an opportunity to improve classical preventive maintenance strategies by transitioning to a dedicated prognostics & health management (PHM) strategy for individual components. This means that time-based fixed maintenance intervals are being replaced by condition-dependent and component-specific interventions. For multi-component, complex systems, such a component-condition-based maintenance strategy yields multiple remaining useful life (RUL) prognostics. Maintainers must combine these into optimal maintenance plans that account for reliability, availability, maintainability, safety and costs (RAMS-C). Combining multiple RUL prognostics with the system's real-world dynamics, including architecture, anticipated future operating conditions, economic interdependence, stochastic interdependence and structural interdependence, yields complex maintenance planning decisions. To address this problem, several studies propose solutions based on multi-objective optimisation algorithms. However, these are often based on a priori knowledge and do not account for the day-to-day dynamics of operational variations. This paper presents a methodology for managing the complexity of maintenance scheduling under these conditions. The model is applied to the maintenance history of a Voith Maxima 30CC cargo locomotive. A priori maintenance models are compared with real-life implementation and optimised models using evolutionary multi-objective optimisation (EMO) to assess the rigour of the a priori plans under varying operating conditions. The study shows that applying such algorithms can significantly reduce costs while increasing overall system reliability and availability.

Automated machine learning (AutoML) systems search for pipelines within a space of preprocessing operators, learners, and hyper-parameters specified in advance: they can select and tune known components, but cannot produce structure outside that space. We present LACE, an AutoML framework that instead searches over complete executable pipeline programs: an evolutionary loop maintains a population of scikit-learn-compatible Python classes, and a large language model acts as the variation operator. To our knowledge, LACE is the first to formulate general tabular pipeline AutoML this way, evaluated on standardized OpenML tasks under a leakage-controlled protocol that withholds dataset identity from the generator. Because every candidate is ordinary Python, the returned pipeline and the search that produced it can be inspected and edited directly, rather than only through a framework's model objects. On 68 OpenML classification tasks, LACE with GPT-5.4-mini significantly outperforms auto-sklearn, H2O, and a fixed XGBoost baseline, with no detectable difference against AutoGluon, the strongest search-based system evaluated, while covering the full benchmark. Newer tabular foundation models are more accurate on the subset of tasks they support, but apply a fixed pretrained predictor rather than returning an editable task-specific program. LACE's contribution is therefore not raw accuracy but a search space defined by code: complete coverage, pipelines practitioners can reuse directly, and a component set extended by editing the prompt rather than the framework.

Virtual sensing aims to infer hard-to-measure quantities from accessible measurements and is central to perception and control in physical systems. Despite rapid progress from first-principle and hybrid models to modern data-driven methods research remains siloed, leaving no established default approach that transfers across processes, modalities, and sensing configurations. We introduce MuViS, a domain-agnostic benchmarking suite for multimodal virtual sensing that consolidates diverse datasets into a unified interface for standardized preprocessing and evaluation. Using this framework, we benchmark established approaches spanning gradient-boosted decision trees and deep neural network (NN) architectures, and show that none of these provides a universal advantage, underscoring the need for generalizable virtual sensing architectures. MuViS is released as an open-source, extensible platform for reproducible comparison and future integration of new datasets and model classes.

Gradient-free black-box optimization (BBO) is widely used in engineering design and provides a flexible framework for topology optimization (TO), enabling the discovery of high-performing structural designs without requiring gradient information from simulations. Yet, its success depends on two key choices: the geometric parameterization defining the search space and the optimizer exploring it. This study investigates this interplay through a compliance minimization problem for a cantilever beam subject to a connectivity constraint. We benchmark three geometric parameterizations, each combined with three representative BBO algorithms: differential evolution, covariance matrix adaptation evolution strategy, and heteroscedastic evolutionary Bayesian optimization, across 10D, 20D, and 50D design spaces. Results reveal that parameterization quality has a stronger influence on optimization performance than optimizer choice: a well-structured parameterization enables robust and competitive performance across algorithms, whereas weaker representations increase optimizer dependency. Overall, this study highlights the dominant role of geometric parameterization in practical BBO-based TO and shows that algorithm performance and selection cannot be fairly assessed without accounting for the induced design space.

The practical deployment of federated learning (FL) on resource-constrained devices is fundamentally limited by the high cost of training large models and the instability caused by heterogeneous [non-independent and identically distributed (non-IID)] client data. Conventional pruning methods often treat data heterogeneity as a problem to be mitigated. In this work, we introduce a paradigm shift: we reframe client diversity as a feature to be harnessed. We propose AutoFLIP, a framework that begins not with training, but with a one-time federated loss exploration. During this phase, clients collaboratively build a map of the collective loss landscape, using their diverse data to reveal the problem's essential structure. This shared intelligence then guides an adaptive pruning strategy that is dynamically refined by client agreement throughout training. This approach allows AutoFLIPto identify robust and efficient subnetworks from the outset. Our extensive experiments show that AutoFLIPreduces computational overhead by an average of 52% and communication costs by over 65% while simultaneously achieving state-of-the-art (SoTA) accuracy in challenging non-IID settings.

Structural bias (SB) refers to systematic preferences of an optimisation algorithm for particular regions of the search space that arise independently of the objective function. While SB has been studied extensively in single-objective optimisation, its role in multi-objective optimisation remains largely unexplored. This is problematic, as dominance relations, diversity preservation and Pareto-based selection mechanisms may introduce or amplify structural effects. In this paper, we extend the concept of structural bias to the multi-objective setting and propose a methodology to study it in isolation from fitness-driven guidance. We introduce a suite of synthetic multi-objective test problems with analytically controlled Pareto fronts and deliberately uninformative objective values. These problems are designed to decouple algorithmic behaviour from problem structure, allowing bias induced purely by algorithmic operators and design choices to be observed. The test suite covers a range of Pareto front shapes, densities and noise levels, enabling systematic analysis of different manifestations of structural bias. We discuss methodological challenges specific to the multi-objective case and outline how existing SB detection approaches can be adapted. This work provides a first step towards behaviour-based benchmarking of multi-objective optimisers, complementing performance-based evaluation and informing more robust algorithm design.

Stochastic multi-objective optimization (SMOOP) requires ranking multivariate distributions; yet, most empirical studies perform scalarization, which loses information and is unreliable. Based on the optimal transport theory, we introduce the center-outward q-dominance relation and prove it implies strong first-order stochastic dominance (FSD). Also, we develop an empirical test procedure based on q-dominance, and derive an explicit sample size threshold, n^*(δ), to control the Type I error. We verify the usefulness of our approach in two scenarios: (1) as a ranking method in hyperparameter tuning; (2) as a selection method in multi-objective optimization algorithms. For the former, we analyze the final stochastic Pareto sets of seven multi-objective hyperparameter tuners on the YAHPO-MO benchmark tasks with q-dominance, which allows us to compare these tuners when the expected hypervolume indicator (HVI, the most common performance metric) of the Pareto sets becomes indistinguishable. For the latter, we replace the mean value-based selection in the NSGA-II algorithm with q-dominance, which shows a superior convergence rate on noise-augmented ZDT benchmark problems. These results establish center-outward q-dominance as a principled, tractable foundation for seeking truly stochastically dominant solutions for SMOOPs.

Fuel cell electric vehicles (FCEVs) are a promising solution for reducing global emissions. As the number of FCEVs grows, operations at hydrogen refueling stations (HRSs) must be optimized to support the hydrogen mobility ecosystem. This is challenging because station operators face multiple sources of uncertainty, such as fluctuating electricity prices and variable hydrogen demand, which are further exacerbated when renewable energy sources are integrated. The nonlinear behavior of hydrogen production technologies also makes it difficult to formulate tractable decision models. A variety of optimization approaches for HRSs operation have been proposed in the literature. These studies differ in how they formulate the problem, in the objectives and constraints they consider, and in the solution methods they employ. For example, heuristics and metaheuristics are often used for offline optimization problems, while learning-based optimization methods are increasingly applied to online problems, especially when the problem size is large. In this paper, we review component-level modeling techniques and optimization-based operational strategies proposed in the literature, and summarize how HRSs operation can be modeled and solved as an optimization problem that accounts for these uncertainties and nonlinearities. Moreover, based on existing research, we identify research gaps in this domain and outline directions for future work.

Unsupervised feature selection is commonly formulated as a multiobjective optimisation problem that jointly optimises subset quality and subset size. Yet the behaviour of this formulation depends critically on the choice of evaluation objective, the direction of subset-size regularisation, and the initialisation strategy. We study these factors in a controlled setting using a synthetic dataset with known informative, redundant, and irrelevant feature types. Six formulations are compared by combining three evaluation objectives: accuracy, silhouette score, and PCA reconstruction loss with subset-size minimisation or maximisation. The results show that formulation strongly affects both search dynamics and the quality of the resulting Pareto front. Silhouette-based formulations exhibit a strong bias toward trivial low-cardinality solutions and remain weak proxies for predictive performance. In contrast, the proposed PCA loss objective produces compact subsets with test accuracy comparable to subsets obtained by directly optimising supervised accuracy. Overall, the study shows that the design of objective(s) is central to effective multiobjective unsupervised feature selection.

Benchmarking in continuous black-box optimisation is hindered by the limited structural diversity of existing test suites such as BBOB. We explore whether large language models embedded in an evolutionary loop can be used to design optimisation problems with clearly defined high-level landscape characteristics. Using the LLaMEA framework, we guide an LLM to generate problem code from natural-language descriptions of target properties, including multimodality, separability, basin-size homogeneity, search-space homogeneity and global–local optima contrast. Inside the loop we score candidates through ELA-based property predictors. We introduce an ELA-space fitness-sharing mechanism that increases population diversity and steers the generator away from redundant landscapes. A complementary basin-of-attraction analysis, statistical testing and visual inspection, verifies that many of the generated functions indeed exhibit the intended structural traits. In addition, a t-SNE embedding shows that they expand the BBOB instance space rather than forming an unrelated cluster. The resulting library provides a broad, interpretable, and reproducible set of benchmark problems for landscape analysis and downstream tasks such as automated algorithm selection.

Black-box optimization is increasingly used in engineering design problems where simulation-based evaluations are costly and gradients are unavailable. In this context, the optimization community has largely analyzed algorithm performance in context-free setups, while not enough attention has been devoted to how problem formulation and domain knowledge may affect the optimization outcomes. We address this gap through a case study in the topology optimization of laminated composite structures, formulated as a black-box optimization problem. Specifically, we consider the design of a cantilever beam under a volume constraint, intending to minimize compliance while optimizing both the structural topology and fiber orientations. To assess the impact of problem formulation, we explicitly separate topology and material design variables and compare two strategies: a concurrent approach that optimizes all variables simultaneously without leveraging physical insight, and a sequential approach that optimizes variables of the same nature in stages. Our results show that context-agnostic strategies consistently lead to suboptimal or non-physical designs. In contrast, the sequential strategy yields better-performing and more interpretable solutions. These findings underscore the value of incorporating, when available, domain knowledge into the optimization process and motivate the development of new black-box benchmarks that reward physically informed and context-aware optimization strategies.