In conventional fitness-based Genetic Programming (GP), parent selection is driven mainly by fitness and has limited access to relationships between programs beyond their observed performance. This work investigates whether pretrained code embeddings can provide complementary guidance for parent selection in symbolic regression. We introduce two mechanisms that combine fitness with distances computed in the latent spaces of CodeBERT and Code2Vec. Probabilistic Embedding-Guided Selection (PEGS) alternates between fitness-based and embedding-based selection under fixed or dynamically scheduled pressure. Two-Stage Embedding-Guided Selection (TSEGS) applies fitness selection followed by an embedding-based criterion aimed either at intensifying the search around elite solutions or at preserving diversity. The methods are evaluated on reduced SRBench datasets against standard GP and, in a targeted experiment, an NSGA-II baseline with constant optimization. Results show that embedding-derived information can improve held-out test performance, but its effectiveness depends on the embedding model, selection mechanism, and role assigned to distances. The diversity-oriented TSEGS criterion is most consistently beneficial with Code2Vec, whereas other configurations show more limited or problem-dependent effects. Model-size effects are mixed, and in the targeted NSGA-II experiment the gains are accompanied by larger median trees. A representative-run analysis further indicates that embedding-guided selection does not uniformly increase exploration. Overall, pretrained code embeddings provide a potentially useful but non-universal signal for GP selection, with benefits that must be weighed against configuration sensitivity, model-size effects, and substantial, implementation-dependent computational overhead.
This article reviews EvoGym, a lightweight Python framework for simulating voxel-based robots in two-dimensional environments. It describes the simulator's core features, including its Gymnasium-compatible API, diverse benchmark tasks, and flexible tools for designing custom environments. The review highlights EvoGym's accessibility, computational efficiency, and suitability for research and education in evolutionary robotics and reinforcement learning. Overall, EvoGym is a practical platform for research and education in embodied AI.
Stock market prediction is a key area in financial time series analysis and quantitative finance. The emergence of deep learning models, e.g., LSTM (Long Short-Term Memory) and BiLSTM (Bidirectional LSTM), have brought breakthroughs to stock market prediction. Nevertheless, their performance is highly dependent on hyperparameter configuration (e.g., layer number and learning rate) by users. Evolutionary algorithms like GA (Genetic Algorithm) address this issue by optimizing neural network setups, but their high computational cost (mainly from fitness evaluation) limits application. In addition, LSTM variants have inherent limitations, e.g., information decay in long-sequence processing. Therefore, this work proposes GAds_saBiLSTM (GA with dynamic surrogate for self-attention enhanced BiLSTM optimization) for stock price forecasting. Specifically, BiLSTM architecture/hyperparameters are co-optimized via GA to replace manual tuning; a dynamic surrogate model is designed to cut GA computational overhead; a self-attention module is integrated into the baseline BiLSTM to mitigate information decay. On two benchmark stock datasets (CSI 300 and S P 500), the proposed method is compared against various time series prediction methods across evaluation metrics (MAE, MSE, and R^2 ). Results show that the evolved saBiLSTM models by the proposed method achieve the lowest prediction error and highest fitting degree (e.g., R^2 =0.8240 on CSI 300 and R^2 =0.9865 on S P 500). Ablation experiments confirm that the dynamic surrogate model drastically reduces computational overhead (e.g., cutting GPU training hours from 2.60 (GA-BiLSTM) to 1.03 (GAds-BiLSTM) on the CSI 300); while the integration of the self-attention module yields higher model accuracy with negligible efficiency costs.
Schema Theory offers a principled lens for analyzing the dynamics of Evolutionary Algorithms (EAs), yet its extension to Genetic Programming (GP) is obstructed by the nonlinear structure of GP trees and the irregular correspondence between syntax and semantics. These characteristics prevent classical, structure-based schema formulations from capturing the mechanisms that determine how information is preserved, disrupted, and propagated during GP evolution. Motivated by the significant role of semantics in GP, this study introduces Valued Schema Theory (VST), which characterizes a schema through both its semantic output and the quantity of effective genetic material it carries. Beyond providing a semantic definition of schemata, the proposed theory models the flow of value through GP populations. It describes schema dynamics through a pessimistic survival inequality that integrates selection pressure, crossover-induced structural disruption, and the differing robustness of significant meaning and zero-valued regions. This formulation yields a tractable account of how meaningful information spreads while non-informative regions function as protective buffers. The novelty of VST lies in redefining schema as a semantic-valued unit characterized by output behavior and effective semantic material, while preserving the disruption-centered form of pessimistic schema analysis to maintain the explainability of Building Blocks. Empirical evaluation across four representative benchmark tasks covering Boolean regression, numerical symbolic regression, and symbolic-regression-like classification shows that VST achieves consistently high accuracy in predicting schema-frequency transitions. These results indicate that VST captures the microscopic mechanisms through which semantic information is redistributed during GP evolution, providing a coherent account of GP’s underlying search dynamics.
Grammatical Evolution (GE) has demonstrated broad success across various domains, from automated circuit synthesis to explainable machine learning. Traditional GE systems, which rely on Context-Free Grammars (CFGs) for genotype-to-phenotype mapping, ensure syntactic correctness but inherently lack mechanisms to enforce semantic constraints. While Attribute Grammars (AGs) were introduced to GE to address this by embedding semantic checks into grammar productions, their rigid structure and computational overhead restrict adaptability to dynamic, context-sensitive requirements. This paper introduces Dynamically Attributed Grammatical Evolution (DAGE), a framework that augments CFGs by allowing the use of functions as non-terminals within the grammar. This enables direct function invocation during genotype-to-phenotype mapping, making semantic checks more straightforward and facilitating runtime-aware behaviour that traditional AGs struggle to capture. By embedding functions directly into the grammar, DAGE can perform on-the-fly constraint checking, adapt to runtime conditions, and interact with external systems as needed. This leads to more expressive representations tailored to complex requirements. Furthermore, offloading complexity into reusable functions simplifies grammar design and maintenance. DAGE is implemented in Python, leveraging the GRAPE and DEAP frameworks. Empirical evaluations on symbolic regression, multiple knapsack, artificial ant, and program synthesis benchmarks show that DAGE adapts well to domains requiring semantic or context-sensitive checking. The results show consistent benefits on constraint-heavy construction tasks. As expected, this comes with a trade-off between improved semantic control and additional mapping overhead.
We report the solution of a benchmark set of ordinary differential equations (ODEs) with genetic programming (GP) within a collocation framework using numerical tuning of the embedded tree constants. Alongside a conventional soft penalty formulation, we also report results from two GP variants that enforce the initial conditions on the ODEs as hard constraints: the first uses the so-called death penalty while the second employs a novel ranking method that orders infeasible individuals using Pareto dominance according to the degree to which they violate the constraints. We investigate the influence of the numbers of collocation points used to solve the problem, and conclude that a few ODEs require more than 10–20 points, otherwise the number of points is not critical. A statistical comparison of the different methods indicates that only a few ODEs display differences, an observation we attribute to the influence of parameter tuning. We obtain highly accurate solutions for all the benchmark ODEs, but identify a problem with certain of the ODEs producing trivial solutions, which we are able to mostly mitigate by introducing an additional constraint on the mean squared amplitude of the evolved solutions. Overall, we infer that the properties of the individual ODEs can impact the solution process.
Languages for describing two-dimensional (2-D) structures have become powerful tools across multiple fields, including pattern recognition, image processing, and the modeling of physical and chemical phenomena. One of such structures is labeled polyominoes, i.e., geometric shapes formed by connected unit squares arranged on a 2-D grid. In previous work, we introduced: (a) a novel grammar-based approach for defining sets of labeled polyominoes that satisfy predefined requirements, and (b) an algorithm to develop labeled polyominoes following the rules of the proposed grammar. We demonstrated that these two components enable optimization within the space of labeled polyominoes, similarly to how grammatical evolution and its extensions operate in string-based search spaces. In this work, we extend our previous approach to a new domain: the evolution of modular soft robots, namely, voxel-based soft robots (VSRs). We evolve VSRs for the task of energy-efficient locomotion, while constraining their physical structure to adhere to a given grammar. We show that the evolved robots successfully perform their assigned tasks and do have the required structure. These results highlight the potential of integrating domain knowledge through grammars to guide the evolutionary design of complex structure as modular soft robots.
The Semantic Learning algorithm based on Inflate and deflate Mutation (SLIM) represents a recent advancement in Geometric Semantic Genetic Programming (GSGP). While maintaining the unique feature of generating a unimodal error surface for all supervised learning tasks, SLIM has the ability of generating models that remain sufficiently compact for human interpretation. In its initial definition, SLIM was characterized by two mutation operators: the traditional geometric semantic mutation, called inflate mutation because it produces offspring larger than their parents, and a new deflate mutation that has the ability of creating smaller offspring, offering a novel approach to managing model complexity. Besides deepening and interpreting in greater depth the experimental results, this work further enriches the foundational concepts of SLIM by integrating a new crossover operator, that, contrarily to the traditional geometric semantic crossover, is able to generate individuals of small size. Our comprehensive analysis explores the wider implications of this innovative operator. The novel variant that integrates this crossover is named Semantic Learning algorithm with Inflate/deflate Mutations and MEaningful Recombination (SLIMMER). Experimental results provide strong support for the potential of both SLIM and SLIMMER as effective approaches worthy of further research. As its name suggests, for some test cases SLIMMER demonstrates an enhanced ability to produce even more compact models than SLIM, further reinforcing its promise for applications where interpretability and model simplicity are essential.
Scheduling is a fundamental component of dynamic and complex manufacturing systems, coordinating resources efficiently and ensuring timely production. However, designing efficient scheduling rules to maximize delivery performance and resource allocation is challenging due to uncertainty in job arrivals, machine status, and routing changes. Existing Genetic Programming (GP) approaches can automatically evolve scheduling rules but remain limited by their dependence on simulation models, extensive data requirements, and limited adaptability to changing conditions. The goal of this research is to overcome the above challenges by developing the first Online Genetic Programming (OGP) framework that learns scheduling strategies directly within the operating environment and without relying on prior knowledge or an explicit simulation models. The novelty of this research lies in the development of an adaptive fitness function that combines real-time performance feedback with predictive evaluation from a phenotypic archive, allowing the search process to balance short-term adaptability and long-term learning stability. A pre-selection strategy further refines candidate solutions while controlling rule complexity, and a soft restart mechanism sustains diversity during extended evolutionary runs. Dynamic flexible job shop scheduling problems (DFJSP) were used as representative test environments to evaluate the method’s effectiveness. Experimental results on DFJSP demonstrate that OGP outperforms existing scheduling algorithms when jointly considering scheduling and routing decisions. When used as an automated heuristic design method, the proposed method can generate competitive rules compared to the state-of-the-art genetic programming methods in terms of test performance and the size of evolved rules. These findings highlight OGP as a robust and generalisable optimisation framework for dynamic decision-making in changing environments.
Medical image classification is pivotal for computer-aided diagnosis, enabling automated and accurate disease detection or severity grading. However, high inter-class similarity, imbalanced data, and limited annotated images often limit performance on important classes, ultimately affecting the overall performance. In this paper, a novel genetic programming (GP) algorithm is proposed, where the selection method in crossover and the mutation operator are modified to allocate increased search pressure to more challenging classes. Specifically, the crossover operation with a new selection method is designed to increase the likelihood that better-performing individuals in the challenging class exchange their good genetic materials, while the new mutation operator mutates worse-performing individuals under the guidance from the better-performing ones, thereby steering the search toward more promising regions. Experimental results on six diverse medical image datasets demonstrate the superiority of the proposed algorithm over nine baseline methods. Further analysis highlights the potential of the new genetic operators to enhance the overall classification performance by targeting challenging classes. Moreover, the analysis of an evolved GP individual illustrates the potential interpretability of GP.
Space, while inherent to the natural world, often finds itself omitted in bio-inspired computational system designs. Spatial genetic programming (SGP) is a GP paradigm that includes space as a fundamental dimension to evolve along with linear genetic programming programs. Here, the spatial blueprint dictates the program’s execution sequence. Despite prior SGP iterations showcasing good performance in solving decision-making problems, the complete spatial mode’s efficacy and the consequential spatial ramifications remain ambiguous. This study embarks on a two-pronged approach: We start with SGP’s application to a comprehensive subset of symbolic regression problems, taken from Feynman’s physics lectures, then deepen the study by an extensive analysis to show the spatial dimension’s impact on the evolution of SGP models. Our intention is to primarily focuses on the spatial dimension’s influence on generational diversity and the emergence of spatially-induced localization within the system. We propose a suite of spatial evolutionary operators that should offer insights into leveraging SGP as a tool to examine spatial impacts within problem-solving techniques. Preliminary results indicate that spatial constructs can indeed serve as a leverage to enhance the evolution of better models.
Artificial intelligence techniques are increasingly being applied to solve control problems, but often rely on black-box methods without transparent output generation. To improve the interpretability and transparency in control systems, models can be defined as white-box symbolic policies described by mathematical expressions. For better performance in partially observable and volatile environments, the symbolic policies are extended with memory represented by continuous-time latent variables, governed by differential equations. Genetic programming is used for optimisation, resulting in policies consisting of symbolic expressions. Our results show that symbolic policies with memory compare with black-box policies on a variety of control tasks. Furthermore, the benefit of the memory in symbolic policies is demonstrated on experiments where memory-less policies fall short. Overall, we present a method for evolving high-performing symbolic policies that offer better interpretability and transparency, which lacks in black-box models.
In this paper, we present an combination of grammatical evolution and least squares support vector regression for solving the Falkner-Skan equation, which arises in real-world and engineering applications, symbolically. Properly selecting the transformation functions is crucial for utilizing least squares support vector regression, as it directly impacts the accuracy of the solutions. In this study, the transformation functions are determined using a powerful genetic programming algorithm called grammatical evolution. The experimental results demonstrate that the proposed method is a capable and powerful approach for solving nonlinear ordinary differential equations.
Generation constructive hyper-heuristics have proven to be very effective at creating construction heuristics for combinatorial optimization problems, with the heuristics derived by these hyper-heuristics often outperforming human-derived heuristics. Genetic programming has been predominantly used by generation constructive hyper-heuristics. This study investigates the effectiveness of two emerging technologies in genetic programming, namely, transfer learning and structure-based genetic programming, in genetic programming constructive generation hyper-heuristics. The study investigates whether structure-based genetic programming in generation constructive hyper-heuristics with (SBGP-HH-TL) and without transfer learning (SBGP-HH). The hyper-heuristics were evaluated on three problem domains, namely, the examination timetabling problem, the one-dimensional bin packing problem and the capacitated vehicle routing problem. Both SBGP-HH-TL and SBGP-HH outperformed the generational hyper-heuristic employing canonical genetic programming (CGP-HH) on a majority of the problem instances for the three problem domains, with SBGP-HH-TL outperforming SBGP-HH. Hence, the study has revealed that both transfer learning and structure-based genetic programming have resulted in performance improvements in genetic programming generation constructive hyper-heuristics for combinatorial optimization.
Multi-objective genetic programming-based feature construction has emerged as a powerful approach to enhancing regression performance. While this approach has demonstrated considerable success across various domains, overfitting remains a significant challenge, particularly when dealing with limited and/or noisy training data. To address this, researchers have developed multi-objective methods that balance model complexity and accuracy. However, these methods invariably produce a Pareto front containing multiple solutions with different trade-offs, raising the critical question of how to select the most appropriate model for deployment. In this paper, we propose a novel Adaptive Complexity Knee Point (ACKP) selection strategy for evolutionary multi-objective feature construction in regression tasks. Our approach adaptively selects between minimal complexity knee points, which favor the simplest knee solution, and traditional knee points, based on estimated dataset difficulty. This adaptive mechanism reduces overfitting in noisy scenarios while mitigating underfitting in low-noise datasets by allowing more complex models when appropriate. Comprehensive experiments on 58 real-world datasets demonstrate that ACKP significantly outperforms nine established model selection strategies, and that genetic programming with ACKP outperforms several mainstream machine learning algorithms, particularly when dealing with sample-limited and noisy datasets, highlighting its practical value in real-world problems.
The incorporation of a Cellular Automata (CA)-like structure into the population of Evolutionary Algorithms (EAs) has been shown to enhance solution quality. However, research on CA-like structures in the context of Genetic Programming (GP) remains limited. This work examines the impact of introducing such structures in Geometric Semantic variants of GP, specifically focusing on the well-established Geometric Semantic GP (GSGP) and the recently proposed SLIM-GSGP, which prioritizes generating smaller and more interpretable individuals. Furthermore, we analyze how cellular structures influence the effectiveness of semantic-based recombination and mutation in both GSGP and SLIM-GSGP. To this end, we conduct a comprehensive evaluation of these genetic operators, examining their effects both individually and in combination. We provide insights into how CA-like structures and semantic genetic operators influence both the quality and size of solutions in GSGP and SLIM-GSGP, offering a clear understanding of the trade-offs associated with these approaches.
The container relocation problem is a critical combinatorial optimisation problem in warehouses and container ports. The goal is to retrieve all containers while minimising unnecessary relocations. As this problem is NP-hard, various heuristics have been proposed, including relocation rules (RRs), simple constructive heuristics that iteratively build solutions by determining how containers should be relocated within the yard for efficient retrieval. However, manually designing effective RRs is challenging, leading to the use of genetic programming to generate them automatically. A key limitation of both manually and automatically designed RRs is their restricted problem view and limited decision-making scope. This often results in suboptimal relocations, negatively impacting future operations and overall efficiency. A crucial aspect of RR design is defining effective relocation schemes that enhance decision-making by considering the long-term impact of relocations. This study investigates several relocation schemes that provide RRs with lookahead capabilities, enabling them to anticipate future consequences and make more informed moves. In addition to two standard schemes, four novel relocation schemes are introduced and evaluated using an established problem set. The results demonstrate that properly adapting relocation schemes can significantly enhance the performance of automatically designed RRs, leading to significantly better results.
