Grid-based random walk crossover (GBRWX) for genetic algorithms (GAs) is proposed. In contrast to traditional crossover operators such as one-point, two-point, and uniform crossover, which exchange genes only at fixed positions in the parent chromosomes, GBRWX allows genes to be copied from one position in a parent and placed in a different position in the offspring. This approach mimics biological transposition, where genes can move within or between chromosomes. More specifically, GBRWX arranges two parent chromosomes into a two-dimensional grid and generates offspring through a random walk guided by Chebyshev distance, encouraging the inheritance of adjacent genes on the grid while allowing the preservation of gene sequences. As a result, GBRWX can produce offspring that traditional crossover operators are not able to generate. The effectiveness of GBRWX is evaluated against different crossover operators on both binary and real-valued optimization problems. Experimental results show that GBRWX leads to better solutions, faster convergence, and greater population diversity. Notably, it successfully solves the deceptive Trap problem, while traditional crossover operators fail to do so. This opens up opportunities to explore adaptive or alternative traversal heuristics in the random walk crossover process tailored to specific problems.

Euler and Venn diagrams are widely used to visualize set-based data, but generating diagrams with desired properties (e.g., well-matchedness, well-formedness, symmetric diagram, and congruent shape) remains challenging. Existing methods typically aim to generate a single diagram with a fixed set of properties, offering users limited flexibility to choose which properties to consider or to explore trade-offs among solutions. To address this limitation, we present EvoEuler, a Python package that uses a multi-objective evolutionary algorithm (NSGA-II or NSGA-III) to search for Euler and Venn diagrams with specific properties using a chosen simple closed shape (e.g., circle, ellipse, or rectangle). EvoEuler provides both a web interface and an application programming interface (API). To the best of our knowledge, EvoEuler is the first method that (1) returns a set of non-dominated solutions (diagrams) in the context of both Euler and Venn diagram generation, with solutions evaluated using objectives that quantify violations of well-matchedness and well-formedness properties, and (2) allows solutions to be generated with optional constraints, namely symmetric diagram and congruent shape, in the context of Euler diagram generation. We conduct experiments to evaluate four configurations of EvoEuler on generating Euler and Venn diagrams. Our results suggest that rectangles are preferred over ellipses for generating Euler diagrams, and show that EvoEuler can find simple Venn diagrams for up to five sets.

In many real optimisation scenarios, instances arrive in a stream and an algorithm selector is used to select the most appropriate solver for a given instance. However, if the data distribution of instances in the stream changes over time, new solvers may be required at some point downstream to continue to provide the best solutions to new instances. This requires a selector to be periodically updated to incorporate new class labels while at the same time, continuing to perform well on previously seen data (i.e. avoid ‘catastrophic forgetting’). Class Incremental Learning (CIL) techniques are designed to deal with this situation, but although commonly used in machine-learning, have rarely been studied in the context of algorithm selection in an optimisation setting. To address this gap, we benchmark 8 CIL methods with respect to their ability to withstand catastrophic forgetting using instances from a commonly used benchmark in continuous optimisation. We find that rehearsal-based CIL methods that save exemplars of previous data and use them when retraining significantly outperform other methods. While there is some evidence of forgetting, the loss is small at around 7% . Overall accuracy of the final model at the end of the stream is ≥ 91% . Hence, these methods appear to be a viable approach to continual learning in streaming optimisation scenarios on continuous optimisation benchmarks.

In many real-world scenarios, instances arrive in a stream whose characteristics change over time, i.e. the stream is subject to data-drift. The ability to detect and respond to drift is crucial to optimise performance over the stream. However, there is currently no suitable stream-generator on which to evaluate the aforementioned methods. We address this by proposing SCOBench: a fully configurable generator that can generate streams of instances with different drift patterns specified by the user. In contrast to existing stream generators that define drift as a shift in the location of the optima over time of a single instance, SCOBench generates a stream of new instances from multiple functions, while drift between consecutive instances can be defined according to a change in three different metrics (landscape features, probing-trajectories and optima location). We verify that SCOBench correctly generates the intended drift patterns, then demonstrate that algorithms perform differently over streams, highlighting the need for drift detection and algorithm-selection per instance. SCOBench is fully integrated with IOHexperimenter for ease of use. All code and documentation are available as open-source.

Coupling Large Language Models (LLMs) with Evolutionary Algorithms has recently shown significant promise as a technique to design new heuristics that outperform existing methods, particularly in the field of combinatorial optimisation. An escalating arms race is both rapidly producing new heuristics and improving the efficiency of the processes evolving them. However, driven by the desire to quickly demonstrate the superiority of new approaches, evaluation of the new heuristics produced for a specific domain is often cursory: testing on very few datasets in which instances all belong to a specific class from the domain, and on few instances per class. Taking bin-packing as an example, to the best of our knowledge we conduct the first rigorous benchmarking study of new LLM-generated heuristics, comparing them to well-known existing heuristics across a large suite of benchmark instances using three performance metrics. For each heuristic, we then evolve new instances won by the heuristic and perform an instance space analysis to understand where in the feature space each heuristic performs well. We show that most of the LLM heuristics do not generalise well when evaluated across a broad range of benchmarks in contrast to existing simple heuristics, and suggest that any gains from generating very specialist heuristics that only work in small areas of the instance space need to be weighed carefully against the considerable cost of generating these heuristics.

Algorithm selection is commonly used to predict the best solver from a portfolio per per-instance. In many real scenarios, instances arrive in a stream: new instances become available over time, while the number of class labels can also grow as new data distributions arrive downstream. As a result, the classification model needs to be periodically updated to reflect additional solvers without catastrophic forgetting of past data. In machine-learning (ML), this is referred to as Class Incremental Learning (CIL). While commonly addressed in ML settings, its relevance to algorithm-selection in optimisation has not been previously studied. Using a bin-packing dataset, we benchmark 8 continual learning methods with respect to their ability to withstand catastrophic forgetting. We find that rehearsal-based methods significantly outperform other CIL methods. While there is evidence of forgetting, the loss is small at around 7%. Hence, these methods appear to be a viable approach to continual learning in streaming optimisation scenarios.

Deep neural networks (DNN) are increasingly being used to perform algorithm-selection in combinatorial optimisation domains, particularly as they accommodate input representations which avoid designing and calculating features. Mounting evidence from domains that use images as input shows that deep convolutional networks are vulnerable to adversarial samples, in which a small perturbation of an instance can cause the DNN to misclassify. However, it remains unknown as to whether deep recurrent networks (DRN) which have recently been shown promise as algorithm-selectors in the bin-packing domain are equally vulnerable. We use an evolutionary algorithm (EA) to find perturbations of instances from two existing benchmarks for online bin packing that cause trained DRNs to misclassify: adversarial samples are successfully generated from up to 56% of the original instances depending on the dataset. Analysis of the new misclassified instances sheds light on the `fragility' of some training instances, i.e. instances where it is trivial to find a small perturbation that results in a misclassification and the factors that influence this. Finally, the method generates a large number of new instances misclassified with a wide variation in confidence, providing a rich new source of training data to create more robust models.

Dynamic algorithm selection aims to exploit the complementarity of multiple optimization algorithms by switching between them during the search. While these kinds of dynamic algorithms have been shown to have potential to outperform their component algorithms, it is still unclear how this potential can best be realized. One promising approach is to make use of landscape features to enable a per-run trajectory-based switch. Here, the samples seen by the first algorithm are used to create a set of features which describe the landscape from the perspective of the algorithm. These features are then used to predict what algorithm to switch to. In this work, we extend this per-run trajectory-based approach to consider a wide variety of potential points at which to perform the switch. We show that using a sliding window to capture the local landscape features contains information which can be used to predict whether a switch at that point would be beneficial to future performance. By analyzing the resulting models, we identify what features are most important to these predictions. Finally, by evaluating the importance of features and comparing these values between multiple algorithms, we show clear differences in the way the second algorithm interacts with the local landscape features found before the switch.

We propose a novel technique for algorithm-selection, applicable to optimisation domains in which there is implicit sequential information encapsulated in the data, e.g., in online bin-packing. Specifically we train two types of recurrent neural networks to predict a packing heuristic in online bin-packing, selecting from four well-known heuristics. As input, the RNN methods only use the sequence of item-sizes. This contrasts to typical approaches to algorithm-selection which require a model to be trained using domain-specific instance features that need to be first derived from the input data. The RNN approaches are shown to be capable of achieving within 5% of the oracle performance on between 80.88 and 97.63% of the instances, depending on the dataset. They are also shown to outperform classical machine learning models trained using derived features. Finally, we hypothesise that the proposed methods perform well when the instances exhibit some implicit structure that results in discriminatory performance with respect to a set of heuristics. We test this hypothesis by generating fourteen new datasets with increasing levels of structure, and show that there is a critical threshold of structure required before algorithm-selection delivers benefit.

Linear diagrams have been shown to be an effective method of representing set-based data. Moreover, a number of guidelines have been proven to improve the efficacy of linear diagrams. One of these guidelines is to minimise the number of line segments appearing in a diagram. We show this problem to be NP-hard.

We propose two new methods for evolving the layout of an instance-space. Specifically we design three different fitness metrics that seek to: (i) reward layouts which place instances won by the same solver close in the space; (ii) reward layouts that place instances won by the same solver and where the solver has similar performance close together; (iii) simultaneously reward proximity in both class and distance by combining these into a single metric. Two optimisation algorithms that utilise these metrics to evolve a model which outputs the coordinates of instances in a 2d space are proposed: (1) a multi-tree version of GP (2) a neural network with the weights evolved using an evolution strategy. Experiments in the TSP domain show that both new methods are capable of generating layouts in which subsequent application of a classifier provides considerably improved accuracy when compared to existing projection techniques from the literature, with improvements of over 10% in some cases. Visualisation of the the evolved layouts demonstrates that they can capture some aspects of the performance gradients across the space and highlight regions of strong performance.

It is known that drawing linear diagrams with few line segments produces cognitively effective diagrams. However, methods to minimise the number of line segments in a diagram have not been investigated. In this Abstract, we report on four different approaches to drawing linear diagrams: creating a drawing order based on sets; creating a drawing order based on set-intersections; simulated annealing; and a transformation to another problem, in this case the Travelling Salesman Problem (TSP). We tested each of these approaches on a corpus of 440 linear diagrams with a range of characteristics. The algorithm which produced the lowest number of line segments in a diagram was based on the Lin-Kernighan algorithm for solving instances of TSP. The second contribution of the work is the corpus of 440 linear diagrams, which can be used as a benchmark set of problems for others who seek other solutions to the problem of drawing linear diagrams.

Both algorithm-selection methods and hyper-heuristic methods rely on a pool of complementary heuristics. Improving the pool with new heuristics can improve performance, however, designing new heuristics can be challenging. Methods such as genetic programming have proved successful in automating this process in the past. Typically, these make use of problem state-information and existing heuristics as components. Here we propose a novel neural approach for generating constructive heuristics, in which a neural network acts as a heuristic by generating decisions. We evaluate two architectures, an Encoder-Decoder LSTM and a Feed-Forward Neural Network. Both are trained using the decisions output from existing heuristics on a large set of instances. We consider streaming instances of bin-packing problems in a continual stream that must be packed immediately in strict order and using a limited number of resources. We show that the new heuristics generated are capable of solving a subset of instances better than the well-known heuristics forming the original pool, and hence the overall value of the pool is improved w.r.t. both Falkenauer’s performance metric and the number of bins used.

The term Artificial Immune Systems (AIS) refers to a class of computational algorithms that are inspired by processes apparent in the natural immune system. As such, they belong to a wider family of biologically inspired algorithms such as evolutionary algorithms, ant-colony algorithms and other swarm inspired algorithms for example. The natural immune system can in the simplest case be viewed as a body-defense system, or more generally as a body-maintenance system of which defense is a special case. From a computer science perspective, it attracted interest due to the recognition that in order to fulfill its defense role, the natural immune system utilizes a number of mechanisms that lead to useful functionalities such as learning and memory that are key elements in many types of problem solving activities typical in computer science. This has led to a variety of applications in domains such as optimization and machinelearning as with other biologically inspired algorithms. As such, it can be viewed as a complementary technique to many existing algorithms.

In the field of combinatorial optimisation, per-instance algorithm selection still remains a challenging problem, particularly with respect to streaming problems such as packing or scheduling. Typical approaches involve training a model to predict the best algorithm based on features extracted from the data, which is well known to be a difficult task and even more challenging with streaming data. We propose a radical approach that bypasses algorithm-selection altogether by training a Deep-Learning model using solutions obtained from a set of heuristic algorithms to directly predict a solution from the instance-data. To validate the concept, we conduct experiments using a packing problem in which items arrive in batches. Experiments conducted on six large datasets using batches of varying size show the model is able to accurately predict solutions, particularly with small batch sizes, and surprisingly in a small number of cases produces better solutions than any of the algorithms used to train the model.

We propose a novel technique for algorithm-selection which adopts a deep-learning approach, specifically a Recurrent-Neural Network with Long-Short-Term-Memory (RNN-LSTM). In contrast to the majority of work in algorithm-selection, the approach does not need any features to be extracted from the data but instead relies on the temporal data sequence as input. A large case-study in the domain of 1-d bin packing is undertaken in which instances can be solved by one of four heuristics. We first evolve a large set of new problem instances that each have a clear "best solver" in terms of the heuristics considered. An RNN-LSTM is trained directly using the sequence data describing each instance to predict the best-performing heuristic. Experiments conducted on small and large problem instances with item sizes generated from two different probability distributions are shown to achieve between 7% to 11% improvement over the single best solver (SBS) (i.e. the single heuristic that achieves the best performance over the instance set) and 0% to 2% lower than the virtual best solver (VBS), i.e the perfect mapping.

Although numbers of heuristic algorithms are successfully developed for solving portfolio optimization problems, this is not for all cases of the large-scale ones. A large-scale portfolio optimization involves dealing with the large search space and dense variance-covariance matrix associated with the problem. This paper proposed a new multi-objective algorithm for solving a large-scale optimization problem based upon the notion of cooperative coevolutionary algorithms (CCA). The new problem decomposition scheme was designed by allowing the species-size to be dynamically adjusted as the runs progress. This scheme enhances capability of traditional CCA in dealing with non-separable optimization problem. The collaborator selection method was modified to allow the proposed CCA to perform in a multi-objective (MO) optimization framework. Additionally, the proposed algorithm, named as “DMOCCA”, was implemented for solving large-scale portfolio optimization problem with cardinality constraint using the real-world data set having scale up to 2196 dimensions. Moreover, its performances were benchmarked with those of the SPEA-II and MOPSO.

This paper tested the ability of machine learning techniques, namely artificial neural networks and random forests, to predict the individual trees within a forest most at risk of damage in storms. Models based on these techniques were developed individually for both a small forest area containing a set of 29 permanent sample plots that were damaged in Storm Martin in December 1999, and from a much larger set of 235 forest inventory plots damaged in Storm Klaus in January 2009. Both data sets are within the Landes de Gascogne Forest in Nouvelle-Aquitaine, France. The models were tested both against the data from which they were developed, and against the data set from the other storm. For comparison with an earlier study using the same data, logistic regression models were also developed. In addition, the ability of machine learning techniques to substitute for a mechanistic wind damage risk model by training them with previous mechanistic model predictions was tested. All models were accurate at identifying whether trees would be damaged or not damaged but the random forests models were more accurate, had higher discriminatory power, and were almost totally unaffected by the removal of any individual input variable. However, if all information relating to a stand was removed the random forests model lost accuracy and discriminatory power. The other models were similarly affected by the removal of all site information but none of the models were affected by removal of all tree information, suggesting that damage in the Landes de Gascogne Forest occurs at stand scale and is not controlled by individual tree characteristics. The models developed with the large comprehensive database were also accurate in identifying damaged trees when applied to the small forest data damaged in the earlier storm. However, none of the models developed with the smaller forest data set could successfully discriminate between damaged and undamaged trees when applied across the whole landscape. All models were very successful in replicating the predictions of the mechanistic wind risk model and using them as a substitute for the mechanistic model predictions of critical wind speed did not affect the damage model results. Overall the results suggest that random forests provide a significant advantage over other statistical modelling techniques and the random forest models were found to be more robust in their predictions if all input variables were not available. In addition, the ability to replace the mechanistic wind damage model suggests that random forests could provide a powerful tool for damage risk assessment over large regions and provide rapid assessment of the impact of different management strategies or be used in the development of optimised forest management with multiple objectives and constraints including the risk of wind damage.

Catastrophic damage to forests resulting from major storms has resulted in serious timber and financial losses within the sector across Europe in the recent past. Developing risk assessment methods is thus one of the keys to finding forest management strategies to reduce future damage. Previous approaches to predicting damage to individual trees have used mechanistic models of wind-flow or logistical regression with mixed results. We propose a novel filter-based Genetic Programming method for constructing a large set of new features which are ranked using the Hellinger distance metric which is insensitive to skew in the data. A wrapper-based feature-selection method that uses a random forest classifier is then applied predict damage to individual trees. Using data collected from two forests within South-West France, we demonstrate significantly improved classification results using the new features, and in comparison to previously published results. The feature-selection method retains a small set of relevant variables consisting only of newly constructed features whose components provide insights that can inform forest management policies.