Genetic and Evolutionary Computation Conference : [proceedings]

Evolutionary rule-based machine learning (ERBML) algorithms can capture complex relationships while still yielding highly interpretable models comprised of IF:THEN rules. During prediction, 'matching' rules contribute to, and form the explanation for, the model's prediction. However, rules and their associated parameters (in their raw form) are likely too technical for their intended users. This study examines the feasibility of using a large language model (LLM) to translate the prediction evidence from matching rules into natural language text for different audiences, e.g. layman, clinician, expert. Using models trained by the 'HEROS' ERBML on MUX benchmarks, we evaluate LLM text quality metrics under different scenarios (i.e. 1,800 prediction explanations). We observe that (1), intuitively, LLM quality performance improves on HEROS models that have been more ideally trained, (2) making a glossary available to the LLM to define feature names generally raises explanation audience-fit scores and sometimes lowers overstatement (beyond rule-evidence), but it also lengthens explanations and often increases hallucination rate, and (3) audience customization creates some LLM performance trade-offs. These results suggest that constrained LLM translation of rules for natural language prediction explanations is feasible, while highlighting the importance of carefully designing the LLM prompts and evidence input from the ERBML.

Interpretable machine learning is important for scientific, biomedical, and other high-stakes domains where transparency, trust, and human understanding of models are required. Recently, an evolutionary rule-based machine learning algorithm called HEROS was proposed, which separated rule from rule-set discovery each with distinct multi-objective optimization strategies yielding a two-phase, noise agnostic framework that found accurate, highly compact, and interpretable solutions on a diverse set of challenging benchmarks without requiring hyperparameter optimizations. However, HEROS is currently limited by its strictly sequential two-phase design which requires committing in advance to a fixed number of rule-discovery iterations. This work extends HEROS with a phase alternation scheme interleaving rule discovery and rule-set optimization. This enables refinement of both tasks throughout training in an 'anytime-algorithm' manner. We also introduce a tree-based rule initialization strategy to accelerate early-stage rule discovery. These extensions aim to preserve the empirical performance of the original HEROS framework under a similar computational budget while providing greater flexibility for algorithm application and problem scalability. Using the same diverse set of benchmark datasets, we evaluate and compare these HEROS extensions to the original HEROS algorithm and established rule-based learning systems including RIPPER and BioHEL. We demonstrate the advantages of this extended HEROS framework.

Rule-based machine learning (RBML) algorithms, e.g. learning classifier systems (LCSs), can capture complex relationships while yielding more interpretable models than most other machine learning algorithms. Traditional LCSs rely on a single fitness function for both rule and/or rule-set optimization. However, ideal rule vs. rule-set discovery often requires distinct and multiple objectives. Recently, hybrid-LCSs were proposed that explicitly separated the task of rule vs. rule-set discovery but relied on distinct single-objective or weighted multi-objective fitness functions. This study introduces a newly developed Heuristic Evolutionary Rule Optimization System (HEROS) that combines previous LCS innovations aimed at tackling noisy, larger-scale, classification tasks, while adopting separation of rule vs. rule-set evolution. Uniquely, HEROS employs a custom Pareto-front-based multi-objective fitness function (for rule discovery) and NSGA-II-style multi-objective optimization (for rule-set discovery) to solve both clean and noisy-signal classification problems agnostically. Rule discovery is driven by rule-accuracy and instance coverage objectives, while rule-set discovery is driven by prediction accuracy and rule-set size objectives. Using diverse simulated benchmark datasets, i.e. noisy (GAMETES) and clean (MUX), we demonstrate proof-of-principle that HEROS can directly discover accurate, highly-compact, interpretable, and ideal solutions when compared to the established 'ExSTraCS' RBML algorithm, without objective weightings or adjusting hyperparameters.

A model is presented that allows for the calculation of the success probability by which a vanilla Evolution Strategy converges to the global optimizer of the Rastrigin test function. As a result a population size scaling formula will be derived that allows for an estimation of the population size needed to ensure a high convergence security depending on the search space dimensionality.

Multidimensional genetic programming represents candidate solutions as sets of programs, and thereby provides an interesting framework for exploiting building block identification. Towards this goal, we investigate the use of machine learning as a way to bias which components of programs are promoted, and propose two semantic operators to choose where useful building blocks are placed during crossover. A forward stagewise crossover operator we propose leads to significant improvements on a set of regression problems, and produces state-of-the-art results in a large benchmark study. We discuss this architecture and others in terms of their propensity for allowing heuristic search to utilize information during the evolutionary process. Finally, we look at the collinearity and complexity of the data representations that result from these architectures, with a view towards disentangling factors of variation in application.