Selection pressure restrains the selection of individuals from the current population to produce a new population in the next generation. It gives individuals of higher quality a higher probability of being used to create the next generation so that evolutionary algorithms (EAs) can focus on promising regions in the search space. An evolutionary learning process is dynamic and requires different selection pressures at different learning stages in order to speed up convergence or avoid local optima. Therefore, it is desirable for selection mechanisms to be able to automatically tune selection pressure during evolution. Tournament selection is a popular selection method in EAs, especially genetic algorithms and genetic programming (GP). This paper focuses on tournament selection and shows that the standard tournament selection scheme is unaware of the dynamics in the evolutionary process and that the standard tournament selection scheme is unable to tune selection pressure automatically. This paper then presents a novel approach which integrates the knowledge of the fitness rank distribution (FRD) of a population into tournament selection. Through mathematical modeling, simulations, and experimental study in GP, this paper shows that the new approach is effective and using the knowledge of FRD is a promising way to modify the standard tournament selection method for tuning the selection pressure dynamically and automatically along evolution.

Tournament selection is one of the most commonly used parent selection schemes in genetic programming (GP). While it has a number of advantages over other selection schemes, it still has some issues that need to be thoroughly investigated. Two of the issues are associated with the sampling process from the population into the tournament. The first one is the so-called “multi-sampled” issue, where some individuals in the population are picked up (sampled) many times to form a tournament. The second one is the “not-sampled” issue, meaning that some individuals are never picked up when forming tournaments. In order to develop a more effective selection scheme for GP, it is necessary to understand the actual impacts of these issues in standard tournament selection. This paper investigates the behaviour of different sampling replacement strategies through mathematical modelling, simulations and empirical experiments. The results show that different sampling replacement strategies have little impact on selection pressure and cannot effectively tune the selection pressure in dynamic evolution. In order to conduct effective parent selection in GP, research focuses should be on developing automatic and dynamic selection pressure tuning methods instead of alternative sampling replacement strategies. Although GP is used in the empirical experiments, the findings revealed in this paper are expected to be applicable to other evolutionary algorithms.

The standard subtree crossover operator in the tree-based genetic programming (GP) has been considered as problematic. In order to improve the standard subtree crossover, controlling depth of crossover points becomes a research topic. However, the existence of many different and inconsistent crossover depth-control schemes and the possibility of many other depth-control schemes make the identification of good depth-control schemes a challenging problem. This paper aims to investigate general heuristics for making good depth-control schemes for crossover in tree-based GP. It analyses the patterns of depth of crossover points in good predecessor programs of five GP systems that use the standard subtree crossover and four approximations of the optimal crossover operator on three problems in different domains. The analysis results show that an effective depth-control scheme is problem-dependent and evolutionary stage-dependent, and that good crossover events have a strong preference for roots and (less strongly) bottoms of parent program trees. The results also show that some ranges of depths between the roots and the bottoms are also preferred, suggesting that unequal-depth-selection-probability strategies are better than equal-depth-selection-probability strategies.

The emergence of service-oriented computing and cloud computing advocate the development of new software applications on the basis of existing services. To make effectively and efficiently use of existing services, it is essential to describe services so that developers can easily discover and correctly exploit them for their own software development endeavours. In this paper, we present a framework in which existing services available through a cloud are modelled by Abstract State Services. For specification, we use TBox and ABox statements bound to XML Schema and XML. Given such a representation, XQuery can be used for service discovery.

In order to drive Genetic Programming (GP) search towards an optimal situation, balancing selection pressure between the parent and offspring selection phases is an important aspect and very challenging. Our previous work showed that stochastic elements cannot be removed from both parent and offspring selections and suggested that maximising diversity in parents and minimising randomness in offspring could provide significantly good performance. This paper conducts additional carefully designed experiments to further investigate how diverse the parent should be if the offspring selection pressure is intensive. This paper shows that any attempt on adding more selection pressure to the parent selection can result in lower GP performance, and the higher the parent selection pressure, the worse the GP performance. The results confirm and strengthen the finding in our previous work.

Artificial Intelligence, volume 170, number 11, pages 953–983, 2006 published a paper titled “Backward-chaining evolutionary algorithm”. It introduced two fitness evaluation saving algorithms which are built on top of standard tournament selection. One algorithm is named Efficient Macro-selection Evolutionary Algorithm (EMS-EA) and the other is named Backward-chaining EA (BC-EA). Both algorithms were claimed to be able to provide considerable fitness evaluation savings, and especially BC-EA was claimed to be much efficient for hard and complex problems which require very large populations. This paper provides an evaluation and analysis of the two algorithms in terms of the feasibility and capability of reducing the fitness evaluation cost. The evaluation and analysis results show that BC-EA would be able to provide computational savings in unusual situations where given problems can be solved by an evolutionary algorithm using a very small tournament size, or a large tournament size but a very large population and a very small number of generations. Other than that, the saving capability of BC-EA is the same as EMS-EA. Furthermore, the feasibility of BC-EA is limited because two important assumptions making it work hardly hold.

Selection pressure controls the selection of individuals from the current population to produce a new population in the next generation. It gives individuals of higher quality a higher probability of being used to create the next generati on so that Evolutionary Algorithms (EAs) can focus on promising regions in the search space. An evolutionary learning proce ss is dynamic and requires different selection pressures at diff erent learning stages in order to speed up convergence or avoid local optima. Therefore, it desires selection mechanisms b eing able to automatically tune selection pressure during evolu tion. Tournament selection is a popular selection method in EAs. This paper focuses on tournament selection and shows that standard tournament selection is unaware of the dynamics in the evolutionary process thus is unable to tune selection pr essure automatically. This paper then presents a novel approach wh ich integrates the knowledge of the Fitness Rank Distribution ( FRD) of a population into tournament selection. Through mathematical modelling, simulations and experimental study, this papershows that the new approach is effective and using the knowledge of FRD is a promising way to modify the standard tournament selection method for tuning the selection pressure dynamic ally and automatically along evolution.

Tournament selection is one of the most commonly used parent selection schemes in Genetic Programming (GP). While it has a number of advantages over other selection sche mes, it still has some issues that need to be thoroughly investiga ted. Two of the issues are assocated with the sampling process fro m the population into the tournament. The first one is the socalled “multi-sampled” issue, where the some individuals i n the population are picked up (sampled) many times to form the tournament. The second one is the “not-sampled” issue, meaning that some individuals are never picked up when formi ng the tournament. In order to develop a more effective selecti on scheme for GP, it is necessary to understand the actual impac ts of these issues in standard tournament selection. This pape r investigates the behaviour of different sampling replacement strategies through mathematical modelling, theoretical s imulations and empirical experiments. The results show that diff erent sampling replacement strategies have little impact on sele ction pressure and cannot tune the selection pressure in dynamic evolution. In order to conduct effective parent selection i n GP, research focuses should be on developing automatic and dyna mic selection pressure tuning methods instead of alternative s ampling replacement strategies. Although GP is used in the empirica l experiments, the findings revealed in this paper are expecte d to be applicable to other evolutionary algorithms.

This thesis presents an analysis of the selection process in tree-based Genetic Programming (GP), covering the optimisation of both parent and offspring selection, and provides a detailed understanding of selection and guidance on how to improve GP search effectively and efficiently. The first part of the thesis providesmodels and visualisations to analyse selection behaviour in standard tournament selection, clarifies several issues in standard tournament selection, and presents a novel solution to automatically and dynamically optimise parent selection pressure. The fitness evaluation cost of parent selection is then addressed and some cost-saving algorithms introduced. In addition, the feasibility of using good predecessor programs to increase parent selection efficiency is analysed. The second part of the thesis analyses the impact of offspring selection pressure on the overall GP search performance. The fitness evaluation cost of offspring selection is then addressed, with investigation of some heuristics to efficiently locate good offspring by constraining crossover point selection structurally through the analysis of the characteristics of good crossover events. The main outcomes of the thesis are three new algorithms and four observations: 1) a clustering tournament selection method is developed to automatically and dynamically tune parent selection pressure; 2) a passive evaluation algorithm is introduced for reducing parent fitness evaluation cost for standard tournament selection using small tournament sizes; 3) a heuristic population clustering algorithm is developed to reduce parent fitness evaluation cost while taking advantage of clustering tournament selection and avoiding the tournament size limitation; 4) population size has little impact on parent selection pressure thus the tournament size configuration is independent of population size; and different sampling replacement strategies have little impact on the selection behaviour in standard tournament selection; 5) premature convergence occurs more often when stochastic elements are removed from both parent and offspring selection processes; 6) good crossover events have a strong preference for whole program trees, and (less strongly) single-node or small subtrees that are at the bottom of parent program trees; 7) the ability of standard GP crossover to generate good offspring is far below what was expected.

Rhythmic stress detection is an important but difficult problem in speech recognition. This paper describes an approach to the automatic detection of rhythmic stress in New Zealand spoken English using a linear genetic programming system with speaker independent prosodic features and vowel quality features as terminals to classify each vowel segment as stressed or unstressed. In addition to the four standard arithmetic operators, this approach also uses other functions such as trigonometric and conditional functions in the function set to cope with the complexity of the task. The error rate on the training set is used as the fitness function. The approach is examined and compared to a decision tree approach and a support vector machine approach on a speech data set with 703 vowels segmented from 60 female adult utterances. The genetic programming approach achieved a maximum average accuracy of 92.6%. The results suggest that the genetic programming approach developed in this paper outperforms the decision tree approach and the support vector machine approach for stress detection on this data set in terms of the detection accuracy, the ability of handling redundant features, and the automatic feature selection capability.

Standard tournament selection samples individuals with replacement. The sampling-with-replacement strategy has its advantages but also has issues. One of the commonly recognised issues is that it is possible to have the same individual sampled multiple times in a tournament. Although the impact of this multi-sampled issue on genetic programming is not clear, some researchers believe that it may lower the probability of some good individuals being sampled or selected. One solution is to use an alternative tournament selection (no-replacement tournament selection), which samples individuals in a tournament without replacement. This paper analyses no-replacement tournament selection to investigate the impact of the scheme and the importance of the issue. Theoretical simulations show that when common tournament sizes and population sizes are used, no-replacement tournament selection does not make the selection behaviour significantly different from that in the standard one and that the multi-sampled issue seldom occurs. In general, the issue is not crucial to the selection behaviour of standard tournament selection.

The standard tournament selection samples individuals with replacement. The sampling-with-replacement strategy has its advantages but also has issues. One of the commonly recognised issues is that it is possible to have some individuals not sampled at all during the selection phase. The not-sampled issue aggravates the loss of program diversity. However, it is not clear how the issue affects Genetic Programming (GP) search. This paper investigates the importance of the issue. The theoretical and experimental results show that the issue can be solved and the loss of diversity contributed by not-sampled individuals can be minimised. However, doing so does not appears to significantly improve a GP system. Our conclusion is that the not-sampled issue does not seriously affect the selection performance in the standard tournament selection.

This paper analyses the distribution of swapped subtree sizes involved in crossover events in approximations of an optimal crossover operator that allows the root node to be crossed over. The goal is to examine how the offspring search space can be effectively reduced for given parents. It concludes that good crossover events have a strong preference for the roots of the parent programs and for nodes with small subtrees. This paper also quantifies the ability of crossover to optimise offspring fitness, and concludes that this ability is far below what was expected.

A common problem in genetic programming search algorithms is destructive crossover in which the offspring of good parents generally has worse performance than the parents. Designing constructive crossover operators and integrating some local search techniques into the breeding process have been suggested as solutions. This paper reports on experiments demonstrating that premature convergence may happen more often when using these techniques in combination with standard parent selection. It shows that modifying the selection pressure in the parent selection process is necessary to obtain a significant performance improvement.

Tournament selection has been widely used and studied in evolutionary algorithms. To supplement the study of tournament selection, this paper provides several models describing the probabilities that a program of a particular rank is sampled and is selected in the standard tournament selection in a simple situation and a complex situation. This paper discovers that, with the same tournament size, trends of sampling probability of a program and selection probability distributions of a population are the same regardless ofthe population size. This paper also models and investigates an alternative tournament selection method which eliminates one of the drawbacks in the standard tournament selection. Finally, this paper proposes a new fitness evaluation saving algorithm via the use of not-sampled individuals, which is a special property of tournament selection.

The standard crossover operator in tree-based genetic programming (GP) is problematic in that it is most often destructive. Selecting crossover points with an implicit bias towards the leaves of a program tree aggravates its destructiveness and causes the code bloat problem in GP. Therefore, a common view has been developed that adjusting the depth of crossover points to eliminate the bias can improve GP performance, and many attempts have been made to create effective crossover operators according to this view. As there are a large number of possible depth-control strategies, it is very difficult to identify the strategy that provides the most significant improvement in performance. This paper explores depth-control strategies by analysing the depth of crossover points in evolutionary process logs of five different GP systems on problems in three different domains. It concludes that controlling the depth of crossover points is an evolutionary stage dependent and problem dependent task, and obtaining a significant performance improvement is not trivial.

Reserve Bank of New Zealand (RBNZ) is one of many inflation-targeting central banks. The effective conduct of monetary policy requires the capacity to make accurate short and medium term predictions about price inflation. The RBNZ's prediction system is very complex, requiring many iterations and significant input from human experts. This paper investigates the capability of Genetic Programming (GP) to predict price inflation over short and medium terms. By using un-preprocessed economic time series over small intervals, the experimental results demonstrate that GP can produce predictions of price inflation with accuracy comparable to the RBNZ's official prediction system, over both short and medium terms.

The standard tournament selection samples individuals with replacement. The sampling-with-replacement strategy has its advantages but also has issues. One of the commonly recognised issues is that it is possible to have some individuals not sampled at all during the selection phase. The not-sampled issue aggravates the loss of diversity. However, it is not clear how the issue affects GP search. This paper uses a round-replacement tournament selection to investigate the importance of the issue. The theoretical and experimental results show that although the issue can be solved and the loss of diversity can be minimised for small tournament sizes, the different selection behaviour in the round-replacement tournament selection cannot significantly improve the GP performance. The not-sampled issue does not seriously affect the selection performance in the standard tournament selection.

Good Predecessor Programs (GPPs) are the ancestors of the best program found in a Genetic Programming (GP) evolution. This paper reports on an investigation into GPPs with the ultimate goal of reducing fitness evaluation cost in tree-based GP systems. A framework is developed for gathering information about GPPs and a series of experiments is conducted on a symbolic regression problem, a binary classification problem, and a multi-class classification program with increasing levels of difficulty in different domains. The analysis of the data shows that during evolution, GPPs typically constitute less than 33% of the total programs evaluated, and may constitute less than 5%. The analysis results further shows that in all evaluated programs, the proportion of GPPs is reduced by increasing tournament size and to a less extent, affected by population size. Problem difficulty seems to have no clear influence on the proportion of GPPs.

This paper proposes an approach to reducing the cost of fitness evaluation whilst improving the effectiveness in Genetic Programming (GP). In our approach, the whole population is first clustered by a heuristic called fitness-case-equivalence. Then a cluster representative is selected for each cluster. The fitness value of the representative is calculated on all training cases. The fitness is then directly assigned to other members in the same cluster. Subsequently, a clustering tournament selection method replaces the standard tournament selection method. A series of experiments were conducted to solve a symbolic regression problem, a binary classification problem, and a multi-class classification problem. The experiment results show that the new GP system significantly outperforms the standard GP system on these problems.