This tutorial provides a basic description of genetic programming.

Genetic programming has now been used to produce at least 76 instances of results that are competitive with human-produced results. These human-competitive results come from a wide variety of fields, including quantum computing circuits, analog electrical circuits, antennas, mechanical systems, controllers, game playing, finite algebras, photonic systems, image recognition, optical lens systems, mathematical algorithms, cellular automata rules, bioinformatics, sorting networks, robotics, assembly code generation, software repair, scheduling, communication protocols, symbolic regression, reverse engineering, and empirical model discovery. This paper observes that, despite considerable variation in the techniques employed by the various researchers and research groups that produced these human-competitive results, many of the results share several common features. Many of the results were achieved by using a developmental process and by using native representations regularly used by engineers in the fields involved. The best individual in the initial generation of the run of genetic programming often contains only a small number of operative parts. Most of the results that duplicated the functionality of previously issued patents were novel solutions, not infringing solutions. In addition, the production of human-competitive results, as well as the increased intricacy of the results, are broadly correlated to increased availability of computing power tracked by Moore's law. The paper ends by predicting that the increased availability of computing power (through both parallel computing and Moore's law) should result in the production, in the future, of an increasing flow of human-competitive results, as well as more intricate and impressive results.

This paper introduces genetic programming (GP) a set of evolutionary computation techniques for getting computers to automatically solve problems without having to tell them explicitly how to do it. Since its inception, GP has been used to solve many practical problems, producing a number of human competitive results and even patentable new inventions. We start with a gentle introduction to the basic representation, initialisation and operators used in GP, complemented by a step by step description of their use for the solution of an illustrative problem. We then progress to discuss a variety of alternative representations for programs and more advance specialisations of GP. A multiplicity of real-world applications of GP are then presented to illustrate the scope of the technique. For the benefits of more advanced readers, this is followed by a series of recommendations and suggestions to obtain the most from a GP system. Although the paper has been written with beginners and practitioners in mind, for completeness we also provide an overview of the theoretical results and models available to date for GP. The paper is concluded by an appendix which provides a plethora of pointers to resources and further reading.

Genetic programming is a systematic method for getting computers to automatically solve problems. Genetic programming starts from a high-level statement of what needs to be done and automatically creates a computer program to solve the problem by means of a simulated evolutionary process. The paper demonstrates that genetic programming (1) now routinely delivers high-return human-competitive machine intelligence; (2) is an automated invention machine; (3) can automatically create a general solution to a problem in the form of a parameterized topology and (4) has delivered a progression of qualitatively more substantial results in synchrony with five approximately order-of-magnitude increases in the expenditure of computer time. These points are illustrated by a group of recent results involving the automatic synthesis of the topology and sizing of analog electrical circuits, the automatic synthesis of placement and routing of circuits, and the automatic synthesis of controllers as well as references to work involving the automatic synthesis of antennas, networks of chemical reactions (metabolic pathways), genetic networks, mathematical algorithms, and protein classifiers.

This paper reviews the recent use of genetic programming to automatically synthesize human-competitive designs of complex structures in six engineering domains, namely analog electrical circuits, optical lens systems, controllers, antennas, mechanical systems, and quantum computing circuits. First, the paper identifies common features observed in the human-competitive results produced by genetic programming in the six domains and suggests possible explanations for the observed similarities. Second, the paper identifies the characteristics that make a particular domain amenable to the application of genetic programming for the automatic synthesis of designs. Third, the paper discusses certain domain-specific adjustments in technique that may increase the efficiency of the automated process in a particular domain. Fourth, the paper discusses several technique issues that have arisen in more than one domain.

The concentrations of substances participating in networks of chemical reactions are often modeled by non-linear continuous-time differential equations. Recent work has demonstrated that genetic programming is capable of automatically creating complex networks (such as analog electrical circuits and controllers) whose behavior is modeled by linear and non-linear continuous-time differential equations and whose behavior matches prespecified output values. This chapter demonstrates that it is possible to automatically induce (reverse engineer) a network of chemical reactions from observed time-domain data. Genetic programming starts with observed time-domain concentrations of substances and automatically creates both the topology of the network of chemical reactions and the rates of each reaction of a network such that the behavior of the automatically created network matches the observed time-domain data. Specifically, genetic programming automatically created a network of four chemical reactions that consume glycerol and fatty acid as input, use ATP as a cofactor, and produce diacyl-glycerol as the final product. The network was created from 270 data points. The topology and sizing of the entire network was automatically created using the time-domain concentration values of diacyl-glycerol (the final product). The automatically created network contains three key topological features, including an internal feedback loop, a bifurcation point where one substance is distributed to two different reactions, and an accumulation point where one substance is accumulated from two sources.

This paper describes the use of genetic programming to automatically synthesize the design for a fixed-length loaded symmetric dipole antenna whose gain at a specific wavelength exceeds that of a commercially-marketed human-designed antenna and that reaches the theoretical maximum value for an antenna of its type. The run of genetic programming started "from scratch" that is, without starting from a pre-existing human-created design; did not employ any knowledge base of human design techniques or principles from the field of antenna design; and did not benefit from any human intervention during the run. The run produced a human-competitive result.

This paper has two aspects. First, it describes the use of genetic programming to automatically synthesize a solution to the challenge problem posed at an international competition held every four years in the field of optical design. In 2002, the competition at the International Optical Design Conference attracted 42 entries from 39 well-known optical designers, commercial consultants, and patent holders from many of the field's most prominent companies, universities, and research institutions. The 39 human contestants spent an average of 34.1 hours working on their entries. Virtually all entries were considered good solutions to the challenge problem. Genetic programming automatically synthesized a design "from scratch" - that is, without starting from a pre-existing human-created design and without pre-specifying the number of lenses, the physical layout of the lenses, or the numerical or non-numerical parameters of the lenses. The run of genetic programming did not employ any knowledge base of design techniques or principles from the field of optical design and did not entail any human intervention during the run. The genetically evolved optical lens system would have ranked in the middle (21st) if it had been entered into the 2002 competition and is therefore an instance of a "human-competitive" result produced by genetic programming. Second, this paper presents a mutation operation for numerical constants that is especially appropriate for problems in which the to-be-designed structure contains a large number of non-linearly interrelated numerical values and for problems in which the topology of the solution is to be automatically created.

This paper describes how genetic programming has been used as an invention machine to automatically synthesize complete designs for four optical lens systems that duplicated the functionality of previously patented lens systems. The automatic synthesis of the complete design is done ab initio, that is, without starting from a preexisting good design and without prespecifying the number of lenses, the topological arrangement of the lenses, or the numerical or nonnumerical parameters associated with any lens. One of the genetically evolved lens systems infringed a previously issued patent, whereas the others were noninfringing novel designs that duplicated (or improved upon) the performance specifications contained in the patents. One of the patents was issued in the 21st century. The designs were created in a substantially similar and routine way, suggesting that the approach described in the paper can be readily applied to other similar problems in the field of optical design. The genetically evolved designs are instances of human-competitive results produced by genetic programming in the field of optical design.

It has been previously established that genetic programming can be used as an automated invention machine to synthesize designs for complex structures. In particular, genetic programming has automatically synthesized structures that infringe, improve upon, or duplicate the functionality of 21 previously patented inventions (including six 21(st)-century patented analog electrical circuits) and has also generated two patentable new inventions (controllers). There are seven promising factors suggesting that these previous results can be extended to deliver industrial-strength automated design of analog circuits, but two countervailing factors. This chapter explores the question of whether the seven promising factors can overcome the two countervailing factors by reviewing progress on an ongoing project in which we are employing genetic programming to synthesize an amplifier circuit. The work involves a multiobjective fitness measure consisting of 16 different elements measured by five different test fixtures. The chapter describes five ways of using general domain knowledge applicable to all analog circuits, two ways for employing problem-specific knowledge, four ways of improving on previously published genetic programming techniques, and four ways of grappling with the multiobjective fitness measures associated with real-world design problems.

Genetic Programming IV: Routine Human-Competitive Machine Intelligence presents the application of GP to a wide variety of problems involving automated synthesis of controllers, circuits, antennas, ge

This paper demonstrates that a design for a low-distortion high-gain 96 decibel (64,860 -to-1) operational amplifier (including both circuit topology and component sizing) can be evolved using genetic programming.

The three dozen or so known instances of human-competitive designs of antennas, mechanical systems, circuits, and controllers produced by genetic programming suggest the question of whether genetic programming can be extended to the design of complex structures from other fields. This paper describes how genetic programming can be used to automatically create a complete design for an optical lens system “from scratch”—without starting from a pre-existing good design and without pre-specifying the number of lenses, the layout of lenses, or the numerical parameters of the lenses. More particularly, genetic programming created an optical system that infringed a previous patent (the Konig patent) and improved upon another previous patent (the Tackaberry-Muller patent). The genetically evolved design is an example of a human-competitive result produced by genetic programming in the field of optical design.