Planning in partially-observable dynamical systems (such as POMDPs and PSRs) is a computationally challenging task. Popular approximation techniques that have proven successful are point-based planning methods including pointbased value iteration (PBVI), which works by approximating the solution at a finite set of points. These point-based methods typically are anytime algorithms, whereby an initial solution is obtained using a small set of points, and the solution may be incrementally improved by including additional points. We introduce a family of anytime PBVI algorithms that use the information present in the current solution for identifying and adding new points that have the potential to best improve the next solution. We motivate and present two different methods for choosing points and evaluate their performance empirically, demonstrating that high-quality solutions can be obtained with significantly fewer points than previous PBVI approaches.

The quality of experimental research in Genetic and Evolutionary Computation can suffer from experiments that are too small to generalize results. Experiments can be too small by not being statistically significant with a given set of algorithm parameters, or by not identifying the effect of modifying an algorithm parameter, or by not understanding the causal relationships between different algorithm parameters. Parameter Sweep Experiments solve these problems by sampling many different parameter configurations enough times to achieve statistical significance and identify trends in parameter modifications. Unfortunately, Parameter Sweep Experiments have a high cost of labor to setup and run experiments and analyze data. As a result, these experiments are rarely conducted in Genetic and Evolutionary Computation research. This chapter argues that although Parameter Sweep Experiments are difficult to conduct, they are required to understand complicated nonlinear interactions between an algorithm's parameters. We present a software solution, Commander, that works in conjunction with a genetic and evolutionary algorithm to assist users in conducting Parameter Sweep Experiments in a distributed computing environment.

Phase transitions occur in computational, as well as thermodynamic systems. Of particular interest is the possibility that phase transitions occur as a consequence of GP search. If this were so, it would allow for a statistical mechanics approach and quantitative comparisons of GP with a broad variety of rigorously described systems. This chapter summarizes our research group's work in this area and describes a case study that illustrates what is involved in establishing the existence of phase transitions in GP search.

Optimal resource scheduling in multiagent systems is a computationally challenging task, particularly when the values of resources are not additive. We consider the combinatorial problem of scheduling the usage of multiple resources among agents that operate in stochastic environments, modeled as Markov decision processes (MDPs). In recent years, efficient resource-allocation algorithms have been developed for agents with resource values induced by MDPs. However, this prior work has focused on static resource-allocation problems where resources are distributed once and then utilized in infinite-horizon MDPs. We extend those existing models to the problem of combinatorial resource scheduling, where agents persist only for finite periods between their (predefined) arrival and departure times, requiring resources only for those time periods. We provide a computationally efficient procedure for computing globally optimal resource assignments to agents over time. We illustrate and empirically analyze the method in the context of a stochastic job-scheduling domain.

Many serious automobile accidents could be avoided if drivers were warned of impending crashes before they occur. Creating such warning systems by hand, however, is a difficult and time-consuming task. This paper describes three advances toward evolving neural networks with NEAT (NeuroEvolution of Augmenting Topologies) to warn about such crashes in real-world environments. First, NEAT was evaluated in a complex, dynamic simulation with other cars, where it outperformed three hand-coded strawman warning policies and generated warning levels comparable with those of an open-road warning system. Second, warning networks were trained using raw pixel data from a simulated camera. Surprisingly, NEAT was able to generate warning networks that performed similarly to those trained with higher-level input and still outperformed the baseline hand-coded warning policies. Third, the NEAT approach was evaluated in the real world using a robotic vehicle testbed. Despite noisy and ambiguous sensor data, NEAT successfully evolved warning networks using both laser rangefinders and visual sensors. The results in this paper set the stage for developing warning networks for real-world traffic, which may someday save lives in real vehicles.

This paper describes a tunably-difficult problem for genetic programming (GP) that probes for limits to building block mixing and assembly. The existence of such a problem can be used to garner insight into the dynamics of what happens during the course of a GP run. The results indicate that the amount of mixing is fairly low in comparison to the amount of content that could be present in an initial population.

This paper describes our procedure and a software application for conducting large parameter sweep experiments in genetic and evolutionary computation research. Both procedure and software allows a researcher to examine multivariate nonlinearities that are common in genetic and evolutionary computation. Experiments of this nature are well suited to distributed computing environments (such as Grids and clusters) and we present an automated system for conducting parameter sweep experiments on heterogeneous networks. Emphasis is placed on experimental sampling, distributed robustness, and data analysis. The parameter sweep experimental procedure is easily applicable to any experiment involving computer simulations but is particularly well suited for evolutionary computation experiments.