One of the strongest approaches for optimal classical planning is A* search with heuristics based on abstractions of the planning task. Abstraction heuristics are well studied in planning formalisms without conditional effects such as SAS+. However, conditional effects are crucial to model many planning tasks compactly. In this paper, we focus on *factored* tasks which allow a specific form of conditional effect, where effects on variable x can only depend on the value of x. We generalize projections, domain abstractions, Cartesian abstractions and the counterexample-guided abstraction refinement method to this formalism. While merge-and-shrink already covers factored task in theory, we provide an implementation that does so. In our experiments, we compare these abstraction-based heuristics to other heuristics supporting conditional effects, as well as symbolic search. On our new benchmark set of factored tasks, pattern database heuristics solve the most problems, followed by symbolic approaches on par with domain abstractions. The more general Cartesian abstractions fall behind in terms of coverage but usually solve problems the fastest among all tested approaches. The generality of merge-and-shrink abstractions does not seem to be beneficial for these factored tasks.

We propose a new approach based on ranking to learn to guide Greedy Best-First Search (GBFS). As previous ranking approaches, ours is based on the observation that directly learning a heuristic function is overly restrictive, and that GBFS is capable of efficiently finding good plans for a much more flexible class of total quasi-orders over states. In order to learn an optimal ranking function, we introduce a new ranking framework capable of leveraging any neural network regression model and efficiently handling the training data through batching. Compared with previous ranking approaches for planning, ours does not require complex loss functions and allows training on states outside the optimal plan with minimal overhead. Our experiments on the domains of the latest planning competition learning track show that our approach substantially improves the coverage of the underlying neural network models without degrading plan quality.

This bundle contains code, scripts and benchmarks for reproducing all experiments reported in the paper. It also contains the data generated for the paper. Except for the code base, it contains the same files as the Zenodo entry for our IPC 2018 planner Delfi: https://zenodo.org/record/6683892. sievers-et-al-aaai2019-fast-downward.zip contains the implementation based on Fast Downward. It also contains the experiment scripts (hopefully) compatible with Lab 2.1 for reproducing all experiments of the paper, under experiments/ipc2018. While this code base contains a few more scripts used for this AAAI paper compared to the IPC 2018 code, it hasn't been updated to contain, e.g., a requirements.txt file for setting up a Python virtual environment like we did for our IPC 2018 planners. Please see katz-et-al-ipc2018-delfi1.zip and katz-et-al-ipc2018-delfi2.zip in the above mentioned Zenodo entry. sievers-et-al-aaai2019-scripts.zip contain all scripts for the learning pipeline used to train the planner selection models. sievers-et-al-aaai2019-benchmarks.zip contains the benchmarks. It consists of the IPC benchmarks used in all optimal sequential tracks of IPCs up to 2014 (suite optimal from https://github.com/aibasel/downward-benchmarks). sievers-et-al-aaai2019-lab.zip contains a copy of Lab 2.1 (https://github.com/aibasel/lab). sievers-et-al-aaai2019-raw-data.zip and sievers-et-al-aaai2019-parsed-data.zip contain the experimental data. Directories in sievers-et-al-aaai2019-raw-data.zip (without the "-eval" ending) contain raw data, distributed over a subdirectory for each experiment. Each of these contain a subdirectory tree structure "runs-*" where each planner run has its own directory. For each run, there are symbolic links to the input PDDL files domain.pddl and problem.pddl (can be resolved by putting the benchmarks directory to the right place), the run log file "run.log" (stdout), possibly also a run error file "run.err" (stderr), the run script "run" used to start the experiment, and a "properties" file that contains data parsed from the log file(s). Directories in sievers-et-al-aaai2019-parsed-data.zip (with the "-eval" ending) contain a "properties" file, which contains a JSON directory with combined data of all runs of the corresponding experiment. In essence, the properties file is the union over all properties files generated for each individual planner run. The image data set used for training can be found online: https://github.com/IBM/IPC-image-data Note on license: we chose GPL v3.0 or later mainly because we consider our implementation based on Fast Downward the main contribution of this package, and Fast Downward comes with GPL v3.0. We only include a copy of Lab and the benchmarks for convenience.

Since no classical planner consistently outperforms all others, it is important to select a planner that works well for a given classical planning task. The two strongest approaches for planner selection use image and graph convolutional neural networks. They have the drawback that the learned models are complicated and uninterpretable. To obtain explainable models, we identify a small set of simple task features and show that elementary and interpretable machine learning techniques can use these features to solve roughly as many tasks as the complex approaches based on neural networks.

How can we train neural network (NN) heuristic functions for classicalplanning, using only states as the NN input? Prior work addressed thisquestion by (a) per-instance imitation learning and/or (b) per-domainlearning. The former limits the approach to instances small enough fortraining data generation, the latter to domains where the necessaryknowledge generalizes across instances. Here we explore three methodsfor (a) that make training data generation scalable throughbootstrapping and approximate value iteration. In particular, weintroduce a new bootstrapping variant that estimates search effortinstead of goal distance, which as we show converges to the perfectheuristic under idealized circumstances. We empirically compare thesemethods to (a) and (b), aligning three different NN heuristic functionlearning architectures for cross-comparison in an experiment ofunprecedented breadth in this context. Key lessons are that ourmethods and imitation learning are highly complementary; thatper-instance learning often yields stronger heuristics than per-domainlearning; and the LAMA planner is still dominant but our methodsoutperform it in one benchmark domain.

Testing is a promising way to gain trust in neural action policies π. Previous work on policy testing in sequential decision making targeted environment behavior leading to failure conditions. But if the failure is unavoidable given that behavior, then π is not actually to blame. For a situation to qualify as a "bug" in π, there must be an alternative policy π' that does better. We introduce a generic policy testing framework based on that intuition. This raises the bug confirmation problem, deciding whether or not a state is a bug. We analyze the use of optimistic and pessimistic bounds for the design of test oracles approximating that problem. We contribute an implementation of our framework in classical planning, experimenting with several test oracles and with random-walk methods generating test states biased to poor policy performance and/or state novelty. We evaluate these techniques on policies π learned with ASNets. We find that they are able to effectively identify bugs in these π, and that our random-walk biases improve over uninformed baselines.

Neural networks (NN) are increasingly investigated in AI Planning, and are used successfully to learn heuristic functions. NNs commonly not only predict a value, but also output a confidence in this prediction. From the perspective of heuristic search with NN heuristics, it is a natural idea to take this into account, e.g. falling back to a standard heuristic where confidence is low. We contribute an empirical study of this idea. We design search methods which prune nodes, or switch between search queues, based on the confidence of NNs. We furthermore explore the possibility of out-of-distribution (OOD) training, which tries to reduce the overconfidence of NNs on inputs different to the training distribution. In experiments on IPC benchmarks, we find that our search methods improve coverage over standard methods, and that OOD training has the desired effect in terms of prediction accuracy and confidence, though its impact on search seems marginal.

Since its invention in 1974, the Rubik’s Cube puzzle fasci- nates people of all ages. Its rules are simple: the player gets a scrambled cube and rotates the six faces until each face contains only stickers of one color. Nevertheless, finding a short sequence of rotations to solve the cube is hard. We present the first model of Rubik’s Cube for general problem solvers. To obtain a concise model, we require conditional ef- fects. Furthermore, we extend counterexample-guided Cartesian abstraction refinement (CEGAR) to support factored ef- fect tasks , a class of planning tasks with a specific kind of conditional effects which includes Rubik’s Cube. Finally, we evaluate how newer types of abstraction heuristics compare against pattern database (PDB) heuristics, the state-of-the-art for solving Rubik’s Cube. We find that PDBs still outper-form the more general Cartesian and merge-and-shrink ab- stractions. However, in contrast to PDBs, Cartesian abstractions yield perfect heuristics up to a certain problem difficulty. These findings raise interesting questions for future research.

Previous work introduced the concept of progressstates. After expanding a progress state, a greedybest-first search (GBFS) will only expand stateswith lower heuristic values. Current methods canidentify progress states only for a single task andonly after a solution for the task has been found.We introduce a novel approach that learns a descriptionlogic formula characterizing all progressstates in a classical planning domain. Using thelearned formulas in a GBFS to break ties in favorof progress states often significantly reduces thesearch effort.

How can we train neural network (NN) heuristic functions for classical planning, using only states as the NN input? Prior work addressed this question by (a) supervised learning and/or (b) per-domain learning generalizing over problem instances. The former limits the approach to instances small enough for training data generation, the latter to domains and instance distributions where the necessary knowledge generalizes across instances. Clearly, reinforcement learning (RL) on large instances can potentially avoid both difficul- ties. We explore this here in terms of three methods drawing on previous ideas relating to bootstrapping and approximate value iteration, including a new bootstrapping variant that estimates search effort instead of goal distance. We empirically compare these methods to (a) and (b), aligning three different NN heuristic function learning architectures for cross- comparison in an experiment of unprecedented breadth in this context. Key lessons from this experiment are that our meth- ods and supervised learning are highly complementary; that per-instance learning often yields stronger heuristics than per- domain learning; and that LAMA is still dominant but is outperformed by our methods in one benchmark domain.

Since no classical planner consistently outperforms all others, it is important to select a planner that works well for a given classical planning task. The two strongest approaches for planner selection use image and graph convolutional neural networks. They have the drawback that the learned models are not interpretable. To obtain explainable models, we identify a small set of simple task features and show that elementary and interpretable machine learning techniques can use these features to solve as many tasks as the approaches based on neural networks.

Neural networks (NN) have been shown to be powerful state-value predictors in several complex games. Can similar successes be achieved in classical planning? Towards a systematic exploration of that question, we contribute a study of hyperparameter space in the most canonical setup: input = state, feed-forward NN, supervised learning, generalization only over initial state. We investigate a broad range of hyperparameters pertaining to NN design and training. We evaluate these techniques through their use as heuristic functions in Fast Downward. The results on IPC benchmarks show that highly competitive heuristics can be learned, yielding substantially smaller search spaces than standard techniques on some domains. But the heuristic functions are costly to evaluate, and the range of domains where useful heuristics are learned is limited. Our study provides the basis for further research improving on current weaknesses.

Informed heuristics are essential for the success of heuristic search algorithms. But, it is difficult to develop a new heuris- tic which is informed on various tasks. Instead, we propose a framework that trains a neural network as heuristic for the tasks it is supposed to solve. We present two reinforcement learning approaches to learn heuristics for fixed state spaces and fixed goals. Our first approach uses approximate value iteration, our second ap- proach uses searches to generate training data. We show that in some domains our approaches outperform previous work, and we point out potentials for future improvements.

There exists no planning algorithm that outperforms all oth- ers. Therefore, it is important to know which algorithm works well on a task. A recently published approach uses either im- age or graph convolutional neural networks to solve this prob- lem and achieves top performance. Especially the transforma- tion from the task to an image ignores a lot of information. Thus, we would like to know what the network is learning and if this is reasonable. As this is currently not possible, we take one step back. We identify a small set of simple graph features and show that elementary and interpretable machine learning techniques can use those features to outperform the neural network based approach. Furthermore, we evaluate the importance of those features and verify that the performance of our approach is robust to changes in the training and test data.

Automated planning is one of the foundational areas of AI. Since no single planner can work well for all tasks and domains, portfolio-based techniques have become increasingly popular in recent years. In particular, deep learning emerges as a promising methodology for online planner selection. Owing to the recent development of structural graph representations of planning tasks, we propose a graph neural network (GNN) approach to selecting candidate planners. GNNs are advantageous over a straightforward alternative, the convolutional neural networks, in that they are invariant to node permutations and that they incorporate node labels for better inference. Additionally, for cost-optimal planning, we propose a two-stage adaptive scheduling method to further improve the likelihood that a given task is solved in time. The scheduler may switch at halftime to a different planner, conditioned on the observed performance of the first one. Experimental results validate the effectiveness of the proposed method against strong baselines, both deep learning and non-deep learning based. The code is available at https://github.com/matenure/GNN_ planner.

Benchmark data sets are an indispensable ingredient of the evaluation of graph-based machine learning methods. We release a new data set, compiled from International Planning Competitions (IPC), for benchmarking graph classification, regression, and related tasks. Apart from the graph construction (based on AI planning problems) that is interesting in its own right, the data set possesses distinctly different characteristics from popularly used benchmarks. The data set, named IPC, consists of two self-contained versions, grounded and lifted, both including graphs of large and skewedly distributed sizes, posing substantial challenges for the computation of graph models such as graph kernels and graph neural networks. The graphs in this data set are directed and the lifted version is acyclic, offering the opportunity of benchmarking specialized models for directed (acyclic) structures. Moreover, the graph generator and the labeling are computer programmed; thus, the data set may be extended easily if a larger scale is desired. The data set is accessible from \url{this https URL}.

As classical planning is known to be computationally hard, no single planner is expected to work well across many planning domains. One solution to this problem is to use online portfolio planners that select a planner for a given task. These portfolios perform a classification task, a well-known and well-researched task in the field of machine learning. The classification is usually performed using a representation of planning tasks with a collection of hand-crafted statistical features. Recent techniques in machine learning that are based on automatic extraction of features have not been employed yet due to the lack of suitable representations of planning tasks. In this work, we alleviate this barrier. We suggest representing planning tasks by images, allowing to exploit arguably one of the most commonly used and best developed techniques in deep learning. We explore some of the questions that inevitably rise when applying such a technique, and present various ways of building practically useful online portfolio-based planners. An evidence of the usefulness of our proposed technique is a planner that won the cost-optimal track of the International Planning Competition 2018.

This thesis transfers and evaluates the work of Action Schema Networks (ASNets) for domain-dependent policy learning for classical automated planning. First, we will introduce the foundational background of automated planning and deep learning in the form of neural networks. Subsequently, the structure and learning of ASNets will be explained as well as their partially already evaluated performance. Afterwards, the definition of the network for application in the Fast-Downward planning system and necessary extensions to this framework will be explained. This also includes an adapted training and sampling strategy for efficient learning of ASNets. Lastly, an extensive empirical evaluation is conducted to compare the network performance in classical planning to state-of-the-art planners and assess whether these neural networks are suited for this planning field. While it could be seen that ASNets are capable of learning effective policies for application in search, they still have major limitations regarding their ability to generalise and scale. In the end, we propose extensions and modifications based on our evaluation results to improve their performance for further research.