Learning Robotic Perception Through Prior Knowledge

Intelligent robots must be able to learn; they must be able to adapt their behavior based on experience. But generalization from past experience is only possible based on assumptions or prior knowledge (priors for short) about how the world works. I study the role of these priors for learning perception. Although priors play a central role in machine learning, they are often hidden in the details of learning algorithms. By making these priors explicit, we can see that currently used priors describe the world from the perspective of a passive disinterested observer. Such generic AI priors are useful because they apply to perception scenarios where there is no robot, such as image classification. These priors are still useful for learning robotic perception, but they miss an important aspect of the problem: the robot. Robots are neither disinterested nor passive. They are trying to achieve tasks by interacting with the world around them, which adds structure to the problem and affords new kinds of priors, which I call robotic priors. The questions are: What are the right robotic priors and how can they be used to enable learning? I investigate these questions in three different perception problems based on raw visual input: 1. learning object segmentation for picking up objects in the Amazon picking challenge, 2. learning state estimation for localization and tracking, and 3. unsupervised learning of state representations that facilitate reinforcement learning. To solve these problems, I propose three sources of prior knowledge—1. the robot’s task, 2. robotic algorithms, and 3. physics—and develop ways to encode these priors for the corresponding learning problems. Some of these priors are best encoded as hard constraints that restrict the space of hypotheses considered during learning. Other priors are more suitable to be encoded as preferences for certain hypotheses in the form of learning objectives. My experiments across these problems consistently show that robotic-specific prior knowledge leads to more efficient learning and improved generalization. Based on these results, I propose to take a prior-centric perspective on machine learning, from which follows that we need robotics-specific machine learning methods that incorporate appropriate priors.