Fast Onboard Stereo Vision for UAVs

In the last decade researchers have built incredible new capabilities for small aircraft, with quadrotors moving from labs to toy stores and with autonomy reaching smaller and smaller vehicles. As the systems, and their payload capacities shrink, we can no longer use typical aircraft sensors such as RADAR, scanning LIDAR, and other active sensing methods for obstacle detection and avoidance. Smaller vehicles must move to lighter weight sensors. Cameras are lightweight, fast, and information dense but can require sophisticated, often slow, processing to be useful for robotic applications. Here, we demonstrate that highspeed embedded stereo vision offers a way forward for fastflying aerial vehicles. The presented solutions are lighter, provide more information, and use less power than laser ranging systems. They work in outdoor environments that overwhelm active IR depth cameras, and do not suffer from scale observability like monocular camera systems. The new algorithms and processing techniques presented here enable us to detect obstacles faster, with less payload, and less computational hardware than ever before. We present the only two embedded stereo (two-camera) vision systems running at high frame rate with low enough power and weight requirements for small flying platforms. These systems provide full 3D positions of points seen by the cameras, which is essential for obstacle avoidance and safe, robust flight. These two systems were developed independently by ETH Zürich and MIT for solving the problem of providing accurate, rich sensing in a package small enough to fit in the payload constraints of mico-aerial vehicles (MAVs). The two systems differ in their processing, with one using an onboard FPGA (field programmable gate array) to perform dense semi-global matching (SGM) [4], [6] and the second using conventional ARM processors to perform pushbroom stereo [1]. Both systems run at 120 frames per second (fps) at 320x240 pixel resolution, on duplicate fixedwing platforms flying over 30 MPH (13.4 m/s).