3 dim : Compact and Low Power Time-of-Flight Sensor for 3 D Capture Using Parametric Signal Processing

We introduce an active optical sensor, 3dim, for capturing three-dimensional (3D) scene structure. 3dim is an active time-of-flight (TOF) technique, which is based on parametric modeling and processing of scene impulse responses. As opposed to estimating one depth value per pixel in a conventional TOF sensor, 3dim operates by estimating multiple object depths at each of the baseline-separated photodetectors. 3dim offers simple hardware construction, high frame rates, small size, insensitivity to ambient light, and low power consumption. Unlike existing depth sensors, it does not provide a depth map with full spatial resolution for general scenes, but rather high 3D object localization accuracy in application-specific scenarios. Our 3dim prototype is built using standard, commercially-available components. We demonstrate centimeter range resolution for gesture tracking applications on mobile platforms, a resolution that is very fine relative to the system bandwidth and optical power. Overview In the post-PC era, 3D sensing is being increasingly integrated in consumer electronic devices to enable gesture-based user interfaces. The demand for accurate, low power and compact depth cameras is thus evident. The past year alone has seen the launch of several sensors for consumer electronics applications such as the Leap Motion Controller [1], ultrasonic touchless technology by Elliptic Labs [2], and compact depth cameras by PrimeSense [3] and PMD [4]. The key performance features that are desirable in 3D sensors are high depth accuracy and resolution, high frame rate, insensitivity to ambient light, and large working volume. It remains difficult to satisfy these performance requirements given the constraints on size and power for mobile devices (see Fig. 1). In this paper, we introduce 3dim, an optical sensor combining a TOF-based module with an RGB camera module to achieve 3D localization and tracking of objects such as users’ hands while gesturing to a mobile device, or a real-world object augmented with virtual information. Unlike TOF cameras and other general-purpose 3D acquisition technologies, 3dim is an application-specific sensor. Its TOF module uses neither focusing optics nor a full-resolution 2D sensor array, and hence it does not produce scene depth maps. Instead, it provides 3D acquisition capabilities specific to tracking gestures and planar objects. The current image processing and computer vision pipeline operates by first capturing a full 2D or 3D image of the scene, then processing to detect objects, and finally performing object parameter estimation. This pipeline works for general scenes but requires significant computation and acquisition resources. It obtains a full-resolution image even though the objects of interest are simple and few. Our acquisition architecture can be used to disrupt the standard image processing and computer vision pipeline. With few sensors and low computational complexity, 3dim can track hands or infer planar object pose and orientation with high accuracy. This obviates high-complexity processing on high-resolution image data. Frame rate Daylight sensitivity Depth resolution Total power Working range 100 fps low < 1 cm < 150 mW 0-2 m Leap 90-100 fps high 1 mm 3-5 W < 0.6 m Ultrasound 50 fps none coarse ~ 2-5 cm ~ 300 mW < 1 m 2D cameras 25 30 fps coarse 200 mW 0 2 m none > 5 cm Structured light 3D 30 fps high coarse > 2 cm 3-5 W 0.8-3 m Time-of-flight 3D 30 fps high < 1 cm 3-5 W 0-2 m Figure 1: Comparison of 3D sensing technologies for gesture-sensing applications. parametric deconvolution! ! recovered peak amplitude and positions! sampled data! source localization! spatial regions! rgb region processing! gesture recognition ...! illuminate scene! + + capture position! process jointly! capture rgb!