STADIE-Net: Stagewise Disparity Estimation from Stereo Event-based Cameras

Event-based cameras complement the frame based cameras in low-light conditions and high dynamic range scenarios that a robot can encounter for scene understanding and navigation. Apart from that, the comparatively cheaper cost in relation to a LiDAR sensor makes this a viable candidate when designing a sensor suite for a robot designed to operate in a dynamic environment. One of the challenges that the sensor suite needs to address is the ability to provide a 3D scene understanding of the environment that would enable the robot to localize obstacles in the environment. This work evaluates the accuracy with which an event-based camera can support this task by providing the disparity estimate between left and right camera frame which can be utilized to calculate the depth of surrounding. A new deep network architecture, named STADIE-Net is proposed that takes advantage of stagewise refinement and prediction of disparity using events from 2 neuromorphic cameras in a stereo setup. The method utilizes voxel grid representation for events as input and proposes a 4 stage network going from coarse to finer disparity prediction. The model is trained and evaluated on the publicly released DSEC dataset that has data recorded from multiple cities using event-based and frame-based cameras mounted on a moving vehicle. Experimental results show comparable accuracy with baseline method provided for DSEC dataset.