Visual Odometry Using a Focal-plane Sensor-processor

With faster processing units and more sophisticated algorithms, the field of computer vision has progressed significantly over the past few years. One of its applications is to provide spatial awareness for the machines, such that they can sense, navigate, and interact with our world. For such machines, ability to react to abrupt changes in the environment is vital. Focal-plane Sensor-processor (FPSP) is a new type of imager, which allows parallel computation to occur on the chip itself. The analog nature of the architecture allows low energy consumption while promising a high frame-rate.Our work presents, to the best of our knowledge, the first successful 6 Degrees of Freedom visual odometry pipeline which uses the data from the FPSP device. It allows a machine to be aware of its position, while it freely moves around in a three-dimesional space. We propose improvements to the existing feature detector for the device and introduce a feature tracker with robustness against noise. Combining the two algorithms, we complete our pipeline with a non-linear pose estimator. Through the exploitation of parallelism and partitioning of the tasks, we achieve latency of less than 5ms per pose estimate and under 15mm in the absolute trajectory pose error.