Semantic Scene Filtering for Event Cameras in Long-Term Outdoor Monitoring Scenarios

Event cameras are biologically inspired devices. They are fundamentally different from conventional frame-based sensors in that they directly transmit an (x, y, t) output stream of asynchronously and independently detected changes in brightness. For the development of monitoring systems, scenario-based long-term experiments are much more representative than day-to-day experiments. However, unconstrained "real-world" factors pose processing challenges. To perform a semantic scene filtering on the output stream of an event camera in such an outdoor monitoring scenario, this paper describes a multi-stage processing chain. The goal is to identify and store only those segments that contain events that were triggered by a specific set of objects of interest. The main idea of the proposed processing pipeline is to pre-process the data stream using different filters to identify Patches-Of-Interest (PoIs). These PoIs, natively represented as space-time event clouds, are further processed by PointNet++, a 3D-based semantic segmentation network. An evaluation was performed on about 89 h of realworld outdoor sensor data, achieving a semantic filtering with a false negative rate of approximate to 3.8% and a true positive rate of approximate to 96.2%.