A Multi-Sensor Video/LiDAR System for Analyzing Intersection Safety.

We introduce an integrated video and LiDAR analytics system for analyzing pedestrian and vehicle behavior at traffic intersections. Subsystems for each modality leverage advanced deep-learning techniques to detect pedestrians and vehicles and then use a Kalman-filter-based tracking algorithm to generate tracks. The video and LiDAR tracks are then aligned spatiotemporally onto the same coordinate system with synchronized clocks. We evaluate the benefits of these two modalities by providing both qualitative and quantitative comparisons, utilizing low-level measures such as detection and tracking accuracy, as well as high-level measures such as severe events. Additionally, we compare the two modalities at different times of the day and show that LiDAR is competitive with video during daylight hours and significantly outperforms video at late evening when lighting conditions are poor. To the best of our knowledge, this study represents the first detailed comparison of these two modalities for observing traffic intersections.