MultiTest: Physical-Aware Object Insertion for Testing Multi-sensor Fusion Perception Systems
CoRR(2024)
摘要
Multi-sensor fusion stands as a pivotal technique in addressing numerous
safety-critical tasks and applications, e.g., self-driving cars and automated
robotic arms. With the continuous advancement in data-driven artificial
intelligence (AI), MSF's potential for sensing and understanding intricate
external environments has been further amplified, bringing a profound impact on
intelligent systems and specifically on their perception systems. Similar to
traditional software, adequate testing is also required for AI-enabled MSF
systems. Yet, existing testing methods primarily concentrate on single-sensor
perception systems (e.g., image-/point cloud-based object detection systems).
There remains a lack of emphasis on generating multi-modal test cases for MSF
systems. To address these limitations, we design and implement MultiTest, a
fitness-guided metamorphic testing method for complex MSF perception systems.
MultiTest employs a physical-aware approach to synthesize realistic multi-modal
object instances and insert them into critical positions of background images
and point clouds. A fitness metric is designed to guide and boost the test
generation process. We conduct extensive experiments with five SOTA perception
systems to evaluate MultiTest from the perspectives of: (1) generated test
cases' realism, (2) fault detection capabilities, and (3) performance
improvement. The results show that MultiTest can generate realistic and
modality-consistent test data and effectively detect hundreds of diverse faults
of an MSF system under test. Moreover, retraining an MSF system on the test
cases generated by MultiTest can improve the system's robustness.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要