A Delay Metric for Video Object Detection: What Average Precision Fails to Tell

2019 IEEE/CVF International Conference on Computer Vision (ICCV)(2019)

Cited 44|Views4
No score
Abstract
Average precision (AP) is a widely used metric to evaluate detection accuracy of image and video object detectors. In this paper, we analyze the object detection from video and point out that mAP alone is not sufficient to capture the temporal nature of video object detection. To tackle this problem, we propose a comprehensive metric, Average Delay (AD), to measure and compare detection delay. To facilitate delay evaluation, we carefully select a subset of ImageNet VID, which we name as ImageNet VIDT with an emphasis on complex trajectories. By extensively evaluating a wide range of detectors on VIDT, we show that most methods drastically increase the detection delay but still preserve mAP well. In other words, mAP is not sensitive enough to reflect the temporal characteristics of a video object detector. Our results suggest that video object detection methods should be evaluated with a delay metric, particularly for latency-critical applications such as autonomous vehicle perception.
More
Translated text
Key words
video object detector,video object detection methods,delay metric,average precision,Average Delay,detection delay,delay evaluation,autonomous vehicle perception,latency-critical applications,ImageNet VID
AI Read Science
Must-Reading Tree
Example
Generate MRT to find the research sequence of this paper
Chat Paper
Summary is being generated by the instructions you defined