Large multimodal datasets gathered from the internet have been a key driver of progress in recent image-text models such as DALL-E, CLIP, and Flamingo. However, structured prediction tasks have not seen the same benefits, as noisy fine-grained annotations do not exist at web-scale. In this paper, we aim to extend the gains enabled by web-sourced training sets to the problem of object detection. First, we show that data curation for grounding and localization necessitates its own approach: filtering methods which produce good datasets for image classification with CLIP models (e.g., the image-text similarity filtering from LAION-5B) do not yield better object detectors. Instead, we introduce new detection-focused filtering methods that match or outperform existing object detectors pretrained on supervised detection pretraining datasets. When trained on 102.4M images from the 12.8B image DataComp pool in a weakly supervised manner, our new filtering method matches the performance of a detector pre-trained on Object365, the largest fully-annotated detection dataset. In addition, our filtering approach shows good scaling with training set size and can be combined with Object365 to yield further improvements. To aid further research in this area, we release a 2.8B image subset of DataComp-12.8B pseudo-labeled with region proposals and detection bounding boxes.
分享
引用
