Deep multi-label classifiers are often learned on partially labeled datasets, where only a small proportion of labels are annotated. While most existing works aim to attain high classification performance, calibration, a crucial property of classifiers in life-critical applications that measures whether the predicted probabilities can reflect the true likelihood of each category, has received very little attention. We found that the label imbalance-aware asymmetric (ASY) loss produces poorly calibrated classifiers, which may aggravate the generation of unwanted wrong pseudo-labels in pseudo-labeling approaches. In this paper, we propose an interactive learning framework that takes advantage of pseudo-labeling and the ASY loss to train accurate, calibrated classifiers, without threshold tuning and prior knowledge of the data. A dedicated neural network calibrates the classifier’s predicted probabilities for pseudo-label generation, mitigating the calibration problems of the ASY loss. The classifier and calibrator improve each other to frequently update pseudo-labels. Additionally, we propose a post-training method that learns an element-wise linear regression model re-parameterizable into the classifier, enabling the classifier to directly produce calibrated probabilities without extra inference costs. Extensive experiments are conducted on nine partially labeled datasets derived from MS-COCO, VG-200, and NUS-WIDE. We achieve 2.9x-8.7x more mAP improvements than the best of the nine compared pseudo-labeling methods, with significantly lower calibration errors.
原文链接
分享
引用
