Improving Calibration of BatchEnsemble with Data Augmentation

Efficient ensembles such as BatchEnsemble are a simple drop-in approach to improving a model’s accuracy and calibration across in-and out-ofdistribution data. While they bridge the performance gap between single model performance and independent deep ensembles, their improvements on calibration are not as substantial as that on accuracy. We examine how to further improve calibration of these models. We investigate the role of data augmentation and show that augmentation techniques which improve single models can surprisingly make the ensemble calibration even worse. We propose a new data augmentation that fixes this pathology and improves BatchEnsemble’s calibration. We empirically demonstrate the effectiveness of our approaches on in-and out-of-distribution CIFAR-10, CIFAR-100.