Selection Bias Correction for Imbalanced Samples

Selection bias correction is generally crucial to have a reliable inference for a nonprobability sample. Often the correction is applied with the two-sample setup. That is, along with the nonprobability sample we are interested in, a probability sample sharing some common auxiliary variables is used for constructing correction weights for the nonprobability sample. The two-sample setup allows one to calculate weighted estimates for population parameters of interest based on the nonprobability sample. Since the nonprobability sample is usually easy to collect, we often end up with a large nonprobability sample and a small probability sample. The imbalance of the two samples may cause difficulties in modelling the propensity of units to be included in the nonprobability sample. This paper discusses some often-seen solutions for imbalanced samples in machine learning literature, i.e. undersampling, Synthetic Minority Oversampling Technique (SMOTE), and a mixture of both. A selection bias correction framework is adjusted to incorporate the imbalance solutions. Three evaluation studies across different types of data sets are shown. The results indicate that SMOTE has the potential to deal with imbalances in selection bias correction, while further study is needed to identify in which scenarios SMOTE works well.