A Version Space Perspective on Differentially Private Pool-Based Active Learning

We analyze pool-based active learning under a differential privacy guarantee. At every active learning step, some samples are selected to be labeled by an oracle, and the new labels are used to update the classifier. We want to preserve differential privacy during both the sample selection step and the classifier update step. To study the evolution of the active learner, we use the concept of a version space of possible hypotheses (classifiers). This concept helps establish a principled notion of the informativeness of a pool sample: When informative samples are labeled and used for training, the version space shrinks, yielding classifiers consistent with the labeled samples. To provide privacy, we query the oracle with both informative and non-informative samples using a simple randomized sampling scheme. We prove the privacy guarantee, and characterize the increase in label complexity resulting from our randomized sampling strategy. To examine how our theoretical analysis manifests in practice, we built an SVM-based active learner, and measured the accuracy and label complexity achieved with and without privacy.