Uncertainty sampling methods iteratively request class labels for training instances whose classes are uncertain despite the previous labeled instances. These methods can greatly reduce the number of instances that an expert need label. One problem with this approach is that the classifier best suited for an application may be too expensive to train or use during the selection of instances. We test the use of one classifier (a highly efficient probabilistic one) to select examples for training another (the C4.5 rule induction program). Despite being chosen by this heterogeneous approach, the uncertainty samples yielded classifiers with lower error rates than random samples ten times larger.

Quinlan's machine learning system ID3 uses a method called windowing to deal economically with large training sets. This paper describes a series of experiments performed to investigate the merits of this technique. In nearly every experiment the use of windowing considerably increased the CPU requirements of ID3, but produced no significant benefits. We conclude that in noisy domains (where ID3 is now commonly used), windowing should be avoided.