An efficient classifier ensemble using SVM

Recently ensemble classification has attracted serious attention of machine learning community as a solution for improving classification accuracy. The effect of the strategies for generating the members, combining the predictions and the size of the ensemble on the accuracy of the ensemble are of utmost interest to the researchers. In this paper, we propose and empirically evaluate a novel method for generating members of ensemble based on 'learning-from-mistakes' paradigm. SVM is used as the base learner, and a series of dependent classifiers is obtained using model based instance selection method. In each iteration, all the wrongly classified records are merged with the support vectors to capture diversity. Classifiers with accuracy higher than the average of the series are selected for ensemble construction. Simple majority voting has been used to combine learners. The approach has been empirically found efficient as very few classifiers need to be generated and even fewer are selected. Accuracy on the test sets is comparable to bagging and boosting based methods.