Pruning-Based Ensemble Tree for Multi-Class Classification

Decision tree is one of the most popular classifiers in supervised learning to address classification tasks. It is easy to build and easy to understand. It is a divide-and-conquer algorithm and provides a clear path for making decisions. Decision tree can be used for both regression and classification problems. It finds the best splitting features as a root node and create sub-datasets. A leaf node of decision tree represents a decision where all and majority of the instances of a sub-dataset belongs to a particular decision or class-value. The commonly used algorithms are ID3 (Iterative Dichotomiser 3), C4.5 (improvement over ID3 algorithm), and CART (Classification And Regression Tree). Decision tree can be easily scalable and apply for Big Data mining. In this paper, we have proposed a new pruning-based tree ensemble method where each tree is build from the each input feature of the dataset. Therefore, $N$ number of trees will be created for $N$ number of features. Then, we apply post pruning technique on each tree and select the best trees according to their cost-complexity. We have considered the size and accuracy of the tree to select in the ensemble process. The proposed approach surpasses the performance of the C4.5 classifier by almost 11 % for the selected benchmark datasets taken from the UCI machine learning repository.