Synergetic KDRF Classifier: A Random Forest-Based Hybrid Approach for Multi-Class Classification Task

Multi-class classification presents a significant challenge in supervised machine learning, and it is frequently applied across various real-world domains. Random Forest (RF) stands out as a popular ensemble technique for multi-class classification tasks that utilizes attribute bagging to construct random trees and employs a majority-voting scheme for classification. However, the existing Random Forest approach often faces limitations in accuracy and efficiency, particularly for more critical tasks. In this study, we have addressed this gap by introducing a novel methodology that integrates the K-Nearest Neighbor (KNN) classifier and Decision Trees into the Random Forest framework to enhance performance. The proposed method has involved training a K-Nearest Neighbor and a Decision Tree (DT) classifier on the extracted features. The output probabilities from these classifiers have been used as meta-features, forming an enriched feature vector fed into a Random Forest as the meta-learner for final classification. Through extensive experimentation on 5 datasets, the proposed approach has demonstrated superior performance in accuracy and efficiency compared to the traditional method. By integrating Random Forest and K-Nearest Neighbor techniques, this research has offered a versatile solution applicable across diverse domains where precise multi-class classification is essential.