Implementation of MARS Metrics and MARS Charts for Evaluating Classifier Exclusivity: the Comparative Uniqueness of Binary Classifier Predictions

Traditionally, performance metrics such as accuracy, precision, recall, F-score, and ROC curve/ Area-Under-Curve (AUC) values have been used to evaluate and understand binary classifier capabilities. However, modern high-performance classifier models frequently have equivalent classification performance according to traditional metrics. We propose a novel approach – the MARS classifier evaluation method – to evaluate classifier exclusivity, using MARS ShineThrough and MARS Occlusion scores. Specifically, the MARS method vividly illustrates the extent to which classifiers spot distinct target-class observations, that other classifiers miss. In this paper, we describe the software artifact utilized to calculate MARS metrics for comparative uniqueness and MARS charts for the visualization of these calculations.