Feature selection is an effective data reduction technique. SHapley Additive exPlanations (SHAP) can be used to provide a feature importance ranking for models built with labeled or unlabeled data. Thus, one may use the SHAP feature importance ranking in a feature selection technique by selecting the k highest ranking features. Furthermore, this SHAP-based feature selection technique is applicable regardless of the availability of labels for data. We use the Kaggle Credit Card Fraud detection dataset to simulate three label availability scenarios. When no labeled data is available, unsupervised learners should be used. We explore feature selection for data reduction with Isolation Forest and SHAP for this case. When data of one class is available, a one-class classifier, such as Gaussian Mixture Model (GMM) can be used in combination with SHAP for determining feature importance, and for feature selection. Finally, if labeled data from both classes is available a binary-class classifier can be used in conjunction with SHAP for data reduction. Our contribution is to provide a comparative analysis of features selected in the three label availability scenarios. Our primary conclusion is that feature sets may be reduced with SHAP without compromising performance. To the best of our knowledge, this is the first study to explore a feature analysis technique, applicable in the three label availability scenarios.

Detecting fraudulent activities in credit card transactions can be challenging due to issues like high dimensionality and class imbalance that are often present in the datasets. To address these challenges, data reduction techniques such as data sampling and feature selection have become essential. In this study, we compare four approaches for data reduction: using data sampling alone, employing feature selection alone, applying data sampling followed by feature selection, and using feature selection followed by data sampling. Additionally, we include results using all features. We build classification models using five Decision Tree-based classifiers and Logistic Regression, and evaluate their performance using two performance metrics: the Area Under the Receiver Operating Characteristic Curve (AUC) and the Area under the Precision-Recall Curve (AUPRC). In this work, we adopt ensemble supervised feature selection (SFS) techniques and Random Undersampling (RUS) for data reduction. The experimental results demonstrate that all four data reduction techniques have the potential to improve the performance of classifiers. These results are valuable since the classifiers available are dependent upon application domains, computing environments, and licensing agreements. However, these techniques can be applied independently of all these dependencies. We recommend utilizing the ensemble SFS followed by RUS (SFS-RUS) approach as the preferred data reduction method due to its ability to run feature selection and data sampling in parallel. Additionally, we find that XGBoost and CatBoost outperform other classifiers.

One consequence of the widespread use of (IoT) devices is an increase in the volume of attacks on (IoT) networks. In this study, we focus on the Bot-IoT dataset, with the aim of classifying its four types of attacks: Denial-of-Service (DoS), Distributed Denial-of-Service (DDoS), Reconnaissance, and Information Theft. Our contribution is based on the evaluation of the Random Undersampling (RUS) technique and ensemble Feature Selection Techniques (FSTs). Our results indicate that RUS has a positive impact on overall classification performance. Furthermore, our results show that the FSTs are beneficial for DoS, Reconnaissance, and Information Theft classification but not for DDoS classification. Finally, we note that the ensemble classifiers have generally outperformed the nonensemble classifiers in our study.

In this paper, we investigate the impact of Random Undersampling (RUS) on a supervised Machine Learning task involving highly imbalanced Big Data. We present the results of experiments in Medicare Fraud detection. To the best of our knowledge, these experiments are conducted with the largest insurance claims datasets ever used for Medicare Fraud detection. We obtain two datasets from two Big Data repositories provided by the United States government's Centers for Medicare and Medicaid Services. The larger of the two datasets contains nearly 174 million instances, with a minority to majority class ratio of approximately 0.0039. Our contribution is to show that RUS has a detrimental effect on a Medicare Fraud detection task when performed on large scale, imbalanced data. The effect of RUS is apparent in the Area Under the Precision Recall Curve (AUPRC) scores recorded from experimental outcomes. We use four popular, open-source classifiers in our experiments to confirm the negative impact of RUS on their AUPRC scores.

In the domain of Medicare insurance fraud detection, handling imbalanced Big Data and high dimensionality remains a significant challenge. This study assesses the combined efficacy of two data reduction techniques: Random Undersampling (RUS), and a novel ensemble supervised feature selection method. The techniques are applied to optimize Machine Learning models for fraud identification in the classification of highly imbalanced Big Medicare Data. Utilizing two datasets from The Centers for Medicare & Medicaid Services (CMS) labeled by the List of Excluded Individuals/Entities (LEIE), our principal contribution lies in empirically demonstrating that data reduction techniques applied to these datasets significantly improves classification performance. The study employs a systematic experimental design to investigate various scenarios, ranging from using each technique in isolation to employing them in combination. The results indicate that a synergistic application of both techniques outperforms models that utilize all available features and data. Moreover, reduction in the number of features leads to more explainable models. Given the enormous financial implications of Medicare fraud, our findings not only offer computational advantages but also significantly enhance the effectiveness of fraud detection systems, thereby having the potential to improve healthcare services.

In the context of high-dimensional credit card fraud data, researchers and practitioners commonly utilize feature selection techniques to enhance the performance of fraud detection models. This study presents a comparison in model performance using the most important features selected by SHAP (SHapley Additive exPlanations) values and the model’s built-in feature importance list. Both methods rank features and choose the most significant ones for model assessment. To evaluate the effectiveness of these feature selection techniques, classification models are built using five classifiers: XGBoost, Decision Tree, CatBoost, Extremely Randomized Trees, and Random Forest. The Area under the Precision-Recall Curve (AUPRC) serves as the evaluation metric. All experiments are executed on the Kaggle Credit Card Fraud Detection Dataset. The experimental outcomes and statistical tests indicate that feature selection methods based on importance values outperform those based on SHAP values across classifiers and various feature subset sizes. For models trained on larger datasets, it is recommended to use the model’s built-in feature importance list as the primary feature selection method over SHAP. This suggestion is based on the rationale that computing SHAP feature importance is a distinct activity, while models naturally provide built-in feature importance as part of the training process, requiring no additional effort. Consequently, opting for the model’s built-in feature importance list can offer a more efficient and practical approach for larger datasets and more intricate models.

Classification models serve as effective tools for Medicare fraud detection, but their performance can be influenced by a number of factors. This paper focuses on addressing two common challenges using the Medicare Part D Big Data: high dimensionality and class imbalance. To tackle these issues, we explore the use of feature selection, Random Undersampling (RUS), and a combination of our novel feature selection technique followed by RUS. RUS is employed to create five different class ratios, while six supervised feature selection methods are utilized within ensemble feature selection techniques. The performance of six machine learning classifiers is evaluated for Medicare fraud detection, using Area Under the Receiver Operating Characteristic Curve (AUC) and Area Under the Precision-Recall Curve (AUPRC). Additionally, we compare the performance of models built with the original dataset to establish the superiority of our technique over a baseline approach. The results clearly demonstrate that reducing the training datasets significantly using RUS with a minority:majority class ratio of 1:81 and a feature subset size of ten leads to similar or improved model performance compared to models built with the original dataset in terms of AUPRC metric. Additionally, the data reduction allows for faster model training times. Therefore, details on how the results were obtained are a contribution to Big Data research.

Using the wrong metrics to gauge classification of highly imbalanced Big Data may hide important information in experimental results. However, we find that analysis of metrics for performance evaluation and what they can hide or reveal is rarely covered in related works. Therefore, we address that gap by analyzing multiple popular performance metrics on three Big Data classification tasks. To the best of our knowledge, we are the first to utilize three new Medicare insurance claims datasets which became publicly available in 2021. These datasets are all highly imbalanced. Furthermore, the datasets are comprised of completely different data. We evaluate the performance of five ensemble learners in the Machine Learning task of Medicare fraud detection. Random Undersampling (RUS) is applied to induce five class ratios. The classifiers are evaluated with both the Area Under the Receiver Operating Characteristic Curve (AUC), and Area Under the Precision Recall Curve (AUPRC) metrics. We show that AUPRC provides a better insight into classification performance. Our findings reveal that the AUC metric hides the performance impact of RUS. However, classification results in terms of AUPRC show RUS has a detrimental effect. We show that, for highly imbalanced Big Data, the AUC metric fails to capture information about precision scores and false positive counts that the AUPRC metric reveals. Our contribution is to show AUPRC is a more effective metric for evaluating the performance of classifiers when working with highly imbalanced Big Data.

As a means of building explainable machine learning models for Big Data, we apply a novel ensemble supervised feature selection technique. The technique is applied to publicly available insurance claims data from the United States public health insurance program, Medicare. We approach Medicare insurance fraud detection as a supervised machine learning task of anomaly detection through the classification of highly imbalanced Big Data. Our objectives for feature selection are to increase efficiency in model training, and to develop more explainable machine learning models for fraud detection. Using two Big Data datasets derived from two different sources of insurance claims data, we demonstrate how our feature selection technique reduces the dimensionality of the datasets by approximately 87.5% without compromising performance. Moreover, the reduction in dimensionality results in machine learning models that are easier to explain, and less prone to overfitting. Therefore, our primary contribution of the exposition of our novel feature selection technique leads to a further contribution to the application domain of automated Medicare insurance fraud detection. We utilize our feature selection technique to provide an explanation of our fraud detection models in terms of the definitions of the selected features. The ensemble supervised feature selection technique we present is flexible in that any collection of machine learning algorithms that maintain a list of feature importance values may be used. Therefore, researchers may easily employ variations of the technique we present.

Machine learning research on Medicare fraud detection is of national importance, primarily due to the extensive financial losses caused by this deceptive practice. Our big data study focuses on the Medicare Part D dataset, which we utilize to detect healthcare fraud perpetrated by physicians. In this paper, we compare and contrast One-Class Classification (OCC) and binary classification by examining eight different classifiers. The metrics applied in this analysis are Area Under the Receiver Operating Characteristic Curve (AUC) and Area Under the Precision-Recall Curve (AUPRC). Our findings indicate that binary classification outperforms OCC in Medicare fraud detection. Furthermore, we establish that the Decision Tree-based classifiers employed in the research are the most effective, with CatBoost delivering the best performance.

One-class classifiers hold promise for applications like fraudulent credit card transaction identification. However, interpreting these models to understand which features drive predictions is challenging. Such an understanding is necessary to avoid brute-force approaches to feature selection. This paper explores SHAP (SHapley Additive exPlanations) for feature selection with one-class classffiers on a credit card fraud dataset. We apply SHAP to select key features and evaluate One-Class Gaussian mixture models and One-Class Support Vector Machines. Statistical analysis tests show Gaussian mixture models built with SHAP-selected features perform significantly better than Gaussian mixture models built without feature selection. To the best of our knowledge, we are the first to show the benefit of SHAP-based feature selection to One-Class Gaussian mixture models. Moreover, we show that robust performance with features of the full dataset may be a prerequisite in order for SHAP feature selection to impart further gains. Our results provide novel evidence that SHAP can identify informative features for one-class classifiers.

Research into machine learning methods for fraud detection is of paramount importance, largely due to the substantial financial implications associated with fraudulent activities. Our investigation is centered around the Credit Card Fraud Dataset and the Medicare Part D dataset, both of which are highly imbalanced. The Credit Card Fraud Detection Dataset is large data and contains actual transactional content, which makes it an ideal benchmark for credit card fraud detection. The Medicare Part D dataset is big data, providing researchers the opportunity to examine national trends and patterns related to prescription drug usage and expenditures. This paper presents a detailed comparison of One-Class Classification (OCC) and binary classification algorithms, utilizing eight distinct classifiers. OCC is a more appealing option, since collecting a second label for binary classification can be very expensive and not possible to obtain within a reasonable time frame. We evaluate our models based on two key metrics: the Area Under the Precision-Recall Curve (AUPRC)) and the Area Under the Receiver Operating Characteristic Curve (AUC). Our results show that binary classification consistently outperforms OCC in detecting fraud within both datasets. In addition, we found that CatBoost is the most performant among the classifiers tested. Moreover, we contribute novel results by being the first to publish a performance comparison of OCC and binary classification specifically for fraud detection in the Credit Card Fraud and Medicare Part D datasets.

The yearly increase in incidents of credit card fraud can be attributed to the rapid growth of e-commerce. To address this issue, effective fraud detection methods are essential. Our research focuses on the Credit Card Fraud Detection Dataset, which is a widely used dataset that contains real-world transaction data and is characterized by high class imbalance. This dataset has the potential to serve as a benchmark for credit card fraud detection. Our work evaluates the effectiveness of two supervised learning classification techniques, binary classification and one-class classification, for credit card fraud detection. The performance of five binary-class classification (BCC) learners and three one-class classification (OCC) learners is evaluated. The metrics used are area under the precision-recall curve (AUPRC) and area under the receiver operating characteristic curve (AUC). Our results indicate that binary classification is a better approach for detecting credit card fraud than one-class classification, with the top binary classifier being CatBoost.

Our study evaluates the impact of training OneClass Classification (OCC) algorithms on the majority class compared to training them on the minority class, using large and big datasets that are highly imbalanced. It is important to note that class availability can present a significant challenge in model training. In reality, there may be situations where one class is readily obtainable within a reasonable time frame while another class is not. Our task involves detecting instances of fraud in the Credit Fraud Detection Dataset and our Medicare dataset derived from Medicare Part D data and List of Excluded Individuals and Entities (LEIE) data. The Credit Card Fraud Detection Dataset has real-world transaction content as well as a significant class imbalance, making it suitable for use as a benchmark for credit card fraud detection. In addition, it is the only publicly available large data for credit card fraud analysis. Part D is big data, allowing researchers to analyze national trends and patterns in prescription drug usage and expenditures. The algorithms used in the study are One-Class Gaussian Mixture Model (GMM), OneClass Adversarial Nets (OCAN), and One-Class Support Vector Machine (SVM). Their performance is measured with the Area Under the Precision-Recall Curve (AUPRC) and Area Under the Receiver Operating Characteristic Curve (AUC). Our results indicate that OCC produces better results when models are trained on the majority class.

Identifying fraudulent activities in credit card transactions is an inherent component of financial computing. The focus of our research is on the Credit Card Fraud Detection Dataset, which is widely used due to its authentic transaction data. In numerous machine learning applications, feature selection has become a crucial step. To improve the chance of discovering the globally optimal feature set, we employ ensembles of feature ranking methods. These ensemble methods merge multiple feature ranking lists through a median approach. We conduct a comprehensive empirical study that examines two different ensembles of feature ranking techniques, including an ensemble of twelve threshold-based feature selection (TBFS) techniques and an ensemble of five supervised feature selection (SFS) techniques. Additionally, we present results where all features are used. We construct classification models using two Decision Tree-based classifiers, CatBoost and XGBoost, and evaluate them using two different performance metrics, the Area Under the Receiver Operating Characteristic Curve (AUC) and the Area under the Precision-Recall Curve (AUPRC). Since AUPRC provides a more accurate representation of the number of false positives, especially for highly imbalanced datasets, evaluating models for AUPRC is a wise choice. The experimental results demonstrate that the ensemble of SFS and all features performs similarly or better than the ensemble of TBFS. Moreover, we find that XGBoost outperforms CatBoost in terms of AUPRC.

In the context of high-dimensional credit card fraud data, feature selection techniques are commonly employed by researchers and practitioners to enhance the performance of credit card fraud detection models. This study presents a comparison in model performance using the most important features selected by SHAP (SHapley Additive exPlanations) values and the model's built-in feature importance list. Both methods rank features and select the most important features for model evaluation. The performance of these feature selection techniques is assessed by building classification models using two classifiers: XGBoost and Decision Tree. The evaluation metric used to measure effectiveness is the Area under the Precision-Recall Curve (AUPRC). All experiments are conducted on the Credit Card Fraud Detection Dataset from Kaggle. The experimental results and z-tests demon-strate that, in most cases, there is no significant difference between the two feature selection methods, regardless of the feature subset size and classifier used. However, for models trained on larger datasets, it is suggested to use the model's built-in feature importance list as the preferred feature selection method over SHAP. The rationale behind this recommendation is that computing SHAP feature importance is a separate activity, and models provide feature importance as a side-effect of training, thus requiring no extra effort. Therefore, opting for the model's built-in feature importance list can offer a more efficient and manageable approach for larger datasets and more complex models.

We present findings from experiments in Medicare fraud detection, that are the result of research on two new, publicly available datasets. In this research, we employ popular, open-source Machine Learning algorithms to identify fraudulent healthcare providers in Medicare insurance claims data. As far as we know, we are the first to publish a study that includes datasets compiled from the latest Medicare Part B and Medicare Part D data. The datasets became available in 2021, and are the largest such datasets that we know of. We report details on two important findings. The first finding is that increased maximum tree depth is associated with the best performance in terms of area under the receiver-operating characteristic curve (AUC) for both datasets. The second finding, which is an important counterbalance to the first finding, is that one may utilize random undersampling (RUS) to reduce the size of the training data and simultaneously achieve similar or better AUC scores.To the best of our knowledge, our study is novel in reporting the importance of maximum tree depth for classifying imbalanced Big Data. Moreover, this work is unique in demonstrating that one may employ RUS to mitigate the increased resource consumption of higher maximum tree depth.

Output thresholding is well-suited for addressing class imbalance, since the technique does not increase dataset size, run the risk of discarding important instances, or modify an existing learner. Through the use of the Credit Card Fraud Detection Dataset, this study proposes a threshold optimization approach that factors in the constraint True Positive Rate (TPR) ≥ True Negative Rate (TNR). Our findings indicate that an increase of the Area Under the Precision–Recall Curve (AUPRC) score is associated with an improvement in threshold-based classification scores, while an increase of positive class prior probability causes optimal thresholds to increase. In addition, we discovered that best overall results for the selection of an optimal threshold are obtained without the use of Random Undersampling (RUS). Furthermore, with the exception of AUPRC, we established that the default threshold yields good performance scores at a balanced class ratio. Our evaluation of four threshold optimization techniques, eight threshold-dependent metrics, and two threshold-agnostic metrics defines the uniqueness of this research.

Practitioners and researchers of machine learning should have a deep understanding about the selection of the right performance metrics for classifier evaluation. Using a credit card fraud dataset, we demonstrate that the Area Under the Precision-Recall Curve (AUPRC) metric is a more reliable measurement, for the classification of highly imbalanced data, than the Area Under the Receiver Operating Characteristic Curve (AUC) metric. Furthermore, we establish that AUC is minimally impacted by the use of Random Undersampling (RUS). The classifiers used in this study are ensemble learners: LightGBM, CatBoost, Extremely Randomized Trees (ET), XGBoost, and Random Forest. Our results are governed by the fact that in a highly imbalanced dataset, the comparatively large number of true negative instances has an influence on AUC but not on AUPRC. Hence, AUPRC is able to accurately detect changes in the number of false positives because it ignores the true negatives.