As the amount of unlabeled data has continued to grow and present challenges to machine learning practitioners, the need for unsupervised solutions is more evident than ever. With many unsupervised algorithms available to classify instances, the challenge remains that these algorithms require fine-tuning and/or appropriate parameter selection to produce reliable results. The difficulty remains that given an unlabeled dataset, the true class distribution is unknown, which impacts appropriateness of the selection of unsupervised algorithms and hyperparameter tuning, as well as the evaluation metrics chosen. Our novel approach addresses this critical gap in current literature. Through a fully automated and unsupervised framework, we take a binary unlabeled dataset, and return the class distribution without prior domain knowledge and regardless of the class distribution - imbalanced or balanced. We thoroughly investigate multiple datasets ranging in size, class distribution, and domain, and our empirical evidence demonstrates the successful determination of the class distribution given this variety of factors. Our approach uses data-driven threshold and parameter settings to improve model performance, particularly in imbalanced class scenarios. This helps in selecting suitable algorithms, guiding appropriate evaluation metrics, and promoting fairer, evidence-based decision making in fields such as fraud detection and healthcare.

The application of machine learning in healthcare continues to gain attention as researchers attempt to prove its potential for the enhancement of diagnosis and prognosis accuracy. Although many applications of machine learning have been well studied, there remain substantial opportunities for advancement. The field of healthcare holds particularly strong potential for improvement from integration with machine learning. In the future, clinicians will likely utilize machine learning to enhance the efficiency of diagnosis and prognosis, optimizing the delivery of care. This study conducts a comprehensive examination of feature selection methodologies, model architectures, and fine-tuning techniques related to diverse diagnostic and prognostic scenarios within the domain of heart health. It addresses some key gaps in earlier research, including the lack of agreement on which data sources are most effective for classifying stroke and heart attack. This review contributes an analysis of current machine learning methods in stroke and heart attack research, highlighting key gaps such as limited use of multimodal data, external validation, and class imbalance mitigation. It suggests improvements, including the adoption of advanced sampling techniques and the use of comprehensive performance metrics. The findings suggest that despite extensive research on machine learning in cardiovascular health, there are gaps to be addressed in methodologies for data collection, preprocessing, model development, evaluation, and feature engineering.

Many datasets remain unlabeled as obtaining labeled data for machine learning is frequently expensive and necessitates a high level of domain expertise. Another challenge facing machine learning practitioners is class imbalance. In domains such as fraud detection, overcoming significant class imbalance presents an additional difficulty, as seen in the Credit Card Fraud and Medicare Part D claims datasets used in this work. Our novel binary labeling method automates the labeling process, with minimal expert input, using the combination of an ensemble unsupervised method with a percentile thresholding technique. The labels are further refined through an iterative minimization process that selects only the highest-confidence instances to receive a final labeling of fraudulent. Our labeling approach successfully overcomes the challenge of generating labels for severely imbalanced data, labeling instances as fraudulent or not, in an entirely unsupervised framework. Additionally, and in contrast to conventional methods, our methodology provides a more efficient evaluation by directly assessing the generated labels’ efficacy without requiring the training of a supervised classifier to evaluate the labels. In order to examine the effect on label efficacy, we report results across a range of positive instance levels for each dataset. The quality of the newly generated class labels is thoroughly assessed using three evaluation metrics: Jaccard Index (JI), Precision, and Matthews Correlation Coefficient (MCC). Our empirical results demonstrate our approach consistently outperforms the baseline, Isolation Forest (IF), for all positive instance levels and metrics. Our novel methodology demonstrates the ability to provide accurate and robust labels and overcome the challenge of class imbalance, which could result in better machine learning applications in highly imbalanced domains and more efficient evaluation of newly generated class labels.

Deep learning (DL), particularly the use of Convolutional Neural Networks (CNNs), has played a significant role in computer vision, including image recognition and classification. This study investigates the performance of three foundational CNN architectures — AlexNet, ResNet, and VGGNet – in addressing class imbalances within image datasets. These architectures are chosen due to their historical significance and widespread adoption as benchmarks, allowing for a consistent evaluation of performance across different scenarios. Specifically, we assess their efficacy using a NASA Mars image dataset characterized by varying levels of class imbalance. Our analysis focuses on comparing how each architecture performs under different imbalance scenarios, ranging from mild to severe. The empirical results demonstrate that while all three models effectively manage mild imbalances, AlexNet’s performance significantly declines as imbalance severity increases, unlike ResNet and VGGNet, which exhibit considerable robustness even under highly imbalanced conditions.

Manual dataset labeling is expensive, time-consuming, and susceptible to noise and inaccuracies, often necessitating significant financial investments with risks of inconsistencies from human annotations. These challenges are further extended in domains such as fraud detection because of privacy concerns due to manual annotations and severe class imbalance, which negatively impact machine learning models. Our unsupervised approach integrates SHapley Additive exPlanations (SHAP) for feature selection with our novel unsupervised labeling method which uses an ensemble unsupervised method in conjunction with a percentile-based threshold technique on the widely used Kaggle Credit Card Fraud Detection dataset. We create subsets with three and five features using unsupervised SHAP-based feature selection to determine the most impactful features, as well as use the full-featured dataset. To evaluate, we compare the newly generated binary class labels to the actual labels, which were only used for evaluation, and calculate Matthews Correlation Coefficient (MCC), Jaccard Index (JI), and Precision. Furthermore, we compare our method to an unsupervised baseline and show significant improvements. Our empirical results demonstrate that unsupervised SHAP-based feature selection consistently improves the quality of our labels, when compared to the baseline unsupervised method. Lastly, unsupervised SHAP-based feature selection improves label quality when comparing feature subsets to the full-feature dataset while reducing computational complexity. Our work provides an unsupervised framework capable of addressing the challenges of labeling highly imbalanced and unlabeled data while preserving data privacy concerns given the unsupervised nature of our methodology and application of unsupervised SHAP-based feature selection.

Early detection of cognitive impairment is essential for timely intervention and treatment of conditions such as Alzheimer's disease. Supervised machine learning models require labeled data for training, which is often scarce, costly to obtain, and subject to diagnostic uncertainty. In this work, we propose CAE-IF, a fully unsupervised hybrid approach that combines Convolutional Autoencoders (CAE) for feature extraction with Isolation Forest (IF) for anomaly detection. Our method is evaluated on a real-world, imbalanced cognitive dataset derived from the Health and Retirement Study (HRS). CAE-IF consistently outperforms two baseline unsupervised models, Local Outlier Factor (LOF) and Isolation Forest, across key evaluation metrics. For AUPRC, CAE-IF achieves 0.3042 compared to 0.2164 for LOF and 0.2490 for IF. For F1-score, CAE-IF achieves 0.3380, outperforming LOF (0.2089) and IF (0.2640). For MCC, CAE-IF scores 0.1818, higher than LOF (0.0323) and IF (0.0892). These results demonstrate the superior performance of CAE-IF in detecting cognitive impairment under class imbalance. These results indicate that CAE-IF can serve as a useful tool for early screening of cognitive impairment using survey-based data.

There is a growing need for labeled data, yet manual annotation is costly, error-prone, and often infeasible in privacy-sensitive, highly imbalanced domains such as fraud detection. We introduce a fully unsupervised framework that combines unsupervised SHapley Additive exPlanations (SHAP) feature selection with our novel unsupervised labeling method. We apply unsupervised SHAP to the Kaggle Credit Card Fraud Detection and Medicare Part D datasets to produce high-impact feature subsets, and then label the datasets with our unsupervised labeling approach. To effectively evaluate the labels generated by our novel methodology, we apply a baseline unsupervised learner, Isolation Forest (IF), to both the original datasets and their subsets. We calculate Matthew’s Correlation Coefficient (MCC), Jaccard Index (JI), Precision, Recall, and F1-score by comparing our generated labels against the ground truth labels. It is important to note, the ground truth labels were used solely for evaluation. Our empirical results surpass the results obtained with the full feature dataset and baseline. By improving label quality while reducing computational complexity and preserving privacy, our approach offers a practical solution for learning from unlabeled, severely imbalanced data.

Recent advances in machine learning have resulted in techniques that are effective in complex scenarios, such as those with many rare classes or with multimodal data; in particular, low-shot learning (LSL) is a challenging task for which multiple strong approaches have been developed. We hypothesize that these techniques’ effectiveness against the data scarcity within LSL may translate to effectiveness against the data scarcity within more “traditional” supervised, imbalanced, binary classification tasks such as fraud detection; however, there has been relatively little research which applies them in these contexts. In this paper, we aim to fill this gap by selecting two LSL papers from prior literature (representing two major approaches to LSL, optimization-based and contrastive), and reevaluate their models on two highly-imbalanced tabular fraud detection datasets, including a “big-data” Medicare dataset. To the best of our knowledge, our work is the first to directly compare optimization-based and contrastive approaches in any setting, and the first work to examine either of these approaches on a tabular big-data task. We find that the contrastive learning method we test, Siamese-RNN, performs on par with state-of-the-art non-LSL baseline learners for especially big and severely imbalanced data, and significantly outperforms them for smaller and less severely imbalanced data.

This study presents a novel approach for generating high-quality class labels for severely imbalanced healthcare fraud data using unsupervised feature selection and labeling techniques. Our results show it is possible that supervised learners can be trained on labels generated with an unsupervised labeling technique and yield performance better than an unsupervised learner. Large unlabeled datasets pose significant challenges for machine learning, particularly in domains like healthcare fraud detection, where class imbalance is extreme; however, manual labeling can be time-consuming and expensive. We propose a labeling technique that combines SHapley Additive exPlanations (SHAP) for unsupervised feature ranking and selection with an autoencoder-based method for label generation. Experiments are conducted on a dataset derived from publicly available Medicare Part D data. The quality of generated labels is evaluated by training multiple supervised classifiers using both our unsupervised labeling method and an unsupervised baseline and comparing their performance. Results also demonstrate that SHAP-based feature selection consistently improves the quality of generated labels across different classifiers and datasets. This work provides an effective framework for addressing the challenges of unlabeled, severely imbalanced healthcare fraud data without requiring manual annotation. The unsupervised nature of the method preserves data privacy while enabling the application of supervised learning techniques to previously unusable datasets.

Detecting anomalies in cognitive data without labeled data can be challenging, especially when data is imbalanced and abnormal cases are rare. Early detection of such anomalies is critical for timely interventions in neurodegenerative disorders. In this study, we present a fully unsupervised approach that combines a Convolutional Autoencoder for feature extraction and Isolation Forest for anomaly detection, called CAE-IF, and we evaluate the impact of class imbalance on the performance of our model (CAE-IF) as well as two baseline models, Isolation Forest (IF) and Local Outlier Factor (LOF). We use five real-world cognitive assessment datasets with varying levels of class imbalance, ranging from balanced (49.3%) to highly imbalanced (3.3%) for our evaluations. The results show that CAE-IF is more resilient to class imbalance than the baseline models. For instance, using the dataset with an imbalance ratio of 19.1%, CAE-IF reaches an F1-score (F1) of 0.3380 and an Area Under the Precision-Recall Curve (AUPRC) of 0.3042 which is significantly higher than IF (F1: 0.2640, AUPRC: 0.2490) and LOF (F1: 0.2123, AUPRC: 0.2164). The performance of IF and LOF for both AUPRC and F1-score steadily declines as the imbalance rate gets more severe. CAE-IF, on the other hand, has more stable performance across all datasets, and significantly outperforms traditional unsupervised models when data is moderately or severely imbalanced.

Feature selection is an effective data reduction technique. SHapley Additive exPlanations (SHAP) can be used to provide a feature importance ranking for models built with labeled or unlabeled data. Thus, one may use the SHAP feature importance ranking in a feature selection technique by selecting the k highest ranking features. Furthermore, this SHAP-based feature selection technique is applicable regardless of the availability of labels for data. We use the Kaggle Credit Card Fraud detection dataset to simulate three label availability scenarios. When no labeled data is available, unsupervised learners should be used. We explore feature selection for data reduction with Isolation Forest and SHAP for this case. When data of one class is available, a one-class classifier, such as Gaussian Mixture Model (GMM) can be used in combination with SHAP for determining feature importance, and for feature selection. Finally, if labeled data from both classes is available a binary-class classifier can be used in conjunction with SHAP for data reduction. Our contribution is to provide a comparative analysis of features selected in the three label availability scenarios. Our primary conclusion is that feature sets may be reduced with SHAP without compromising performance. To the best of our knowledge, this is the first study to explore a feature analysis technique, applicable in the three label availability scenarios.

This survey paper explores the application of machine learning (ML) techniques in the domain of workplace injuries, focusing on three key areas: risk prediction, return to work (RTW) strategies, and demographic analysis. Through an extensive review of literature from January 2015 to July 2024, this paper examines the latest advancements in ML-driven approaches to workplace safety and identifies important research gaps. This paper highlights how classical ML techniques, such as ensemble models and decision trees, have become essential tools for identifying workplace injury risks, enabling more accurate interventions. It emphasizes the importance of leveraging ML in personalized RTW programs, which use data-driven insights to improve recovery outcomes and reduce economic demands. In the context of demographic analysis, this paper explores how ML algorithms can uncover disparities in injury rates across various age groups, industries, and occupations, underscoring the need for targeted safety measures. Moreover, research gaps are identified, particularly regarding the emerging potential of advanced ML techniques, such as deep learning and large language models (LLMs), for analyzing structured and unstructured safety data, methods that have not yet been widely applied in workplace injury research. As such, future research should apply recent advances in ML, integrating these approaches with comprehensive and accessible datasets to enhance the prediction and prevention of workplace injuries, provide more detailed analytics and insights, and improve safety protocols across all industries. This comprehensive survey is an invaluable resource for researchers and practitioners leveraging ML to address complex challenges in workplace safety.

Reliable software is mandatory for complex mission-critical systems. Classifying modules as fault-prone, or not, is a valuable technique for guiding development processes, so that resources can be focused on those parts of a system that are most likely to have faults. Logistic regression offers advantages over other classification modeling techniques, such as interpretable coefficients. There are few prior applications of logistic regression to software quality models in the literature, and none that we know of account for prior probabilities and costs of misclassification. A contribution of this paper is the application of prior probabilities and costs of misclassification to a logistic regression-based classification rule for a software quality model. This paper also contributes an integrated method for using logistic regression in software quality modeling, including examples of how to interpret coefficients, how to use prior probabilities, and how to use costs of misclassifications. A case study of a major subsystem of a military, real-time system illustrates the techniques.

There is not a general consensus as to which performance metrics provide more reliable and informative results compared to others. While there are studies which investigate and compare different metrics, they are typically focused on the performance of a classifier, and do not provide a clear understanding as to the specific relationships between metrics, nor their reliability in different settings (such as highly imbalanced datasets). This study examines the underlying relationships among 17 commonly used performance metrics and their suitability for datasets of varying sizes and class distribution levels, using factor analysis to uncover latent factors. We analyzed 23 publicly available datasets from diverse domains, ranging in size from 309 to over five million instances and distribution levels from 0.17% to 44.87%, using two gradient boosting algorithms, LightGBM and XGBoost, and one unsupervised anomaly detection algorithm, Isolation Forest. Factor analysis was used to group the metrics into distinct latent factors, enabling a framework for researchers to select appropriate metrics and avoid redundant or misleading ones based on dataset characteristics.

Healthcare insurance fraud is a major problem, with an estimated 300 billion lost annually in the United States. Machine learning has been explored as a tool for fraud detection for over a decade, but challenges remain, including class imbalance, dataset diversity, and model interpretability. This survey reviews 22 supervised, unsupervised, and semi-supervised classification techniques published between December 2017 and October 2024 which are novel within the healthcare fraud detection domain. Supervised techniques, which make up a majority of the identified works, are divided into deep learning, graph-based, and meta-learning methods. We also find a small set of novel unsupervised and semi-supervised classifiers, which we examine with a focus on model explainability. We find that deep learning is a popular and effective approach for supervised learning, while research gaps remain in areas such as transfer learning and incremental learning for all three classification approaches. Most concerningly, we find few to no works which establish benchmarks across diverse techniques. We assert that addressing these gaps could lead to more effective, transparent, and adaptable fraud detection systems in healthcare insurance.

Automobile repossession and impoundment is crucial to enforcing loan repayments and the law, but remains unpredictable and challenging. A wide range of factors can prevent a successful repossession attempt, such as miscommunication on vehicle location or lien status, or debtor hostility. Automated risk estimation models may provide a valuable resource in allowing repossession companies to evaluate different jobs and allocate resources effectively; however, we find no existing work investigating statistical or machine-learning-based models in this field. To this end, we investigate the application of machine learning classification techniques to a private auto repossession dataset containing information on 200,000 distinct repossession attempts, including location, license plate status, and address type. We train a logistic regression classifier and two Gradient Boosted Decision Tree (GBDT) algorithms (CatBoost and XGBoost) to predict whether any given repo attempt will be a success or failure. We find that all three models, when optimized, give moderate classification performance (almost 80% accuracy and >0.50 F1 score), indicating that they provide valuable information for repossession companies and may support more effective decision-making and resource allocation. This is supported by extracted feature importance values from our GBDT models, which may provide additional insights to practitioners when prioritizing assignments. However, the application domain of auto repossession remains largely unexplored, and we highly encourage future work to explore other machine learning approaches or data collection techniques within this field.

In the aviation industry, inventory management based on non-smooth demand forecasting is an ongoing challenge. Generally, aircraft parts lack easily observable demand patterns. Through twenty-six articles published from 2012 to 2023, this literature review discusses the performance of several forecasting techniques using time series data. These methodologies include statistical forecasting methods, such as Croston’s method and single exponential smoothing (SES), and machine learning (ML) models built using ensembles of decision trees or neural networks. Research into demand forecasting does not only focus on achieving and maintaining accurate predictions, but also on lowering costs associated with over- or under-forecasting. While classical methods remain popular due to their ease of use, explainability, and cost-effectiveness, ML models demonstrate potential for higher accuracy under optimal conditions. With a focus on the aviation domain, this survey provides an extensive overview of demand forecasting methods that can assist businesses with improving inventory control of aircraft stock.

Fraud detection through the classification of highly imbalanced Big Data is an exciting area of Machine Learning research. On the one hand, in certain fraud detection application domains, the use of One-Class classifiers is an overlooked opportunity. On the other hand, for researchers faced with the task of building Machine Learning models for identifying fraud, when only legitimate transaction data is available, One-Class Classifiers are indispensable. We investigate the efficacy of SHapley Additive exPlanations (SHAP) as a feature selection technique for One-Class classification tasks. In this study we utilize authentic data from the Credit Card fraud and Medicare insurance fraud application domains. Our contribution is to show that researchers can use SHAP in conjunction with One-Class Classifiers to do feature selection on highly imbalanced datasets, and then build models, with the selected features, that yield performance similar to, or better than, models built using all features. Our results in Big Medicare data fraud detection show that an over 90% data reduction through feature selection can nevertheless coincide with the best performance in terms of Area under the Precision Recall Curve.