Data Envelopment Analysis Clustering: an Optimization Based Machine Learning

Clustering algorithms are commonly used to group units based on their intrinsic characteristics, represented by various features, typically divided into cost features and benefit features. Most existing clustering methods analyze these features based on distance functions, and the cost-benefit relationship is not a focus. In this study, we approach clustering from the perspective of constructing a composite index reflecting the cost-benefit ratio. We utilize data envelopment analysis (DEA) to develop a "k-DEA-weights" clustering method to identify clusters and reveal the relative degree of importance of different features within each cluster to enhance the explainability of clustering outcomes. The proposed method can be viewed as an optimization-based explainable machine learning technique. We first employ a DEA cross-efficiency model to calculate cross-evaluated weights for units and a group DEA cross-efficiency model to obtain common weights for clusters. Through an iterative heuristic algorithm, we optimize the clustering process by minimizing the distance between the cross-evaluated weights and the common weights within a cluster. To validate its effectiveness, we conduct numerical experiments that demonstrate strong clustering performance. We then apply it to Chunyu Doctor platform, a Chinese online healthcare consultation platform. The results provide actionable insights for optimizing services and improving user experience.