Optimizing Energy in Supervised Learning with Data Summarization: A Comparative Study.

The significant computational demands of modern machine learning (ML) models raise growing concerns about their environmental impact and economic cost during training. To address this, data summarization techniques, which involve selecting a smaller, representative subset of the training data, offer a promising avenue for optimizing energy consumption without compromising model performance. This article investigates the influence of various data summarization algorithms—specifically random sampling, Facility Location (FL), and CRAIG—on three critical aspects of supervised learning (SL) training: model accuracy, energy consumption, and overall efficiency, which we quantify as the ratio of accuracy to energy consumption. Our extensive experimental findings reveal that while sophisticated methods like FL and CRAIG aim to select highly representative subsets, random sampling consistently achieved a robust balance of accuracy and efficiency. This is particularly evident when accounting for the often significant pre-processing energy overheads incurred by more complex selection strategies. Furthermore, we emphasize the pivotal role of early stopping criteria in optimizing the overall energy efficiency of the training process. Our analysis demonstrates that strategic adjustments to these criteria can substantially reduce the number of training epochs required for convergence, thereby mitigating considerable energy waste.