MadFed: Enhancing Federated Learning With Marginal-Data Model Fusion

As the demand for intelligent applications at the network edge grows, so does the need for effective federated learning (FL) techniques. However, FL often relies on non-identically and non-independently distributed local datasets across end devices, which could result in considerable performance degradation. Prior solutions, such as model-driven approaches based on knowledge distillation, meta-learning, and transfer learning, have provided some reprieve. However, their performance suffers under heterogeneous local datasets and highly skewed data distributions. To address these challenges, this study introduces the MArginal Data fusion FEDerated Learning (MadFed) approach, a groundbreaking fusion of model- and data-driven methodologies. By utilizing marginal data, MadFed mitigates data distribution skewness, improves the maximum achievable accuracy, and reduces communication costs. Furthermore, the study demonstrates that the fusion of marginal data can significantly improve performance even with minimal data entries, such as a single entry. For instance, it provides up to a 15.4% accuracy increase and 70.4% communication cost savings when combined with established model-driven methodologies. Conversely, relying solely on these model-driven methodologies can result in poor performance, especially with highly skewed datasets. Significantly, MadFed extends its effectiveness across various FL algorithms and offers a unique method to augment label sets of end devices, thereby enhancing the utility and applicability of federated learning in real-world scenarios. The proposed approach is not only efficient but also adaptable and versatile, promising broader application and potential for widespread adoption in the field.