Improving Krum’s Byzantine Resilience in Federated Learning Via Layerwise Aggregation

With the rising usage of artificial intelligence systems, social concerns around them and the need for regulation is also increasing, including requirements for data privacy. Federated Learning addresses data privacy concerns in distributed machine learning by training models collaboratively without centralizing data. However, Federated Learning is susceptible to Byzantine attacks, where malicious nodes submit corrupted updates. Krum, a robust aggregation algorithm, has been widely adopted as a defense mechanism. However, recent studies have shown that Krum’s performance degrades significantly in high-dimensional settings. This work proposes Layerwise Krum, a novel aggregation method that enhances Krum’s robustness in high-dimensional spaces while maintaining computational efficiency. We provide theoretical analysis of the improved robustness of Layerwise Krum compared to standard Krum. Furthermore, we empirically evaluate Layerwise Krum on various image classification datasets under diverse data distributions and Byzantine attack scenarios, consistently demonstrating superior performance compared to the original Krum operator.