FedPFT: Federated Proxy Fine-Tuning of Foundation Models
arxiv(2024)
摘要
Adapting Foundation Models (FMs) for downstream tasks through Federated
Learning (FL) emerges a promising strategy for protecting data privacy and
valuable FMs. Existing methods fine-tune FM by allocating sub-FM to clients in
FL, however, leading to suboptimal performance due to insufficient tuning and
inevitable error accumulations of gradients. In this paper, we propose
Federated Proxy Fine-Tuning (FedPFT), a novel method enhancing FMs adaptation
in downstream tasks through FL by two key modules. First, the sub-FM
construction module employs a layer-wise compression approach, facilitating
comprehensive FM fine-tuning across all layers by emphasizing those crucial
neurons. Second, the sub-FM alignment module conducts a two-step
distillations-layer-level and neuron-level-before and during FL fine-tuning
respectively, to reduce error of gradient by accurately aligning sub-FM with FM
under theoretical guarantees. Experimental results on seven commonly used
datasets (i.e., four text and three vision) demonstrate the superiority of
FedPFT.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要