Bias Mitigation in Fine-tuning Pre-trained Models for Enhanced Fairness and Efficiency
arxiv(2024)
摘要
Fine-tuning pre-trained models is a widely employed technique in numerous
real-world applications. However, fine-tuning these models on new tasks can
lead to unfair outcomes. This is due to the absence of generalization
guarantees for fairness properties, regardless of whether the original
pre-trained model was developed with fairness considerations. To tackle this
issue, we introduce an efficient and robust fine-tuning framework specifically
designed to mitigate biases in new tasks. Our empirical analysis shows that the
parameters in the pre-trained model that affect predictions for different
demographic groups are different, so based on this observation, we employ a
transfer learning strategy that neutralizes the importance of these influential
weights, determined using Fisher information across demographic groups.
Additionally, we integrate this weight importance neutralization strategy with
a matrix factorization technique, which provides a low-rank approximation of
the weight matrix using fewer parameters, reducing the computational demands.
Experiments on multiple pre-trained models and new tasks demonstrate the
effectiveness of our method.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要