LLM-based Privacy Data Augmentation Guided by Knowledge Distillation with a Distribution Tutor for Medical Text Classification
CoRR(2024)
摘要
As sufficient data are not always publically accessible for model training,
researchers exploit limited data with advanced learning algorithms or expand
the dataset via data augmentation (DA). Conducting DA in private domain
requires private protection approaches (i.e. anonymization and perturbation),
but those methods cannot provide protection guarantees. Differential privacy
(DP) learning methods theoretically bound the protection but are not skilled at
generating pseudo text samples with large models. In this paper, we transfer
DP-based pseudo sample generation task to DP-based generated samples
discrimination task, where we propose a DP-based DA method with a LLM and a
DP-based discriminator for text classification on private domains. We construct
a knowledge distillation model as the DP-based discriminator: teacher models,
accessing private data, teaches students how to select private samples with
calibrated noise to achieve DP. To constrain the distribution of DA's
generation, we propose a DP-based tutor that models the noised private
distribution and controls samples' generation with a low privacy cost. We
theoretically analyze our model's privacy protection and empirically verify our
model.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要