WTA-MT: Weakness-Targeted Augmentation with Large Language Models for Machine Translation

Large Language Models excel in machine translation but face challenges with high computational costs and latency, making knowledge transfer to smaller models crucial. Traditional distillation methods, however, overlook student models’ existing capabilities, leading to redundant training on mastered knowledge and insufficient coverage of novel content. This paper presents WTA-MT, a Weakness-Targeted Augmentation framework for targeted knowledge transfer from LLMs to smaller models. By focusing on the student’s specific weaknesses instead of indiscriminate distillation, and leveraging LLM teachers to generate targeted supplementary data, WTA-MT efficiently enhances performance. Experiments in both general (En → Zh, De → En) and medical (En → De) domains demonstrate that WTA-MT outperforms traditional Sequence Knowledge Distillation (SeqKD) methods, achieving comparable or superior performance with significantly less data. Notably, our method eliminates performance deficiencies in low-scoring samples and while preserving high-quality translations, effectively narrowing the knowledge gap between small and large models.