CrossIn: An Efficient Instruction Tuning Approach for Cross-Lingual Knowledge Alignment
arxiv(2024)
摘要
Multilingual proficiency presents a significant challenge for large language
models (LLMs). English-centric models are usually suboptimal in other
languages, particularly those that are linguistically distant from English.
This performance discrepancy mainly stems from the imbalanced distribution of
training data across languages during pre-training and instruction tuning
stages. To address this problem, we propose a novel approach called CrossIn,
which utilizes a mixed composition of cross-lingual instruction tuning data.
Our method leverages the compressed representation shared by various languages
to efficiently enhance the model's task-solving capabilities and multilingual
proficiency within a single process. In addition, we introduce a multi-task and
multi-faceted benchmark to evaluate the effectiveness of CrossIn. Experimental
results demonstrate that our method substantially improves performance across
tasks and languages, and we provide extensive insights into the impact of
cross-lingual data volume and the integration of translation data on enhancing
multilingual consistency and accuracy.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要