Benchmarking and Improving Compositional Generalization of Multi-aspect Controllable Text Generation
arxiv(2024)
摘要
Compositional generalization, representing the model's ability to generate
text with new attribute combinations obtained by recombining single attributes
from the training data, is a crucial property for multi-aspect controllable
text generation (MCTG) methods. Nonetheless, a comprehensive compositional
generalization evaluation benchmark of MCTG is still lacking. We propose
CompMCTG, a benchmark encompassing diverse multi-aspect labeled datasets and a
crafted three-dimensional evaluation protocol, to holistically evaluate the
compositional generalization of MCTG approaches. We observe that existing MCTG
works generally confront a noticeable performance drop in compositional
testing. To mitigate this issue, we introduce Meta-MCTG, a training framework
incorporating meta-learning, where we enable models to learn how to generalize
by simulating compositional generalization scenarios in the training phase. We
demonstrate the effectiveness of Meta-MCTG through achieving obvious
improvement (by at most 3.64
cases.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要