Using Mechanical Turk to Build Machine Translation Evaluation Sets.
Mturk@HLT-NAACL(2014)
摘要
Building machine translation (MT) test sets is a relatively expensive task. As MT becomes increasingly desired for more and more language pairs and more and more domains, it becomes necessary to build test sets for each case. In this paper, we investigate using Amazon's Mechanical Turk (MTurk) to make MT test sets cheaply. We find that MTurk can be used to make test sets much cheaper than professionally-produced test sets. More importantly, in experiments with multiple MT systems, we find that the MTurk-produced test sets yield essentially the same conclusions regarding system performance as the professionally-produced test sets yield.
更多查看译文
关键词
machine translation evaluation set,building machine translation,mechanical turk,test set,professionally-produced test set,mturk-produced test set,professionally-produced test,language pair,multiple mt system,mt test,expensive task,crowdsourcing,artificial intelligence,computational linguistics,human language technology,natural language processing,machine translation,computer science
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络