Mix and Match: Collaborative Expert-Crowd Judging for Building Test Collections Accurately and Affordably

DESIRES, pp. 42-46, 2018.

Cited by: 0|Bibtex|Views19|Links
EI

Abstract:

Crowdsourcing offers an affordable and scalable means to collect relevance judgments for IR test collections. However, crowd assessors may show higher variance in judgment quality than trusted assessors. In this paper, we investigate how to effectively utilize both groups of assessors in partnership. We specifically investigate how agreem...More

Code:

Data:

Your rating :
0

 

Tags
Comments