Going Beyond Relevance: Incorporating Effort In Information Retrieval

Primary focus of Information retrieval (IR) systems has been to optimize for Relevance. Existing approaches used to rank documents or evaluate IR systems do not account for "user effort". At present, relevance captures topical overlap between document and user query. This mechanism does not take into consideration either time or effort of end user to satisfy information need. While a judge may spend time assessing a document, an end user may not thoroughly examine a document. We identified factors that are associated with effort for a single document and gathered judgments for same. We also investigated the role of several features in predicting effort on webpage. In future, we shall investigate role of effort on mobile and investigate effort based evaluation methodology that also takes into account user's search task.