Testing the Cluster Hypothesis with Focused and Graded Relevance Judgments.

The cluster hypothesis is a fundamental concept in ad hoc retrieval. Heretofore, cluster hypothesis tests were applied to documents using binary relevance judgments. We present novel tests that utilize graded and focused relevance judgments; the latter are markups of relevant text in relevant documents. Empirical exploration reveals that the cluster hypothesis holds not only for documents, but also for passages, as measured by the proposed tests. Furthermore, the hypothesis holds to a higher extent for highly relevant documents and for those that contain a high fraction of relevant text.