Querytext – using queries and clicks to improve text matching for web search

User queries and their associated clicks have been extensively explored to improve Web search relevance. Very little existing work explores how user clicks can be used to improve text matching for Web search. In this paper, we treat user queries that result in clicks as off-page annotations. These queries, like anchor text, provide a valuable additional source of relevance information for Web pages. We call the queries that are used to annotate Web pages in this way the QueryText. We propose using the QueryText as a new weighted textual field for Web pages, where the weights are based on user click behavior. We derive two sets of text matching features from the new field – BM25F-based features and n-gram features. We implement the features within a commercial search engine and evaluate the effectiveness of our approach on real large-scale Web data. Our evaluation results show significant improvements in retrieval effectiveness using text match features derived from the QueryText.