Topic by Topic Performance of Information Retrieval Systems

Formulation of topic properties is the goal of this paper. These properties are to be used in judging the difficulty of topics appropriate to the Text REtrieval Conference (TREC) ad hoc task. Applying statistical methods to both TREC-6 and TREC-7 information retrieval results, we identify topic pa irs that exemplify topic properties useful in relating topic statements to system performance. From two topics that seem the same with respect to the challenge they provide information r etrieval systems, we formulate topic properties by relating the corresponding topic statements to what is known abo ut information retrieval systems. Some properties app arent in the topic pairs identified are linked to topic expansio n. These pairs exemplify both the need for expansion and the dange r in automatic expansion.