Challenges in the Development of Effective Systems for Professional Legal Search.

The key objective of an information retrieval (IR) system is to identify and return to the user content relevant or useful in addressing the information need which required them to use the system. The development and evaluation of IR systems relies on the availability of suitable datasets or test collections. These typically consist of a target document collection, example search queries representative of those that users of the system to be developed, are expected to enter, and relevance data indicating which documents in the collection are relevant to the information needed as expressed in each query. Public research in IR has focused on popular content, e.g. news corpora or web content, for which average users can pose queries expressing information needs and judge the relevance of retrieved documents. This is not the case for professional search applications, for example legal, medical, financial search where domain experts are required for these tasks. We describe our experiences from the development of a professional legal IR application employing semantic search technologies. Our activities indicatethe vital need for close interaction between the professionals for which the application is being developed and the IR researchers throughout the development life cycle of the search system. Such engagement is vital in order for the IR researchers to understand the working practices of the professional searchers, the specifications of their information needs and the domain in which they are searching, and to study how they engage and interact with information. A key reason to seek to understand these topics so deeply is to facilitate meaningful evaluation of the effectiveness of IR technologies as components in the system being developed.