Improving Short Query Representation in LDA Based Information Retrieval Systems.

Incorporation of topic modeling techniques into Information Retrieval (IR) systems has been a promising area of research in the last years. Typically, queries submitted into IR systems are concise and made up using only the essential keywords. This leads to the formulation of short length queries, which have a negative impact on the LDA algorithm accuracy and relevant documents retrieval. This work presents a novel method to improve short query representation in information retrieval systems. The new technique (LDAW), modifies its representation based on the Latent Dirichlet Allocation (LDA) model. LDAW is tested with three biomedical corpora (TREC Genomics 2004, TREC Genomics 2005, and OHSUMED) and one legal cases corpus (FIRE 2017). Results prove that the application of the proposed method clearly outperforms the baseline methods (BM25 and non-modified LDA).