Generative AI-Based Virtual Assistant Using Retrieval-Augmented Generation: an Evaluation Study.

The integration of large language models (LLMs) with external sources has established Retrieval-Augmented Generation (RAG) as a key technology for improving reliability and reducing hallucinations. While RAG has proven effective, the paradigm has evolved to incorporate various techniques at different stages, aiming to further enhance its potential. However, a key challenge lies in identifying configurations of advanced techniques that improve performance. This study evaluates several technique configurations in a modular RAG system, with the objective of identifying those that deliver the best performance. Technologies such as LangChain, ChromaDB, and LLaMA 3.1:8B are employed. The evaluation focuses on metrics such as precision, recall, and F1-Score using a dataset from the academic postgraduate domain. The results highlight variability across configurations and emphasize how properly designing the processing of retrieved context can improve the quality of generated answers.