Multi-Layer Open-Domain Bangla Conversational Chatbot with a Hybrid approach

Bangla is one of the most spoken language globally, with approximately 228 million native speakers. But there isn't much research done in the Bangla chatbot field. There are a few closed domain retrievals based on Bangla chatbots like “Golpo”, “Doly”, and“ Anirudha”. But the author is not aware of any other work where an open-domain Bengali chatbot is implemented. This scientific research paper proposes “Kothok”, an open-domain three-layer Bangla chatbot with a hybrid mechanism consisting of both generative and retrieval approaches that can converse with users in Bangla. The first layer with a retrieval mechanism provides a quick controlled response, and if the question is informative, the second layer comes to play and retrieves the response from the web and acts as the knowledge base of the bot. The last layer with a deep learning transformer-based model uses a generative approach and acts as the brain of the system and responds to empathic and any type of conversation outside the first two layer's capabilities. We deployed our model in the browser, where users can interact directly with the bot. We tested it in both open domains and closed domain situations using independent Human raters to evaluate the quality of conversation and analysis the effect of three layers in open domain situations, revealing this approach's strength, limitations, and potential for future work. We also used an evaluation metric called SSA, which evaluation method was proposed in a research paper for the ‘Meena’ bot from google research in 2018 to evaluate the human likeliness. Our bot scored an SSA of 70.8% in the open domain scenario and an extraordinary score of 87.5% in the domain-specific application on a multi-turn evaluation.