A Data-centric Solution to Improve Online Performance of Customer Service Bots

The online performance of customer service bots is often less than satisfactory because of the gap between limited training data and real-world user questions. As a straightforward way to improve online performance, model iteration and re-deployment are time-consuming and labor-intensive, and therefore difficult to sustain. To fix badcases and improve online performance of chatbots in a timely and continuous manner, we propose a data-centric solution consisting of three main modules: badcase detection, badcase correction, and answer extraction. By making full use of online model signals, implicit user feedback and artificial customer service log, the proposed solution can fix online badcases automatically. Our solution has been deployed and bringing consistently positive impacts for hundreds of customer service bots used by Alipay app.