Most existing dialog models are trained on 001 static dialog datasets or in an interactive way 002 with user simulators, and evaluated in the same 003 way. Such methods mostly make an ideal hy-004 pothesis that the user behaves consistently to 005 the goal. Nevertheless, inconsistent behaviors 006 are often observed from real users due to un-007 predictable mind changes or language under-008 standing errors. In this paper, we give a sys-009 tematic investigation of the inconsistent prob-010 lem in real-world dialog systems and introduce 011 three kinds of inconsistencies, namely Goal 012 Change , Action Disloyalty and Understanding 013 Deviation . We propose a user model to simu-014 late those three kinds of inconsistencies, which 015 can be used to examine the model robustness. 016 The simulation model is further utilized to sup-017 port Reinforcement Learning and inconsistent 018 data augmentation, which boosts the perfor-019 mance of pipeline and end-to-end dialog mod-020 els under inconsistent situation. 021
分享
引用
