Chrome Extension
WeChat Mini Program
Use on ChatGLM

Reliable and Interpretable Drift Detection in Streams of Short Texts

Computing Research Repository (CoRR)(2023)

IBM Research

Cited 0|Views46
Abstract
Data drift is the change in model input data that is one of the key factors leading to machine learning models performance degradation over time. Monitoring drift helps detecting these issues and preventing their harmful consequences. Meaningful drift interpretation is a fundamental step towards effective re-training of the model. In this study we propose an end-to-end framework for reliable model-agnostic change-point detection and interpretation in large task-oriented dialog systems, proven effective in multiple customer deployments. We evaluate our approach and demonstrate its benefits with a novel variant of intent classification training dataset, simulating customer requests to a dialog system. We make the data publicly available.
More
Translated text
Key words
Change Detection,Concept Drift,Regret Analysis,Robust Learning,Data Streams
PDF
Bibtex
AI Read Science
Must-Reading Tree
Example
Generate MRT to find the research sequence of this paper
Related Papers
Data Disclaimer
The page data are from open Internet sources, cooperative publishers and automatic analysis results through AI technology. We do not make any commitments and guarantees for the validity, accuracy, correctness, reliability, completeness and timeliness of the page data. If you have any questions, please contact us by email: report@aminer.cn
Chat Paper

要点】:本文提出了一种端到端的框架,用于在大型任务导向对话系统中可靠地检测和解释模型无关的变化点,有效防止机器学习模型性能随时间退化的问题。

方法】:研究采用了一种模型无关的变化点检测方法,并加入了可解释性机制,使得检测到的变化点可以被有效理解和用于模型的重新训练。

实验】:作者通过使用一种新型的意图分类训练数据集,模拟客户对对话系统的请求,验证了方法的有效性,并将数据集公开。