Out-of-Distribution Detection by Leveraging Between-Layer Transformation Smoothness
arxiv(2023)
摘要
Effective out-of-distribution (OOD) detection is crucial for reliable machine
learning models, yet most current methods are limited in practical use due to
requirements like access to training data or intervention in training. We
present a novel method for detecting OOD data in Transformers based on
transformation smoothness between intermediate layers of a network (BLOOD),
which is applicable to pre-trained models without access to training data.
BLOOD utilizes the tendency of between-layer representation transformations of
in-distribution (ID) data to be smoother than the corresponding transformations
of OOD data, a property that we also demonstrate empirically. We evaluate BLOOD
on several text classification tasks with Transformer networks and demonstrate
that it outperforms methods with comparable resource requirements. Our analysis
also suggests that when learning simpler tasks, OOD data transformations
maintain their original sharpness, whereas sharpness increases with more
complex tasks.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要