Is Label Smoothing Truly Incompatible with Knowledge Distillation: An Empirical Study

international conference on learning representations, 2020.

Cited by: 0|Views26
Weibo:
This work empirically clarifies a recently discovered perspective that label smoothing is incompatible with knowledge distillation.

Abstract:

This work aims to empirically clarify a recently discovered perspective that label smoothing is incompatible with knowledge distillation. We begin by introducing the behind motivation on how this incompatibility is raised, i.e., label smoothing erases relative information between teacher logits. We provide a novel connection on how label ...More

Code:

Data:

0
Full Text
Bibtex
Weibo
Your rating :
0

 

Tags
Comments