Online Knowledge Distillation with Diverse Peers

national conference on artificial intelligence, 2020.

Cited by: 22|Bibtex|Views64
Other Links: academic.microsoft.com|arxiv.org

Abstract:

Distillation is an effective knowledge-transfer technique that uses predicted distributions of a powerful teacher model as soft targets to train a less-parameterized student model. A pre-trained high capacity teacher, however, is not always available. Recently proposed online variants use the aggregated intermediate predictions of multi...More

Code:

Data:

Full Text
Your rating :
0

 

Tags
Comments