Online Knowledge Distillation with Diverse Peers
national conference on artificial intelligence, 2020.
Distillation is an effective knowledge-transfer technique that uses predicted distributions of a powerful teacher model as soft targets to train a less-parameterized student model. A pre-trained high capacity teacher, however, is not always available. Recently proposed online variants use the aggregated intermediate predictions of multi...More
PPT (Upload PPT)