The intriguing role of module criticality in the generalization of deep networks

Niladri Chatterji

Behnam Neyshabur

[0]

Hanie Sedghi

[0]

ICLR, 2020.

Cited by: 1|Views30

Keywords:

Module Criticality Phenomenon Complexity Measure Deep Learning

Weibo:

We studied the module criticality phenomenon and proposed a complexity measure based on module criticality that is able to correctly predict the superior performance of some deep neural networks

Abstract:

We study the phenomenon that some modules of deep neural networks (DNNs) are more critical than others. Meaning that rewinding their parameter values back to initialization, while keeping other modules fixed at the trained parameters, results in a large drop in the network's performance. Our analysis reveals interesting properties of the ...More

Code:

Data:

Full Text

View via Publisher