Adversarial Training is a Form of Data-dependent Operator Norm Regularization

NIPS 2020, 2020.

Cited by: 3|Views17
EI
Weibo:
We present a data-dependent variant of spectral norm regularization that directly regularizes large singular values of a neural network in regions that are supported by the data, as opposed to existing methods that regularize a global, data-independent upper bound

Abstract:

We establish a theoretical link between adversarial training and operator norm regularization for deep neural networks. Specifically, we prove that $\ell_p$-norm constrained projected gradient ascent based adversarial training with an $\ell_q$-norm loss on the logits of clean and perturbed inputs is equivalent to data-dependent (p, q) ope...More

Code:

Data:

0
Your rating :
0

 

Tags
Comments