TargetDrop: A Targeted Regularization Method for Convolutional Neural Networks

Dropout regularization has been widely used in deep learning but performs less effective for convolutional neural networks since the spatially correlated features allow dropped information to still flow through the networks. Some structured forms of dropout have been proposed to address this but are prone to result in over or under regularization as features are dropped randomly. In this paper, we propose a targeted regularization method, TargetDrop, which incorporates the attention mechanism to drop several discriminative feature units. Specifically, it masks out the target regions in the feature maps corresponding to the target channels. We conduct comprehensive experiments and demonstrate that TargetDrop outperforms the other dropout-based regularization methods.