Projection Based Weight Normalization: Efficient Method for Optimization on Oblique Manifold in DNNs
Pattern Recognition(2020)
摘要
•We propose to constrain the incoming weights of each neuron to be unit norm to address ill conditioned problem in DNNs.•Constrain ing weight s can be formulated as an optimization problem over the Oblique manifold.•We propose a simple yet efficient method referred to as projection based weight normalization (PBWN) to solve the optimization problem.•PBWN has the property of regularization and collaborates well with the commonly used batch normalization technique.•Extensive experiments on several widely used image datasets show the consistent performance improvement over the baseline DNNs.
更多查看译文
关键词
Deep learning,Weight normalization,Oblique manifold,Image classification
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络