A New Pruning Method to Train Deep Neural Networks

Deep neural networks are very powerful models for machine learning tasks. However, suffering from overfitting and gradient vanishing problems, they are difficult to train. We proposed a method of gradually pruning the weakly connected weights to train deep neural networks and an effective strategy to identify the weak connections. Our method can improve the conventional stochastic gradient descent and can get even better performance than the widely used dropout method for deeper models.