Dropout: a simple way to prevent neural networks from overfittingEI
Deep neural nets with a large number of parameters are very powerful machine learning systems. However, overfitting is a serious problem in such networks. Large networks are also slow to use, making it difficult to deal with overfitting by combining the predictions of many different large neural nets at test time. Dropout is a technique for addressing this problem. The key idea is to randomly drop units (along with their connections) from the neural network during training. This prevents units from co-adapting too much. During trainin...更多
- 2Guoliang Kang, Jun Li, Dacheng Tao. Shakeout: A New Regularized Deep Neural Network Training Scheme.AAAI, pp. 1751-1757, 2016.
- 3Ranti Dev Sharma, Sudhanshu Mittal, Samarth Tripathi, Shrinivas Acharya. Using Modern Neural Networks to Predict the Decisions of Supreme Court of the United States with State-of-the-Art Accuracy.ICONIP, pp. 475-483, 2015.
- 4Hao Zheng, Mingming Chen, Wenju Liu, Zhanlei Yang, Shan Liang. Improving deep neural networks by using sparse dropout strategy.ChinaSIP, pp. 21-26, 2014.
- 5Haibing Wu, Xiaodong Gu. Towards Dropout Training for Convolutional Neural Networks.Neural Networks, 2015.
- 6Burhani, Hasham; Feng, Wenying; Hu, Gongzhu. Denoising AutoEncoder in Neural Networks with Modified Elliott Activation Function and Sparsity-Favoring Cost Function.ACIT-CSI '15 Proceedings of the 2015 3rd International Conference on Applied Computing and Information Technology/2nd International Conference on Computational Science and Intelligence, pp. 343-348, 2015.
- 7Bluche, Theodore; Kermorvant, Christopher; Louradour, Jerome. Where to apply dropout in recurrent neural networks for handwriting recognition?.ICDAR '15 Proceedings of the 2015 13th International Conference on Document Analysis and Recognition (ICDAR), pp. 681-685, 2015.
- 8Pham, V.; Bluche, T.; Kermorvant, C.; Louradour, J.. Dropout Improves Recurrent Neural Networks for Handwriting Recognition.Frontiers in Handwriting Recognition, pp. 285-290, 2014.
- 10Yongquan Zhang, Youmei Li, Jianyong Sun, Jiabing Ji. Estimates on compressed neural networks regression.Neural Networks, pp. 10-17, 2015.
- 11U. Bodenhausen, A. Waibel. Learning the architecture of neural networks for speech recognition.ICASSP '91 Proceedings of the Acoustics, Speech, and Signal Processing, 1991. ICASSP-91., 1991 International Conference, pp. 117-120, 1991.
- 12W.R. Foster, F. Collopy, L.H. Ungar. Neural network forecasting of short, noisy time series.Computers and Chemical Engineering, pp. 293-297, 1992.
- 13R.M Zur, Y Jiang, C.E Metz. Comparison of two methods of adding jitter to artificial neural network training.CARS, pp. 886-889, 2004.
- 15Pawalai Kraipeerapun, Somkid Amornsamankul. One-against-all multiclass classification based on multiple complementary neural networks.ACMIN'12 Proceedings of the 14th international conference on Automatic Control, Modelling & Simulation, and Proceedings of the 11th international conference on Microelectronics, Nanoelectronics, Optoelectronics, pp. 68-72, 2012.
- 16Yinyin Liu, Janusz A. Starzyk, Zhen Zhu. Optimizing number of hidden neurons in neural networks.Artificial Intelligence and Applications, pp. 121-126, 2007.
- 17Mark Ring, Sequence Learning with Incremental Higher-Order Neural Networks.Sequence Learning with Incremental Higher-Order Neural Networks, 1993.
Journal of Machine Learning Research, pp. 1929-1958, 2014.