Denoising AutoEncoder in Neural Networks with Modified Elliott Activation Function and Sparsity-Favoring Cost Function
ACIT-CSI '15 Proceedings of the 2015 3rd International Conference on Applied Computing and Information Technology/2nd International Conference on Computational Science and Intelligence(2015)
摘要
Neural networks (NN) are architectures and algorithms for machine learning. They are quite powerful for tasks like classification, clustering, and pattern recognition. Large neural networks can be considered a universal function that can approximate any function, and hence are effective at learning from the training data, not only the useful information but also the noise in the training data. However, as the number of neurons and the number of hidden layers grow, the number of connections in the network increases exponentially, and the over fitting problem becomes more severe biased towards noise. Various methods have been proposed to address this problem such as AutoEncoder, Dropout, DropConnect, and Factored Mean training. In this paper, we propose a denoising autoencoder approach using a modified Elliott activation function and a cost function that favors sparsity in the input data. Preliminary experiments using the modified algorithm on several real data sets showed that the proposed approach performed well.
更多查看译文
关键词
Deep neural networks,activation function,cost function,denoising autoencoder
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络