# Training recurrent neural networksEI

摘要

Recurrent Neural Networks (RNNs) are powerful sequence models that were believed to be difficult to train, and as a result they were rarely used in machine learning applications. This thesis presents methods that overcome the difficulty of training RNNs, and applications of RNNs to challenging problems. We first describe a new probabilistic sequence model that combines Restricted Boltzmann Machines and RNNs. The new model is more powerful than similar models while being less difficult to train.Next, we present a new variant of the Hes...更多

- 1Graves, A.; Mohamed, A.-R.; Hinton, G.. Speech recognition with deep recurrent neural networks.
*Acoustics, Speech and Signal Processing*, pp. 6645-6649, 2013. - 2Sirisongkol, R.; Xiaodong Liu. Stability analysis of recurrent neural networks with time-varying delay and disturbances via quadratic convex technique.
*Intelligent Control and Information Processing*, pp. 130-137, 2014. - 4Nikita Barabanov, Danil V. Prokhorov. Stability analysis of discrete-time recurrent neural networks.
*IEEE Transactions on Neural Networks*, pp. 292-303, 2002. - 5Li-Chiu Chang, Pin-An Chen, Fi-John Chang. Reinforced two-step-ahead weight adjustment technique for online training of recurrent neural networks.
*IEEE Trans. Neural Netw. Learning Syst.*, pp. 1269-1278, 2012. - 6Yi Shen, Jun Wang. Robustness analysis of global exponential stability of recurrent neural networks in the presence of time delays and random disturbances.
*IEEE Trans. Neural Netw. Learning Syst.*, pp. 87-96, 2012. - 7Jürgen Schmidhuber, Daan Wierstra, Matteo Gagliolo, Faustino Gomez. Training recurrent networks by Evolino.
*Neural Computation*, pp. 757-779, 2007. - 8Ilya Sutskever, Geoffrey Hinton. Temporal-kernel recurrent neural networks.
*Neural Networks*, pp. 239-243, 2010. - 9Paul Rodriguez, Jeff Elman. WATCHING THE TRANSIENTS : VIEWING A SIMPLE RECURRENT NETWORK AS A LIMITED COUNTER.
*Behaviormetrika*, pp. 51-74, 1999. - 10P. Tino, M. Cernansky, L. Benuskova. Markovian architectural bias of recurrent neural networks.
*IEEE Transactions on Neural Networks*, pp. 6-15, 2004. - 11Alex Graves, Santiago Fernández, Jürgen Schmidhuber. Multi-dimensional recurrent neural networks.
*Clinical Orthopaedics and Related Research*, pp. 549-558, 2007. - 12David Sussillo, Omri Barak. Opening the black box: low-dimensional dynamics in high-dimensional recurrent neural networks.
*Neural Computation*, pp. 626-649, 2013. - 13M. Kimura, R. Nakano. Learning dynamical systems by recurrent neural networks from orbits.
*Neural Networks*, pp. 1589-1599, 1998. - 14Sebastian Bitzer, Stefan J. Kiebel. Recognizing recurrent neural networks (rRNN): Bayesian inference for recurrent neural networks.
*Biological Cybernetics*, pp. 201-217, 2012. - 15Zhao Xu, Qing Song, Danwei Wang. A robust recurrent simultaneous perturbation stochastic approximation training algorithm for recurrent neural networks.
*Neural Computing and Applications*, pp. 1851-1866, 2014. - 16Zhang Yi, Foundations of implementing the competitive layer model by Lotka-Volterra recurrent neural networks.
*IEEE Transactions on Neural Networks*, pp. 494-507, 2010. - 17T. Wessels, Christian W. Omlin. Refining Hidden Markov Models with Recurrent Neural Networks.
*IJCNN (2)*, pp. 2271-2271, 2000. - 19Vincenzo Di Massa, Gabriele Monfardini, Lorenzo Sarti, Franco Scarselli, Marco Maggini, Marco Gori. A Comparison between Recursive Neural Networks and Graph Neural Networks.
*IJCNN*, pp. 778-785, 2006. - 20Rui Xu; Ganesh K. Venayagamoorthy; Donald C. Wunsch, II. Modeling of gene regulatory networks with hybrid differential evolution and particle swarm optimization.
*Neural Networks*, pp. 917-927, 2007.

个人信息

Training recurrent neural networks, 2013.

被引用次数：

**252**|引用|19标签

作者

评论