Training recurrent neural networksEI
Recurrent Neural Networks (RNNs) are powerful sequence models that were believed to be difficult to train, and as a result they were rarely used in machine learning applications. This thesis presents methods that overcome the difficulty of training RNNs, and applications of RNNs to challenging problems. We first describe a new probabilistic sequence model that combines Restricted Boltzmann Machines and RNNs. The new model is more powerful than similar models while being less difficult to train.Next, we present a new variant of the Hes...更多
- 1Graves, A.; Mohamed, A.-R.; Hinton, G.. Speech recognition with deep recurrent neural networks.Acoustics, Speech and Signal Processing, pp. 6645-6649, 2013.
- 2Sirisongkol, R.; Xiaodong Liu. Stability analysis of recurrent neural networks with time-varying delay and disturbances via quadratic convex technique.Intelligent Control and Information Processing, pp. 130-137, 2014.
- 4Nikita Barabanov, Danil V. Prokhorov. Stability analysis of discrete-time recurrent neural networks.IEEE Transactions on Neural Networks, pp. 292-303, 2002.
- 5Li-Chiu Chang, Pin-An Chen, Fi-John Chang. Reinforced two-step-ahead weight adjustment technique for online training of recurrent neural networks.IEEE Trans. Neural Netw. Learning Syst., pp. 1269-1278, 2012.
- 6Yi Shen, Jun Wang. Robustness analysis of global exponential stability of recurrent neural networks in the presence of time delays and random disturbances.IEEE Trans. Neural Netw. Learning Syst., pp. 87-96, 2012.
- 9Paul Rodriguez, Jeff Elman. WATCHING THE TRANSIENTS : VIEWING A SIMPLE RECURRENT NETWORK AS A LIMITED COUNTER.Behaviormetrika, pp. 51-74, 1999.
- 10P. Tino, M. Cernansky, L. Benuskova. Markovian architectural bias of recurrent neural networks.IEEE Transactions on Neural Networks, pp. 6-15, 2004.
- 12David Sussillo, Omri Barak. Opening the black box: low-dimensional dynamics in high-dimensional recurrent neural networks.Neural Computation, pp. 626-649, 2013.
- 13M. Kimura, R. Nakano. Learning dynamical systems by recurrent neural networks from orbits.Neural Networks, pp. 1589-1599, 1998.
- 14Sebastian Bitzer, Stefan J. Kiebel. Recognizing recurrent neural networks (rRNN): Bayesian inference for recurrent neural networks.Biological Cybernetics, pp. 201-217, 2012.
- 15Zhao Xu, Qing Song, Danwei Wang. A robust recurrent simultaneous perturbation stochastic approximation training algorithm for recurrent neural networks.Neural Computing and Applications, pp. 1851-1866, 2014.
- 16Zhang Yi, Foundations of implementing the competitive layer model by Lotka-Volterra recurrent neural networks.IEEE Transactions on Neural Networks, pp. 494-507, 2010.
- 17T. Wessels, Christian W. Omlin. Refining Hidden Markov Models with Recurrent Neural Networks.IJCNN (2), pp. 2271-2271, 2000.
- 19Vincenzo Di Massa, Gabriele Monfardini, Lorenzo Sarti, Franco Scarselli, Marco Maggini, Marco Gori. A Comparison between Recursive Neural Networks and Graph Neural Networks.IJCNN, pp. 778-785, 2006.
- 20Rui Xu; Ganesh K. Venayagamoorthy; Donald C. Wunsch, II. Modeling of gene regulatory networks with hybrid differential evolution and particle swarm optimization.Neural Networks, pp. 917-927, 2007.
Training recurrent neural networks, 2013.