Memory-Efficient Backpropagation Through Time

ADVANCES IN NEURAL INFORMATION PROCESSING SYSTEMS 29 (NIPS 2016), pp. 4125-4133, 2016.

Cited by: 111|Bibtex|Views128
EI
Other Links: dblp.uni-trier.de|arxiv.org

Abstract:

We propose a novel approach to reduce memory consumption of the backpropagation through time (BPTT) algorithm when training recurrent neural networks (RNNs). Our approach uses dynamic programming to balance a trade-off between caching of intermediate results and recomputation. The algorithm is capable of tightly fitting within almost any ...More

Code:

Data:

Full Text
Your rating :
0

 

Tags
Comments