On Sparsifying Encoder Outputs in Sequence-to-Sequence Models

Zhang Biao
Zhang Biao
Cited by: 2|Views49

Abstract:

Sequence-to-sequence models usually transfer all encoder outputs to the decoder for generation. In this work, by contrast, we hypothesize that these encoder outputs can be compressed to shorten the sequence delivered for decoding. We take Transformer as the testbed and introduce a layer of stochastic gates in-between the encoder and the...More

Code:

Data:

Full Text
Bibtex
Your rating :
0

 

Tags
Comments