On-the-Fly Attention Modularization for Neural Generation

Ximing Lu
Ximing Lu
Jena D. Hwang
Jena D. Hwang
Antoine Bosselut
Antoine Bosselut
Cited by: 0|Bibtex|Views6
Other Links: arxiv.org

Abstract:

Despite considerable advancements with deep neural language models (LMs), neural text generation still suffers from degeneration: generated text is repetitive, generic, self-inconsistent, and lacking commonsense. The empirical analyses on sentence-level attention patterns reveal that neural text degeneration may be associated with insuf...More

Code:

Data:

Full Text
Your rating :
0

 

Tags
Comments