Transformer-NADE for Piano Performances

Self-attention based Transformers are compelling sequence models because they can capture relatively long-term dependencies by having direct access to the past. However, their memory requirements grow quadratically with sequence length, making it prohibitive to model long sequences with global attention. In contrast to previous representations of music that “flatten” a note’s performance attributes such as velocity, note on, and note off into a single sequence, we reduce sequence length by proposing a new representation, NoteTuple, which groups a note’s attributes as one event. This makes it natural to factorize a musical performance into a sequence of notes and model a note as a NADE on its attributes. The resulting models require fewer parameters and have faster generation. NoteTuples is a promising extension to Music Transformer (Huang et al., 2018b), enabling rich downstream tasks such as infilling of piano performances, eg, Ippolito et al.(2018). Samples can be heard at https://goo. gl/magenta/notetuple-examples.