Comparing Recursive Autoencoder and Convolutional Network for Phrase-Level Sentiment Polarity Classification

We present a comparative evaluation of two neural network architectures, which can be used to compute representations of phrases or sentences. The Semi-Supervised Recursive Autoencoder (SRAE) and the Convolutional Neural Network (CNN) are both methods that directly operate on sequences of words represented via word embeddings and jointly model the syntactic and semantic peculiarities of phrases. We compare both models with respect to their classification accuracy on the task of binary sentiment polarity classification. Our evaluation shows that a single-layer CNN produces equally accurate phrase representations and that both methods profit from the initialization with word embeddings trained by a language model. We observe that the initialization with domain specific word embeddings has no significant effect on the accuracy of both phrase models. A pruning experiment revealed that up to 95% of the parameters used to train the CNN could be removed afterwards without affecting the model's accuracy.