Pattern (https://www.clips.uantwerpen.be/pages/pattern-nl) is an open-source Python package for NLP that is developed and maintained by the CLiPS Computational Linguistics group at Universiteit Antwerpen. The submodule for Dutch, pattern.nl, contains a rule-based sentiment analyzer, which is based on a built-in lexicon of about 4,000 Dutch lemmas. The lexicon contains a subjectivity and polarity score for each word, which are used to calculate a score for an input sentence. The usefulness of the lexicon was evaluated in 2012 by using it to classify book reviews. However, the applicability of Pattern in more general-domain sentiment analysis tasks is limited. For example, the sentences During the war, my youngest daughter died. or just broke up with my significant other and I don't want to live anymore. will receive a neutral judgement from the sentiment analysis function of pattern.nl. In order to generalise pattern.nl's sentiment analysis functionality, we propose to supplement its emotion lexicon with additional Dutch words and an associated subjectivity/polarity score. In this talk, we describe our attempt to extend pattern.nl with words from Moors lexicon (http://crr.ugent.be/papers/Moors_et_al_BRM_norms_Valence_Arousal_Dominance_AoA.pdf). Moors lexicon contains manually-annotated scores of valence, arousal and dominance for about 4,300 Dutch words. The ratings of valence were first rescaled to the [-1;1] range used by pattern.nl, and then added to its lexicon, increasing the coverage to a total of 6,877 unique words. We compared the effect of this extension by measuring the mean average error (MAE) of the original version of pattern.nl and our extended version against a balanced dataset of 11,180 book reviews and the associated ratings (1 to 5 stars) collected from bol.com. Preliminary experiments on the correlation between the common subset between pattern.nl and Moors' (0.8) bode well, but despite the increase in coverage, preliminary results are negative: the original version of pattern.nl seems to perform better than after the lexicon expansion. This was also confirmed by further tests, where we tried, to no avail, – removing Moors' words centered around 0 (i.e., neutral ones); – removing stopwords from Moors'; – replicating the original pattern.nl evaluation by binarizing the dataset and using F1 score. Part of the problem might lie in the dataset used for the evaluation: reviews are related to sentiment, but indirectly; furthermore, the dataset is very noisy. Different results could also be obtained by PoS-tagging and lemmatizing the data, a step that is not technically required but might be beneficial to increase the coverage of Moors' lexicon in sentences. We are currently looking for suitable datasets for Dutch that can be used to evaluate our extension, preferably datasets that are more general domain than product reviews.
更多
分享
引用
