AI helps you reading Science

AI generates interpretation videos

AI extracts and analyses the key points of the paper to generate videos automatically


pub
Go Generating

AI Traceability

AI parses the academic lineage of this thesis


Master Reading Tree
Generate MRT

AI Insight

AI extracts a summary of this paper


Weibo:
We explored the potential of BERT to domain adaptation, and proposed a unified feature and instance-based adaptation approach for both tasks of cross-domain End2End Aspect Based Sentiment Analysis and cross-domain aspect extraction

Unified Feature and Instance Based Domain Adaptation for Aspect Based Sentiment Analysis

EMNLP 2020, (2020)

Cited by: 0|Views193
Full Text
Bibtex
Weibo

Abstract

The supervised models for aspect-based sentiment analysis (ABSA) rely heavily on labeled data. However, fine-grained labeled data are scarce for the ABSA task. To alleviate the dependence on labeled data, prior works mainly focused on feature-based adaptation, which used the domain-shared knowledge to construct auxiliary tasks or domain a...More

Code:

Data:

0
Introduction
  • Aspect extraction and aspect sentiment classification are two important sub-tasks in Aspect Based Sentiment Analysis (ABSA) (Liu, 2012; Pontiki et al, 2016), which aim to extract aspect terms and predict the sentiment polarities of the given aspect terms, respectively
  • Since these two sub-tasks have been well studied in the literature, a number of recent studies focus on the End2End ABSA task by employing a unified tagging scheme to tackle the two sub-tasks in an end-to-end manner (Mitchell et al, 2013; Zhang et al, 2015; Li et al, 2019a).
  • It will be very attractive to explore the End2End ABSA task in a cross-domain setting, which allows them to train a robust ABSA model for a resource-poor target domain based on enough annotated data in a resource-rich source domain
Highlights
  • Aspect extraction and aspect sentiment classification are two important sub-tasks in Aspect Based Sentiment Analysis (ABSA) (Liu, 2012; Pontiki et al, 2016), which aim to extract aspect terms and predict the sentiment polarities of the given aspect terms, respectively. Since these two sub-tasks have been well studied in the literature, a number of recent studies focus on the End2End ABSA task by employing a unified tagging scheme to tackle the two sub-tasks in an end-to-end manner (Mitchell et al, 2013; Zhang et al, 2015; Li et al, 2019a)
  • Experimental results on four benchmark datasets show that our method can significantly improve the performance of cross-domain End2End ABSA and cross-domain aspect extraction, and we further carry out ablation studies to quantitatively measure the effectiveness of each component in our unified framework
  • We propose a Unified Domain Adaptation (UDA) framework encompassing both featurebased adaptation and instance-based adaptation, which can significantly improve the performance of the fine-tuned BERT model without domain adaptation
  • Different from their work, we primarily focus on the cross-domain End2End ABSA task in this work, which aims to first extract aspect terms followed by identifying the sentiment towards each detected aspect term
  • By comparing BERT-based approaches, we can clearly see that performing adversarial training (i.e., DANN) for each word does not give satisfactory improvements over BERTB and BERTE, whereas our UDA approach can significantly outperform all the BERT-based baselines and consistently achieve the best performance on all the transfer pairs
  • We explored the potential of BERT to domain adaptation, and proposed a unified feature and instance-based adaptation approach for both tasks of cross-domain End2End ABSA and cross-domain aspect extraction
Methods

  • The overall comparison results on cross-domain End2End ABSA are shown in Table 2.
  • By comparing BERT-based approaches, the authors can clearly see that performing adversarial training (i.e., DANN) for each word does not give satisfactory improvements over BERTB and BERTE, whereas the UDA approach can significantly outperform all the BERT-based baselines and consistently achieve the best performance on all the transfer pairs
  • All these observations demonstrate the effectiveness of the UDA framework
Results
  • The evaluation metric the authors used is Micro-F1.
  • Following the setting in existing work, only exact match could be counted as correct.
  • All experiments are repeated 5 times and the authors report the average results over 5 runs.
  • 4.2 Baselines & Main Results.
  • The authors compare the Unified Domain Adaptation (UDA) approach with several highly competitive DA methods as follows: 2https://www.yelp.com/dataset/ challenge
Conclusion
  • The authors explored the potential of BERT to domain adaptation, and proposed a unified feature and instance-based adaptation approach for both tasks of cross-domain End2End ABSA and cross-domain aspect extraction.
  • In feature-based domain adaptation, the authors use domain-shared syntactic relations and POS tags to construct auxiliary tasks, which can help learn domain-invariant representations for domain adaptation.
  • In instance-based domain adaptation, the authors employ a domain classifier to learn to assign appropriate weights for each word.
  • Extensive experiments on four benchmark datasets demonstrate the superiority of the Unified Domain Adaptation (UDA) approach over existing methods in both cross-domain End2End ABSA and cross-domain aspect extraction
Tables
  • Table1: Statistics of the datasets
  • Table2: Comparison results for cross-domain End2End ABSA based on Micro-F1. The results marked by † are extracted from Li et al (2019b). It is worth noting that different from Li et al (2019b), we did not remove training/test samples where all the tokens are labeled as ‘O’ in our experiments, because a moderate amount of product reviews only contain implicit aspects in real scenarios. If we remove these samples, we can get an extra improvement of around 5% on Micro-F1 for all the BERT-based methods in our preliminary experiments
  • Table3: Comparison results for cross-domain Aspect Extraction (AE) based on Micro-F1
  • Table4: Ablation study of our UDA approach based on BERTE for cross-domain End2End ABSA
  • Table5: Words with higher instance weights in the instance-based adaptation component of our UDA approach
Download tables as Excel
Related work
  • Aspect extraction and aspect-level sentiment classification are two important subtasks in AspectBased Sentiment Analysis (ABSA), which aim to extract aspect terms and identify the sentiment orientations towards them, respectively (Liu, 2012). As two fundamental tasks, aspect extraction (Qiu

    S→R L→R D→R contentious, bearing, hated, beauty, ##mi, amazement, ##ant, canned, mistake, madden, accused, nicely, employee, proud, difficulty, impressive, likely, catalogue, ##working enjoying, lesson, strongly, reality, comfortably, artwork, food, loving, dissatisfaction, spice, ##kind, fork, appears, weary, desk, projects, monster, covering, recipients, purchases displayed, desk, robust, lightly, capable, waking, satisfactory, birthday, releasing, kitchen, noises, appearing, experiences, sophisticated, extreme, providing, nuts, interaction, recommendations et al, 2011; Liu et al, 2015; Poria et al, 2016; Wang et al, 2016a, 2017; Li et al, 2018a; Xu et al, 2018) and aspect-level sentiment classification (Dong et al, 2014; Tang et al, 2016; Wang et al, 2016b; Ma et al, 2017; Wang et al, 2018; Li et al, 2019c) have been extensively studied in the literature.

    Since these two tasks are strongly related with each other, a number of previous studies propose to tackle them together in an end-to-end manner (Mitchell et al, 2013; Zhang et al, 2015). Some recent studies have further demonstrated that a unified tagging scheme can effectively eliminate the error propagation issue of traditional pipeline methods, and thus achieve the state-of-the-art performance. However, since annotating each word with fine-grained label is time-consuming, it is next to impossible to obtain enough annotated data for the ABSA task in every new domain. Therefore, in this work, we resort to transfer learning, and focus on proposing an effective domain adaptation approach for the ABSA task.
Funding
  • This work was supported by the Natural Science Foundation of China (No 61672288, 62076133, 62006117, and 72001102), and the Natural Science Foundation of Jiangsu Province for Young Scholars (SBK2020040749) and Distinguished Young Scholars (SBK2020010154)
Study subjects and analysis
benchmark datasets: 4
Finally, we propose a unified framework to jointly perform feature and instance-based adaptation via sequential learning and joint learning, respectively. Experimental results on four benchmark datasets show that our method can significantly improve the performance of cross-domain End2End ABSA and cross-domain aspect extraction, and we further carry out ablation studies to quantitatively measure the effectiveness of each component in our unified framework. The main contributions of this paper can be summarized as follows:

benchmark datasets: 4
4.1 Data & Experiment Setup. Datasets: We conduct experiments on four benchmark datasets: Laptop (L), Restaurant(R), Device (D), and Service (S). L contains reviews from the laptop domain in SemEval-2014 ABSA challenge (Pontiki et al, 2014)

transfer pairs: 3
It is easy to see that Joint performs slightly better than Sequential, which shows the advantages of joint optimization. To qualitatively show the effect of our wordlevel instance weighting method, we show the most important words for the target domain on three transfer pairs in Table 5. The results show that the common opinion words (e.g., beauty, amazement and satisfactory) or aspect words (e.g., employee, desk and kitchen) gain more weight in the wordlevel instance weighting

benchmark datasets: 4
In instance-based domain adaptation, we employ a domain classifier to learn to assign appropriate weights for each word. Extensive experiments on four benchmark datasets demonstrate the superiority of our Unified Domain Adaptation (UDA) approach over existing methods in both cross-domain End2End ABSA and cross-domain aspect extraction. We would like to thank three anonymous reviewers for their insightful comments and helpful suggestions

Reference
  • Rie Kubota Ando and Tong Zhang. 2005. A framework for learning predictive structures from multiple tasks and unlabeled data. Journal of Machine Learning Research, 6(Nov):1817–1853.
    Google ScholarLocate open access versionFindings
  • Steffen Bickel, Michael Bruckner, and Tobias Scheffer. 2007. Discriminative learning for differing training and test distributions. In Proceedings of the 24th international conference on Machine learning.
    Google ScholarLocate open access versionFindings
  • John Blitzer, Mark Dredze, and Fernando Pereira. 2007.
    Google ScholarFindings
  • Biographies, bollywood, boom-boxes and blenders: Domain adaptation for sentiment classification. In Proceedings of the 45th annual meeting of the association of computational linguistics.
    Google ScholarLocate open access versionFindings
  • Danushka Bollegala, David Weir, and John Carroll. 2012. Cross-domain sentiment classification using a sentiment sensitive thesaurus. IEEE transactions on knowledge and data engineering, 25(8):1719–1731.
    Google ScholarLocate open access versionFindings
  • Minmin Chen, Zhixiang Xu, Kilian Q Weinberger, and Fei Sha. 2012. Marginalized denoising autoencoders for domain adaptation. In Proceedings of the 29th International Coference on International Conference on Machine Learning.
    Google ScholarLocate open access versionFindings
  • Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805.
    Findings
  • Ying Ding, Jianfei Yu, and Jing Jiang. 2017. Recurrent neural networks with auxiliary labels for crossdomain opinion target extraction. In Thirty-First AAAI Conference on Artificial Intelligence.
    Google ScholarLocate open access versionFindings
  • Li Dong, Furu Wei, Chuanqi Tan, Duyu Tang, Ming Zhou, and Ke Xu. 2014. Adaptive recursive neural network for target-dependent twitter sentiment classification. In Proceedings of the 52nd annual meeting of the association for computational linguistics (volume 2: Short papers).
    Google ScholarLocate open access versionFindings
  • Yaroslav Ganin, Evgeniya Ustinova, Hana Ajakan, Pascal Germain, Hugo Larochelle, Francois Laviolette, Mario Marchand, and Victor Lempitsky. 2016. Domain-adversarial training of neural networks. The Journal of Machine Learning Research, 17(1):2096–2030.
    Google ScholarLocate open access versionFindings
  • Xavier Glorot, Antoine Bordes, and Yoshua Bengio. 20Domain adaptation for large-scale sentiment classification: a deep learning approach. In Proceedings of the 28th International Conference on International Conference on Machine Learning.
    Google ScholarLocate open access versionFindings
  • Ruining He and Julian McAuley. 2016. Ups and downs: Modeling the visual evolution of fashion trends with one-class collaborative filtering. In proceedings of the 25th international conference on world wide web.
    Google ScholarLocate open access versionFindings
  • Sinno Jialin Pan, Xiaochuan Ni, Jian-Tao Sun, Qiang Yang, and Zheng Chen. 2010. Cross-domain sentiment classification via spectral feature alignment. In Proceedings of the 19th international conference on World wide web.
    Google ScholarLocate open access versionFindings
  • Minqing Hu and Bing Liu. 2004. Mining and summarizing customer reviews. In Proceedings of the tenth ACM SIGKDD international conference on Knowledge discovery and data mining.
    Google ScholarLocate open access versionFindings
  • Jing Jiang and ChengXiang Zhai. 2007. Instance weighting for domain adaptation in nlp. In Proceedings of the 45th annual meeting of the association of computational linguistics.
    Google ScholarLocate open access versionFindings
  • Diederik P Kingma and Jimmy Ba. 2014. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980.
    Findings
  • Xin Li, Lidong Bing, Piji Li, and Wai Lam. 2019a. A unified model for opinion target extraction and target sentiment prediction. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 33.
    Google ScholarLocate open access versionFindings
  • Xin Li, Lidong Bing, Piji Li, Wai Lam, and Zhimou Yang. 2018a. Aspect term extraction with history attention and selective transformation. arXiv preprint arXiv:1805.00760.
    Findings
  • Zheng Li, Xin Li, Ying Wei, Lidong Bing, Yu Zhang, and Qiang Yang. 2019b. Transferable end-to-end aspect-based sentiment analysis with selective adversarial learning. arXiv preprint arXiv:1910.14192.
    Findings
  • Zheng Li, Ying Wei, Yu Zhang, and Qiang Yang. 2018b. Hierarchical attention transfer network for cross-domain sentiment classification. In ThirtySecond AAAI Conference on Artificial Intelligence.
    Google ScholarLocate open access versionFindings
  • Zheng Li, Ying Wei, Yu Zhang, Xiang Zhang, and Xin Li. 2019c. Exploiting coarse-to-fine task transfer for aspect-level sentiment classification. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 33.
    Google ScholarLocate open access versionFindings
  • Bing Liu. 2012. Sentiment analysis and opinion mining. Synthesis lectures on human language technologies, 5(1):1–167.
    Google ScholarLocate open access versionFindings
  • Pengfei Liu, Shafiq Joty, and Helen Meng. 2015. Finegrained opinion mining with recurrent neural networks and word embeddings. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing.
    Google ScholarLocate open access versionFindings
  • Dehong Ma, Sujian Li, Xiaodong Zhang, and Houfeng Wang. 2017. Interactive attention networks for aspect-level sentiment classification. arXiv preprint arXiv:1709.00893.
    Findings
  • Margaret Mitchell, Jacqui Aguilar, Theresa Wilson, and Benjamin Van Durme. 2013. Open domain targeted sentiment. In Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing.
    Google ScholarLocate open access versionFindings
  • Matthew E Peters, Mark Neumann, Mohit Iyyer, Matt Gardner, Christopher Clark, Kenton Lee, and Luke Zettlemoyer. 2018. Deep contextualized word representations. arXiv preprint arXiv:1802.05365.
    Findings
  • Maria Pontiki, Dimitrios Galanis, Haris Papageorgiou, Ion Androutsopoulos, Suresh Manandhar, Mohammad Al-Smadi, Mahmoud Al-Ayyoub, Yanyan Zhao, Bing Qin, Orphee De Clercq, et al. 2016. Semeval-2016 task 5: Aspect based sentiment analysis. In 10th International Workshop on Semantic Evaluation (SemEval 2016).
    Google ScholarLocate open access versionFindings
  • Maria Pontiki, Dimitrios Galanis, Harris Papageorgiou, Suresh Manandhar, and Ion Androutsopoulos. 2015. Semeval-2015 task 12: Aspect based sentiment analysis. In Proceedings of the 9th international workshop on semantic evaluation (SemEval 2015).
    Google ScholarLocate open access versionFindings
  • Maria Pontiki, Dimitris Galanis, John Pavlopoulos, Harris Papageorgiou, Ion Androutsopoulos, and Auresh Manandhar. 2014. Semeval-2014 task 4: Aspect based sentiment analysis. In Proceedings of the 8th international workshop on semantic evaluation (SemEval 2014).
    Google ScholarLocate open access versionFindings
  • Soujanya Poria, Erik Cambria, and Alexander Gelbukh. 2016. Aspect extraction for opinion mining with a deep convolutional neural network. KnowledgeBased Systems, 108:42–49.
    Google ScholarLocate open access versionFindings
  • Guang Qiu, Bing Liu, Jiajun Bu, and Chun Chen. 2011. Opinion word expansion and target extraction through double propagation. Computational linguistics, 37(1):9–27.
    Google ScholarLocate open access versionFindings
  • Alec Radford, Karthik Narasimhan, Tim Salimans, and Ilya Sutskever. 2018. Improving language understanding by generative pre-training. URL https://s3-us-west-2.amazonaws.com/openaiassets/researchcovers/languageunsupervised/language understanding paper.pdf.
    Findings
  • Alexander Rietzler, Sebastian Stabinger, Paul Opitz, and Stefan Engl. 2019. Adapt or get left behind: Domain adaptation through bert language model finetuning for aspect-target sentiment classification. arXiv preprint arXiv:1908.11860.
    Findings
  • Duyu Tang, Bing Qin, and Ting Liu. 2016. Aspect level sentiment classification with deep memory network. arXiv preprint arXiv:1605.08900.
    Findings
  • Cigdem Toprak, Niklas Jakob, and Iryna Gurevych. 2010. Sentence and expression level annotation of opinions in user-generated discourse. In Proceedings of the 48th Annual Meeting of the Association for Computational Linguistics. Association for Computational Linguistics.
    Google ScholarLocate open access versionFindings
  • Shuai Wang, Sahisnu Mazumder, Bing Liu, Mianwei Zhou, and Yi Chang. 2018. Target-sensitive memory networks for aspect sentiment classification. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics.
    Google ScholarLocate open access versionFindings
  • Wenya Wang and Sinno Jialin Pan. 2018. Recursive neural structural correspondence network for crossdomain aspect and opinion co-extraction. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers).
    Google ScholarLocate open access versionFindings
  • Wenya Wang, Sinno Jialin Pan, Daniel Dahlmeier, and Xiaokui Xiao. 2016a. Recursive neural conditional random fields for aspect-based sentiment analysis. arXiv preprint arXiv:1603.06679.
    Findings
  • Wenya Wang, Sinno Jialin Pan, Daniel Dahlmeier, and Xiaokui Xiao. 2017. Coupled multi-layer attentions for co-extraction of aspect and opinion terms. In Thirty-First AAAI Conference on Artificial Intelligence.
    Google ScholarLocate open access versionFindings
  • Yequan Wang, Minlie Huang, Xiaoyan Zhu, and Li Zhao. 2016b. Attention-based lstm for aspectlevel sentiment classification. In EMNLP 2016: Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing.
    Google ScholarLocate open access versionFindings
  • Rui Xia, Xuelei Hu, Jianfeng Lu, Jian Yang, and Chengqing Zong. 2013. Instance selection and instance weighting for cross-domain sentiment classification via pu learning. In Twenty-Third International Joint Conference on Artificial Intelligence.
    Google ScholarLocate open access versionFindings
  • Rui Xia, Jianfei Yu, Feng Xu, and Shumei Wang. 2014. Instance-based domain adaptation in nlp via in-target-domain logistic approximation. In Proceedings of the Twenty-Eighth AAAI Conference on Artificial Intelligence.
    Google ScholarLocate open access versionFindings
  • Hu Xu, Bing Liu, Lei Shu, and Philip S Yu. 2018. Double embeddings and cnn-based sequence labeling for aspect extraction. arXiv preprint arXiv:1805.04601.
    Findings
  • Hu Xu, Bing Liu, Lei Shu, and Philip S Yu. 2019. Bert post-training for review reading comprehension and aspect-based sentiment analysis. arXiv preprint arXiv:1904.02232.
    Findings
  • Jianfei Yu and Jing Jiang. 2016. Learning sentence embeddings with auxiliary tasks for cross-domain sentiment classification. In EMNLP 2016: Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing.
    Google ScholarLocate open access versionFindings
  • Jianfei Yu and Jing Jiang. 2017. Leveraging auxiliary tasks for document-level cross-domain sentiment classification. In Proceedings of the Eighth International Joint Conference on Natural Language Processing.
    Google ScholarLocate open access versionFindings
  • Meishan Zhang, Yue Zhang, and Duy-Tin Vo. 2015. Neural networks for open domain targeted sentiment. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing.
    Google ScholarLocate open access versionFindings
  • Fuzhen Zhuang, Xiaohu Cheng, Ping Luo, Sinno Jialin Pan, and Qing He. 2015. Supervised representation learning: Transfer learning with deep autoencoders. In Twenty-Fourth International Joint Conference on Artificial Intelligence.
    Google ScholarLocate open access versionFindings
Your rating :
0

 

Tags
Comments
数据免责声明
页面数据均来自互联网公开来源、合作出版商和通过AI技术自动分析结果,我们不对页面数据的有效性、准确性、正确性、可靠性、完整性和及时性做出任何承诺和保证。若有疑问,可以通过电子邮件方式联系我们:report@aminer.cn
小科