AI helps you reading Science

AI generates interpretation videos

AI extracts and analyses the key points of the paper to generate videos automatically


pub
Go Generating

AI Traceability

AI parses the academic lineage of this thesis


Master Reading Tree
Generate MRT

AI Insight

AI extracts a summary of this paper


Weibo:
Different from traditional keyphrase extraction models mainly focusing on text, SMARTKPE illustrates the advantage of incorporating other modalities to help keyphrases location and salience prediction

Incorporating Multimodal Information in Open Domain Web Keyphrase Extraction

EMNLP 2020, pp.1790-1800, (2020)

Cited by: 0|Views121
Full Text
Bibtex
Weibo

Abstract

Open-domain Keyphrase extraction (KPE) on the Web is a fundamental yet complex NLP task with a wide range of practical applications within the field of Information Retrieval. In contrast to other document types, web page designs are intended for easy navigation and information finding. Effective designs encode within the layout and format...More

Code:

Data:

0
Introduction
  • The authors present a novel multi-modal approach to KeyPhrase Extraction (KPE), which is the task of automatically extracting salient phrases from a given document.
  • The KPE task requires accurate selection of the phrases that best capture the web document’s topic.
  • The earliest KPE approaches were mainly limited to domain-specific keyphrase extraction.
  • The recent release of OpenKP (Xiong et al, 2019), a large-scale feature-rich dataset developed for open-domain web-page keyphrase extraction, has encouraged further research related to the KPE task.
  • A novel characteristic of this data set is the inclusion of features related to visual properties
Highlights
  • We present a novel multi-modal approach to KeyPhrase Extraction (KPE), which is the task of automatically extracting salient phrases from a given document
  • We propose a multimodal framework, Strategy-based Multimodal ARchiTecture for KeyPhrase Extraction (SMART-KPE), addressing the web KPE task in two steps: Multimodal Strategy Induction to apply specific extraction tactics with a refined use of micro-level features and Strategy Selection Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, pages 1790–1800, November 16–20, 2020. c 2020 Association for Computational Linguistics to choose results from different tactics using macro-level features
  • In this work we propose a Strategy-based Multimodal Architecture for Keyphrase Extraction (SMART-KPE) as a new state-of-the-art method for multimodal web-page keyphrase extraction
  • Different from traditional keyphrase extraction models mainly focusing on text, SMARTKPE illustrates the advantage of incorporating other modalities to help keyphrases location and salience prediction
  • Our proposed model outperforms several state-of-the-art baselines with the introduction of multimodal information
Methods
  • 4.1 Dataset

    The authors set OpenKP as the main dataset for the task. OpenKP consists of ∼150K documents sampled from the Bing search engine, within which neither the domain nor type of original web pages are restricted.

    For each document, the following information is given:

    URL: The link to the respective web page.

    Text: Cleaned body text of a document.

    Visual DOM features: A set of vectors representing the visual characteristics of text terms, listed in Table 1.
  • The keyphrases for each document in the given dataset were labeled by expert annotators, with each document assigned 1-3 keyphrases.
  • All the keyphrases were ones that appeared in the original document.
  • The detailed statistics of OpenKP are displayed in Table 2
Results
  • The authors perform experiments on the full SMART-KPE model and its 3 variants, where only micro-level visual features (SMART-KPEMicro), only macro-level meta-features (SMARTKPE-Macro), and neither set of features (SMARTKPE-Skeleton) are applied respectively.
  • The authors see that all variants of BERT-based SMART-KPE outperform BERT2Tag and BERT2Joint on all metrics, suggesting the effectiveness of feature construction and strategy selection.
  • Wagon Wheel Old Crow Medicine Show Produced by David Rawlings Album Old Crow Medicine Show.
  • Wagon Wheel Lyrics.
  • Verse 1 Headed down south to the land of the pines
Conclusion
  • In this work the authors propose a Strategy-based Multimodal Architecture for Keyphrase Extraction (SMART-KPE) as a new state-of-the-art method for multimodal web-page keyphrase extraction.
  • Different from traditional keyphrase extraction models mainly focusing on text, SMARTKPE illustrates the advantage of incorporating other modalities to help keyphrases location and salience prediction.
  • The authors' proposed model outperforms several state-of-the-art baselines with the introduction of multimodal information.
Summary
  • Introduction:

    The authors present a novel multi-modal approach to KeyPhrase Extraction (KPE), which is the task of automatically extracting salient phrases from a given document.
  • The KPE task requires accurate selection of the phrases that best capture the web document’s topic.
  • The earliest KPE approaches were mainly limited to domain-specific keyphrase extraction.
  • The recent release of OpenKP (Xiong et al, 2019), a large-scale feature-rich dataset developed for open-domain web-page keyphrase extraction, has encouraged further research related to the KPE task.
  • A novel characteristic of this data set is the inclusion of features related to visual properties
  • Objectives:

    The authors formalize the keyphrase extraction task (KPE) under the web page setting: Given a document D = {W, V, M }, where W = {w1, w2, .
  • Wn} are the text terms of the web page with length n, V = {v1, v2, .
  • Vn} are the respective visual features of each term and M is the set of macro-level meta-features describing the document, the authors aim to find the set of word sub-sequences S = {S1, S2, ..., SK } where Si = {wji, .
  • Methods:

    4.1 Dataset

    The authors set OpenKP as the main dataset for the task. OpenKP consists of ∼150K documents sampled from the Bing search engine, within which neither the domain nor type of original web pages are restricted.

    For each document, the following information is given:

    URL: The link to the respective web page.

    Text: Cleaned body text of a document.

    Visual DOM features: A set of vectors representing the visual characteristics of text terms, listed in Table 1.
  • The keyphrases for each document in the given dataset were labeled by expert annotators, with each document assigned 1-3 keyphrases.
  • All the keyphrases were ones that appeared in the original document.
  • The detailed statistics of OpenKP are displayed in Table 2
  • Results:

    The authors perform experiments on the full SMART-KPE model and its 3 variants, where only micro-level visual features (SMART-KPEMicro), only macro-level meta-features (SMARTKPE-Macro), and neither set of features (SMARTKPE-Skeleton) are applied respectively.
  • The authors see that all variants of BERT-based SMART-KPE outperform BERT2Tag and BERT2Joint on all metrics, suggesting the effectiveness of feature construction and strategy selection.
  • Wagon Wheel Old Crow Medicine Show Produced by David Rawlings Album Old Crow Medicine Show.
  • Wagon Wheel Lyrics.
  • Verse 1 Headed down south to the land of the pines
  • Conclusion:

    In this work the authors propose a Strategy-based Multimodal Architecture for Keyphrase Extraction (SMART-KPE) as a new state-of-the-art method for multimodal web-page keyphrase extraction.
  • Different from traditional keyphrase extraction models mainly focusing on text, SMARTKPE illustrates the advantage of incorporating other modalities to help keyphrases location and salience prediction.
  • The authors' proposed model outperforms several state-of-the-art baselines with the introduction of multimodal information.
Tables
  • Table1: Visual features in OpenKP
  • Table2: Statistics of OpenKP
  • Table3: Model performances on the OpenKP development set. F1@3 is the main metric for this task. SMARTKPE-Full is the complete model and Skeleton, Micro and Macro denote for ablations where no additional features, only micro-level visual features, or only macro-level features are introduced respectively. SMART-KPE+R2J is our complete model equipped with the state-of-the-art extracting method (RoBERTa2Joint)
  • Table4: Parameters used for training SMART-KPE
  • Table5: Case Study of 3 web pages. Part of the original text, all golden keyphrases and top 3 predicted keyphrases are presented for each case. The correctly predicted keyphrases are highlighted in red. The snapshots of these 3 web pages are shown in Figure 2. Note that in the original data, punctuation is absent and keyphrases are case-insensitive. The text snippets we show here are restored from the original websites
  • Table6: Overlap rate of SMART-KPE predictions
Download tables as Excel
Related work
  • 2.1 Development of Open-domain Web Keyphrase Extraction

    Originally, the concept keyphrase was first used by authors of scientific papers when they indicated by hand a few phrases they decided best summarized their paper (Cano and Bojar, 2019). The first corpora for automated keyphrase extraction were likewise assembled out of publications from scientific fields including technical reports (Witten et al, 1999), paper abstracts (Hulth, 2003), and scientific papers (Nguyen and Kan, 2007; Medelyan et al, 2009; Kim et al, 2010). To this day, scientific publications still serve as a fundamental fixed-domain benchmark for neural KPE methods (Meng et al, 2017; Alzaidy et al, 2019; Sahrawat et al, 2019) due to the availability of ample data of this kind. However, experiments have revealed that KPE methods trained directly on such corpora do not generalize well to other web-related genres or other types of documents (Chen et al, 2018; Xiong et al, 2019), where there may be far more heterogeneity in topics, content and structure, and there may be more variation in terms of where a key phrase may appear.

    Past researchers have collected corpora for KPE in Internet and social media environments, including web pages (Yih et al, 2006; Hammouda et al, 2005), blogs (Grineva et al, 2009), email (Dredze et al, 2008), news articles (Wan and Xiao, 2008; Hulth and Megyesi, 2006) and live chats (Kim and Baldwin, 2012), but most of these existing corpora fall prey to similar problems with respect to robust model training for neural models due to data sparsity and lack of representativeness in topic distribution. The recently released OpenKP (Xiong et al, 2019) is the first large-scale KPE dataset with a broad distribution of topic domains. This recent dataset facilitates work on model generalization and the opportunity to develop nuanced models that can adapt their performance based on the type of document they are applied to. This property of the dataset has inspired our proposed method where strategies are selected based on the detected type of document using macro-level features.
Funding
  • This work was funded in part by NSF grants IIS 1822831 and 1917955 and funding from Microsoft
Study subjects and analysis
Characters: 4
Naira Crashes Below N360 Per Dollar at the Parallel Market as External. Part 4 Characters. These are the characters featured in Diamond Is Unbreakable

cases: 3
5.2 Case Study of Visual Feature Usage. We demonstrate the effects of introducing microlevel visual features by showing 3 cases from the validation set of OpenKP in Table 5. We present prediction results from SMART-KPESkeleton (SMART-KPE using only text features) and the complete SMART-KPE model

Reference
  • Rabah Alzaidy, Cornelia Caragea, and C Lee Giles. 2019. Bi-lstm-crf sequence labeling for keyphrase extraction from scholarly documents. In The world wide web conference, pages 2551–2557.
    Google ScholarLocate open access versionFindings
  • Erion Cano and Ondrej Bojar. 2019. Keyphrase generation: A text summarization struggle. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 666–672.
    Google ScholarLocate open access versionFindings
  • Jun Chen, Xiaoming Zhang, Yu Wu, Zhao Yan, and Zhoujun Li. 2018. Keyphrase generation with correlation constraints. arXiv preprint arXiv:1808.07185.
    Findings
  • Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805.
    Findings
  • Mark Dredze, Hanna M Wallach, Danny Puller, and Fernando Pereira. 2008. Generating summary keywords for emails using topics. In Proceedings of the 13th international conference on Intelligent user interfaces, pages 199–206.
    Google ScholarLocate open access versionFindings
  • Maria Grineva, Maxim Grinev, and Dmitry Lizorkin. 2009. Extracting key terms from noisy and multitheme documents. In Proceedings of the 18th international conference on World wide web, pages 661– 670.
    Google ScholarLocate open access versionFindings
  • Khaled M Hammouda, Diego N Matute, and Mohamed S Kamel. 2005. Corephrase: Keyphrase extraction for document clustering. In International workshop on machine learning and data mining in pattern recognition, pages 265–274. Springer.
    Google ScholarLocate open access versionFindings
  • Kazi Saidul Hasan and Vincent Ng. 2014. Automatic keyphrase extraction: A survey of the state of the art. In Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1262–1273.
    Google ScholarLocate open access versionFindings
  • Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770– 778.
    Google ScholarLocate open access versionFindings
  • Anette Hulth. 2003. Improved automatic keyword extraction given more linguistic knowledge. In Proceedings of the 2003 conference on Empirical methods in natural language processing, pages 216–223.
    Google ScholarLocate open access versionFindings
  • Anette Hulth and Beata B Megyesi. 2006. A study on automatically extracted keywords in text categorization. In Proceedings of the 21st International Conference on Computational Linguistics and the 44th annual meeting of the Association for Computational Linguistics, pages 537–544. Association for Computational Linguistics.
    Google ScholarLocate open access versionFindings
  • Su Nam Kim and Timothy Baldwin. 20Extracting keywords from multi-party live chats. In Proceedings of the 26th Pacific Asia Conference on Language, Information, and Computation, pages 199– 208.
    Google ScholarLocate open access versionFindings
  • Su Nam Kim, Olena Medelyan, Min-Yen Kan, and Timothy Baldwin. 2010. Semeval-2010 task 5: Automatic keyphrase extraction from scientific articles. In Proceedings of the 5th International Workshop on Semantic Evaluation, pages 21–26.
    Google ScholarLocate open access versionFindings
  • Zhiyuan Liu, Wenyi Huang, Yabin Zheng, and Maosong Sun. 2010. Automatic keyphrase extraction via topic decomposition. In Proceedings of the 2010 conference on empirical methods in natural language processing, pages 366–376. Association for Computational Linguistics.
    Google ScholarLocate open access versionFindings
  • Zhiyuan Liu, Peng Li, Yabin Zheng, and Maosong Sun. 2009. Clustering to find exemplar terms for keyphrase extraction. In Proceedings of the 2009 Conference on Empirical Methods in Natural Language Processing: Volume 1-Volume 1, pages 257– 266. Association for Computational Linguistics.
    Google ScholarLocate open access versionFindings
  • Olena Medelyan, Eibe Frank, and Ian H Witten. 2009. Human-competitive tagging using automatic keyphrase extraction. In Proceedings of the 2009 Conference on Empirical Methods in Natural Language Processing: Volume 3-Volume 3, pages 1318– 1327. Association for Computational Linguistics.
    Google ScholarLocate open access versionFindings
  • Olena Medelyan and Ian Witten. 2002. Thesaurus based automatic keyphrase indexing. In Proceedings of the 6th ACM/IEED-CS joint conference on Digital libraries, pages 296–297.
    Google ScholarLocate open access versionFindings
  • Rui Meng, Sanqiang Zhao, Shuguang Han, Daqing He, Peter Brusilovsky, and Yu Chi. 2017. Deep keyphrase generation. arXiv preprint arXiv:1704.06879.
    Findings
  • Thuy Dung Nguyen and Min-Yen Kan. 2007. Key phrase extraction in scientific publications. In Proceeding of International Conference on Asian Digital Libraries, pages 317–326.
    Google ScholarLocate open access versionFindings
  • Tri Nguyen, Mir Rosenberg, Xia Song, Jianfeng Gao, Saurabh Tiwary, Rangan Majumder, and Li Deng. 2016. Ms marco: a human-generated machine reading comprehension dataset.
    Google ScholarFindings
  • Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Kopf, Edward Yang, Zachary DeVito, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, Junjie Bai, and Soumith Chintala. 2019. Pytorch: An imperative style, high-performance deep learning library. In H. Wallach, H. Larochelle, A. Beygelzimer, F. dAlche-Buc, E. Fox, and R. Garnett, editors, Advances in Neural Information Processing Systems 32, pages 8024–8035. Curran Associates, Inc.
    Google ScholarLocate open access versionFindings
  • Matthew E Peters, Mark Neumann, Mohit Iyyer, Matt Gardner, Christopher Clark, Kenton Lee, and Luke Zettlemoyer. 2018. Deep contextualized word representations. In Proceedings of NAACL-HLT, pages 2227–2237.
    Google ScholarLocate open access versionFindings
  • Dhruva Sahrawat, Debanjan Mahata, Mayank Kulkarni, Haimin Zhang, Rakesh Gosangi, Amanda Stent, Agniv Sharma, Yaman Kumar, Rajiv Ratn Shah, and Roger Zimmermann. 2019. Keyphrase extraction from scholarly articles as sequence labeling using contextualized embeddings. arXiv preprint arXiv:1910.08840.
    Findings
  • Si Sun, Chenyan Xiong, Zhenghao Liu, Zhiyuan Liu, and Jie Bao. 2020. Joint keyphrase chunking and salience ranking with bert.
    Google ScholarFindings
  • Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In Advances in neural information processing systems, pages 5998–6008.
    Google ScholarLocate open access versionFindings
  • Xiaojun Wan and Jianguo Xiao. 2008. Collabrank: Towards a collaborative approach to single-document keyphrase extraction. In Proceedings of 22nd International Conference on Computational Linguistics, pages 969–976, Manchester, UK.
    Google ScholarLocate open access versionFindings
  • Ian Witten, Gordon Paynter, Eibe Frank, Car Gutwin, and Graig Nevill-Manning. 1999. Kea:practical automatic key phrase extraction. In Proceedings of the fourth ACM conference on Digital libraries, pages 254–256.
    Google ScholarLocate open access versionFindings
  • Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, R’emi Louf, Morgan Funtowicz, and Jamie Brew. 2019. Huggingface’s transformers: State-of-the-art natural language processing. ArXiv, abs/1910.03771.
    Findings
  • Lee Xiong, Chuan Hu, Chenyan Xiong, Daniel Campos, and Arnold Overwijk. 2019. Open domain web keyphrase extraction beyond language modeling. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 5178– 5187.
    Google ScholarLocate open access versionFindings
  • Wen-tau Yih, Joshua Goodman, and Vitor R Carvalho. 2006. Finding advertising keywords on web pages. In Proceedings of the 15th international conference on World Wide Web, pages 213–222.
    Google ScholarLocate open access versionFindings
  • Yong Zhang, Yang Fang, and Xiao Weidong. 2017. Deep keyphrase generation with a convolutional sequence to sequence model. In 2017 4th International Conference on Systems and Informatics (ICSAI), pages 1477–1485. IEEE.
    Google ScholarLocate open access versionFindings
Author
Yansen Wang
Yansen Wang
Zhen Fan
Zhen Fan
Your rating :
0

 

Tags
Comments
小科