In this work, we introduce the NavProc 1.0 Corpus - a medium-scale, annotated corpus of procedural texts within the naval domain - for use as a first step in modeling procedural structures derived from real-world data sources. In particular, we have rigorously produced annotations of frame semantics (i.e., PropBank-inspired trigger/role links) across verbal, nominal, and adjectival frames. Furthermore, we have annotated 21 distinct types of semantic markers and structural links between textual elements (e.g., frame triggers, entities, modifiers) which, taken together, result in a text-focused graph of semantic elements. Such a graph can be used to derive a more complex procedure structure for use in personnel training, simulation, or collaborative procedure execution. Altogether, this annotation effort has encompassed 158 procedural units composed of 2,316 sentences, 44,459 tokens, and 48,137 distinct span annotations. Furthermore, we describe and report LLM-based extraction scores for use as a baseline in future research using this dataset.

In this work, we introduce the NavProc 1.0 Corpus – a medium-scale, annotated corpus of procedural texts within the naval domain – for use as a first step in modeling procedural structures derived from real-world data sources. In particular, we have rigorously produced annotations of frame semantics (i.e., PropBank-inspired trigger/role links) across verbal, nominal, and adjectival frames. Furthermore, we have annotated 21 distinct types of semantic markers and structural links between textual elements (e.g., frame triggers, entities, modifiers) which, taken together, result in a text-focused graph of semantic elements. Such a graph can be used to derive a more complex procedure structure for use in personnel training, simulation, or collaborative procedure execution. Altogether, this annotation effort has encompassed 158 procedural units composed of 2,316 sentences, 44,459 tokens, and 48,137 distinct span annotations. Furthermore, we describe and report LLM-based extraction scores for use as a baseline in future research using this dataset.

This corpus and its creation is described in the following publication:Jonathan Gordon, Jerry R. Hobbs, Jonathan May, Michael Mohler, Fabrizio Morbini, Bryan Rink, Marc Tomlinson, and Suzanne Wertheim. 2015. “A Corpus of Rich Metaphor Annotation”. In Proceedings of the Third Workshop on Metaphor in NLP.When linking to this data, please use the URL http://purl.org/net/metaphor-corpus, which will point to its current location.Data Attribution RequirementPlease insert the following statement in any product, report, publication, presentation and/or document that includes or references the data:This effort contains or makes use of the the IARPA-funded Metaphor Program USC/ISI annotated metaphorical language collection, release iarpa_metaphor_isi.edu_metaphor_corpus_2015-04-03.

In this work, we present the Language Computer Corporation (LCC) annotated metaphor datasets, which represent the largest and most comprehensive resource for metaphor research to date. These datasets were produced over the course of three years by a staff of nine annotators working in four languages (English, Spanish, Russian, and Farsi). As part of these datasets, we provide (1) metaphoricity ratings for within-sentence word pairs on a four-point scale, (2) scored links to our repository of 114 source concept domains and 32 target concept domains, and (3) ratings for the affective polarity and intensity of each pair. Altogether, we provide 188,741 annotations in English (for 80,100 pairs), 159,915 annotations in Spanish (for 63,188 pairs), 99,740 annotations in Russian (for 44,632 pairs), and 137,186 annotations in Farsi (for 57,239 pairs). In addition, we are providing a large set of likely metaphors which have been independently extracted by our two state-of-the-art metaphor detection systems but which have not been analyzed by our team of annotators.

Metaphor is a central phenomenon of language, and thus a central problem for natural language understanding.Previous work on the analysis of metaphors has identified which target concepts are being thought of and described in terms of which source concepts, but this is not adequate to explain what motivates the use of particular metaphors.This work proposes the use of conceptual schemas to represent the underspecified scenarios that motivate a metaphoric mapping.To support the creation of systems that can understand metaphors in this way, we have created and are publicly releasing a corpus of manually validated metaphor annotations.

In this work, we describe a supervised cross-lingual methodology for detecting novel and conventionalized metaphors that derives generalized semantic patterns from a collection of metaphor annotations. For this purpose, we model each metaphor annotation as an abstract tuple – (source, target, relation, metaphoricity) – that packages a metaphoricity judgement with a relational grounding of the source and target lexical units in text. From these annotations, we derive a set of semantic patterns using a three-step process. First, we employ several generalized representations of the target using a variety of WordNet information and representative domain terms. Then, we generalize relations using a rule-based, pseudo-semantic role labeling. Finally, we generalize the source by partitioning a semantic hierarchy (defined by the target and the relation) into metaphoric and non-metaphoric regions so as to optimally account for the evidence in the annotated data. Experiments show that by varying the generality of the source, target

In this work, we present two complementary methods for the expansion of psycholinguistics norms. The first method is a random-traversal spreading activation approach which transfers existing norms onto semantically related terms using notions of synonymy, hypernymy, and pertainymy to approach full coverage of the English language. The second method makes use of recent advances in distributional similarity representation to transfer existing norms to their closest neighbors in a high-dimensional vector space. These two methods (along with a naive hybrid approach combining the two) have been shown to significantly outperform a state-of-the-art resource expansion system at our pilot task of imageability expansion. We have evaluated these systems in a cross-validation experiment using 8,188 norms found in existing pscholinguistics literature. We have also validated the quality of these combined norms by performing a small study using Amazon Mechanical Turk (AMT).

We present a tiered-approach to the recognition of metaphor. The first tier is made up of highly precise expert-driven lexico-syntactic patterns which are automatically expended on in the second tier using lexical and dependency transformations. The final tier utilizes an SVM classifier using a variety of syntactic, semantic, and psycholinguistic features to determine if an expression is metaphoric. We focus on the recognition of metaphors in which the target is associated with the concept of "Economic Inequality" and examine the effectiveness of our approach for metaphors expressed in English, Farsi, Russian, and Spanish. Through experimental analysis we show that the proposed approach is capable of achieving 67.4% to 77.8% F-Measure depending on the language.

We present a novel approach to the problem of multilingual conceptual metaphor recognition. Our approach extends recent work in conceptual metaphor discovery by combining a complex methodology for facet-based concept induction with a distributional vector space model of linguistic and conceptual metaphor. In the evaluation of our system in English, Spanish, Russian, and Farsi, we experiment with several state-of-the-art vector space models and demonstrate a clear benefit to the fine-grained concept representation that forms the basis of our methodology for conceptual metaphor recognition.

Metaphor is a pervasive feature of human language that enables us to conceptualize and communicate abstract concepts using more concrete terminology. Unfortunately, computational models of natural language understanding - including systems for question answering, textual entailment, lexical substitution, and word-sense disambiguation - are unable to appropriately grasp the semantic content of metaphor and other forms of figurative language. In particular, we address the problem of understanding metaphoric language in the context of entailment (or paraphrase) detection. We build upon our existing state-of-the-art textual entailment system to specifically address issues of lexical entailment within a metaphoric context and have performed an in-depth experimental analysis to determine which techniques are most effective at interpreting metaphorical text. Our results suggest that a machine learning system trained on metaphor-rich data can achieve an accuracy above 90% for verbal metaphors using a combination of lexical, semantic, and contextual measures of term similarity.

ii Introduction Characteristic to all areas of human activity (from poetic to ordinary to scientific) and, thus, to all types of discourse, metaphor becomes an important problem for natural language processing. Its ubiquity in language has been established in a number of corpus studies and the role it plays in human reasoning has been confirmed in psychological experiments. This makes metaphor an important research area for computational and cognitive linguistics, and its automatic identification and interpretation indispensable for any semantics-oriented NLP application. The work on metaphor in NLP and AI started in the 1980s, providing us with a wealth of ideas on the structure and mechanisms of the phenomenon. The last decade witnessed a technological leap in natural language computation, whereby manually crafted rules gradually give way to more robust corpus-based statistical methods. This is also the case for metaphor research. In the recent years, the problem of metaphor modeling has been steadily gaining interest within the NLP community, with a growing number of approaches exploiting statistical techniques. Compared to more traditional approaches based on hand-coded knowledge, these more recent methods tend to have a wider coverage, as well as be more efficient, accurate and robust. However, even the statistical metaphor processing approaches so far often focused on a limited domain or a subset of phenomena. At the same time, recent work on computational lexical semantics and lexical acquisition techniques, as well as a wide range of NLP methods applying machine learning to open-domain semantic tasks, open many new avenues for creation of large-scale robust tools for recognition and interpretation of metaphor. This workshop is the first one focused on modelling of metaphor using NLP techniques. Recent related events include workshops on Computational Approaches to Figurative Language (NAACL 2007) and on Computational Approaches to Linguistic Creativity (NAACL 2009, NAACL 2010). We received 14 submissions and accepted 10. Each paper was carefully reviewed by at least 3 members of the Program Committee. The selected papers offer explorations into the following directions: (1) creation of metaphor-annotated datasets; (2) identification of new features that are useful for metaphor identification; (3) cross-lingual metaphor identification. The papers represent a variety of approaches to utilization and creation of datasets. While existing annotated corpora were used in some papers (Dunn, Tsvetkov et al), most papers describe creation of new annotated materials. Along with annotation guidelines adapted from the MIP and MIPVU procedures (Badryzlova et al), more intuitive …

We present a method of constructing the semantic signatures of target concepts expressed in metaphoric expressions as well as a method to determine the conceptual space of a metaphor using the constructed semantic signatures and a semantic expansion. We evaluate our methodology by focusing on metaphors where the target concept is Governance. Using the semantic signature constructed for this concept, we show that the conceptual spaces generated by our method are judged to be highly acceptable by humans.

This article provides a detailed overview of the CPN text-to-text similarity system that we participated with in the Semantic Textual Similarity task evaluations hosted at *SEM 2013. In addition to more traditional components, such as knowledge-based and corpus-based metrics leveraged in a machine learning framework, we also use opinion analysis features to achieve a stronger semantic representation of textual units. While the evaluation datasets are not designed to test the similarity of opinions, as a component of textual similarity, nonetheless, our system variations ranked number 38, 39 and 45 among the 88 participating systems.

In this dissertation, I explore unsupervised techniques for the task of automatic short answer grading. I compare a number of knowledge-based and corpus-based measures of text similarity, evaluate the effect of domain and size on the corpus-based measures, and also introduce a novel technique to improve the performance of the system by integrating automatic feedback from the student answers. I continue to combine graph alignment features with lexical semantic similarity measures and employ machine learning techniques to show that grade assignment error can be reduced compared to a system that considers only lexical semantic measures of similarity. I also detail a preliminary attempt to align the dependency graphs of student and instructor answers in order to utilize a structural component that is necessary to simulate human-level grading of student answers. I further explore the utility of these techniques to several related tasks in natural language processing including the detection of text similarity, paraphrase, and textual entailment.

This paper presents the systems that we participated with in the Semantic Text Similarity task at SEMEVAL 2012. Based on prior research in semantic similarity and relatedness, we combine various methods in a machine learning framework. The three variations submitted during the task evaluation period ranked number 5, 9 and 14 among the 89 participating systems. Our evaluations show that corpus-based methods display a more robust behavior on the training data, yet combining a variety of methods allows a learning algorithm to achieve a superior decision than that achievable by any of the individual parts.

In this work we address the task of computerassisted assessment of short student answers. We combine several graph alignment features with lexical semantic similarity measures using machine learning techniques and show that the student answers can be more accurately graded than if the semantic measures were used in isolation. We also present a first attempt to align the dependency graphs of the student and the instructor answers in order to make use of a structural component in the automatic grading of student answers.

In this paper, we explore unsupervised techniques for the task of automatic short answer grading. We compare a number of knowledge-based and corpus-based measures of text similarity, evaluate the effect of domain and size on the corpus-based measures, and also introduce a novel technique to improve the performance of the system by integrating automatic feedback from the student answers. Overall, our system significantly and consistently outperforms other unsupervised methods for short answer grading that have been proposed in the past.