Including children with autism spectrum disorders (ASD) in mainstream environments creates a need for new interventions whose efficacy must be assessed in situ. This article presents a tablet-based application for activity schedules that has been designed following a participatory design approach involving mainstream teachers, special education teachers, and school aides. This application addresses two domains of activities: classroom routines and verbal communications. We assessed the efficiency of our application with two overlapping user studies in mainstream inclusion, sharing a group of children with ASD. The first experiment involved 10 children with ASD, where five children were equipped with our tabled-based application and five were not equipped. We show that (1) the use of the application is rapidly self-initiated (after 2 months for almost all the participants) and (2) the tablet-supported routines are better performed after 3 months of intervention. The second experiment involved 10 children equipped with our application; it shared the data collected for the five children with ASD and compared them with data collected for five children with intellectual disability (ID). We show that (1) children with ID are not autonomous in the use of the application at the end of the intervention, (2) both groups exhibited the same benefits on classroom routines, and (3) children with ID improve significantly less their performance on verbal communication routines. These results are discussed in relation with our design principles. Importantly, the inclusion of a group with another neurodevelopmental condition provided insights about the applicability of these principles beyond the target population of children with ASD.
We study the problem of semantic interpretation of noun compounds such as bee honey , malaria mosquito , apple cake , and stem cell . In particular, we explore the potential of using predicates that make explicit the hidden relation that holds between the nouns that form the noun compound. For example, mosquito that carries malaria is a paraphrase of the compound malaria mosquito in which the verb explicitly states the semantic relation between the two nouns. We study the utility of using such paraphrasing verbs, with associated weights, to build a representation of the semantics of a noun compound, for example, malaria mosquito can be represented as follows: carry (23), spread (16), cause (12), transmit (9), and so on. We also explore the potential of using multiple paraphrasing verbs as features for predicting abstract semantic relations such as CAUSE, and we demonstrate that using explicit paraphrases can help improve statistical machine translation.
What is a multiword expression (MWE) and how many are there? Mark Liberman gave a great invited talk at ACL-89, titled “How Many Words Do People Know?” where he spent the entire hour questioning the question. Many of the same questions apply to multiword expressions. What is a word? An expression? What is many? What is a person? What does it mean to know? Rather than answer these questions, this article will use them as Liberman did, as an excuse for surveying how such issues are addressed in a variety of fields: computer science, Web search, linguistics, lexicography, educational testing, psychology, statistics, and so on.
Natural language search relies strongly on perceiving semantics in a query sentence. Semantics is captured by the relationship among the query words, represented as a network (graph). Such a network of words can be fed into larger ontologies, like DBpedia or Google Knowledge Graph, where they appear as subgraphs— fashioning the name subnetworks (subnets). Thus, subnet is a canonical form for interfacing a natural language query to a graph database and is an integral step for graph-based searching. In this article, we present a novel standalone NLP technique that leverages the cognitive psychology notion of semantic strata for semantic subnetwork extraction from natural language queries. The cognitive model describes some of the fundamental structures employed by the human cognition to construct semantic information in the brain, called semantic strata. We propose a computational model based on conditional random fields to capture the cognitive abstraction provided by semantic strata, facilitating cognitive canonicalization of the query. Our results, conducted on approximately 5000 queries, suggest that the cognitive canonicals based on semantic strata are capable of significantly improving parsing and role labeling performance beyond pure lexical approaches, such as parts-of-speech based techniques. We also find that cognitive canonicalized subnets are more semantically coherent compared to syntax trees when explored in graph ontologies like DBpedia and improve ranking of retrieved documents.
The use of figurative language is ubiquitous in natural language texts and it is a serious bottleneck in automatic text understanding. A system capable of interpreting figurative expressions would be an invaluable addition to the real-world natural language processing (NLP) applications that need to access semantics, such as machine translation, opinion mining, question answering and many others. In this article we focus on one type of figurative language, logical metonymy, and present a computational model of its interpretation bringing together statistical techniques and the insights from linguistic theory. Compared to previous approaches this model is both more informative and more accurate. The system produces sense-level interpretations of metonymic phrases and then automatically organizes them into conceptual classes, or roles, discussed in the majority of linguistic literature on the phenomenon.
This research presents a new benchmark dataset for evaluating Short Text Semantic Similarity (STSS) measurement algorithms and the methodology used for its creation. The power of the dataset is evaluated by using it to compare two established algorithms, STASIS and Latent Semantic Analysis. This dataset focuses on measures for use in Conversational Agents; other potential applications include email processing and data mining of social networks. Such applications involve integrating the STSS algorithm in a complex system, but STSS algorithms must be evaluated in their own right and compared with others for their effectiveness before systems integration. Semantic similarity is an artifact of human perception; therefore its evaluation is inherently empirical and requires benchmark datasets derived from human similarity ratings. The new dataset of 64 sentence pairs, STSS-131, has been designed to meet these requirements drawing on a range of resources from traditional grammar to cognitive neuroscience. The human ratings are obtained from a set of trials using new and improved experimental methods, with validated measures and statistics. The results illustrate the increased challenge and the potential longevity of the STSS-131 dataset as the Gold Standard for future STSS algorithm evaluation.
The search space of Phrase-Based Statistical Machine Translation (PBSMT) systems can be represented as a directed acyclic graph (lattice). By exploring this search space, it is possible to analyze and understand the failures of PBSMT systems. Indeed, useful diagnoses can be obtained by computing the so-called oracle hypotheses, which are hypotheses in the search space that have the highest quality score. For standard SMT metrics, this problem is, however, NP-hard and can only be solved approximately. In this work, we present two new methods for efficiently computing oracles on lattices: the first one is based on a linear approximation of the corpus bleu score and is solved using generic shortest distance algorithms; the second one relies on an Integer Linear Programming (ILP) formulation of the oracle decoding that incorporates count clipping constraints. It can either be solved directly using a standard ILP solver or using Lagrangian relaxation techniques. These new decoders are evaluated and compared with several alternatives from the literature for three language pairs, using lattices produced by two PBSMT systems.
Extracting semantic relations from text is a preliminary step towards understanding the meaning of text. The more semantic relations are extracted from a sentence, the better the representation of the knowledge encoded into that sentence. This article introduces a framework for the Composition of Semantic Relations (CSR). CSR aims to reveal more text semantics than existing semantic parsers by composing new relations out of previously extracted relations. Semantic relations are defined using vectors of semantic primitives, and an algebra is suggested to manipulate these vectors according to a CSR algorithm. Inference axioms that combine two relations and yield another relation are generated automatically. CSR is a language-agnostic, inventory-independent method to extract semantic relations. The formalism has been applied to a set of 26 well-known relations and results are reported.
In this article, we investigate word sense distributions in noun compounds (NCs). Our primary goal is to disambiguate the word sense of component words in NCs, based on investigation of “semantic collocation” between them. We use sense collocation and lexical substitution to build supervised and unsupervised word sense disambiguation (WSD) classifiers, and show our unsupervised learner to be superior to a benchmark WSD system. Further, we develop a word sense-based approach to interpreting the semantic relations in NCs.
Light verb constructions consist of a verbal and a nominal component, where the noun preserves its original meaning while the verb has lost it (to some degree). They are syntactically flexible and their meaning can only be partially computed on the basis of the meaning of their parts, thus they require special treatment in natural language processing. For this purpose, the first step is to identify light verb constructions. In this study, we present our conditional random fields-based tool—called FXTagger—for identifying light verb constructions. The flexibility of the tool is demonstrated on two, typologically different, languages, namely, English and Hungarian. As earlier studies labeled different linguistic phenomena as light verb constructions, we first present a linguistics-based classification of light verb constructions and then show that FXTagger is able to identify different classes of light verb constructions in both languages. Different types of texts may contain different types of light verb constructions; moreover, the frequency of light verb constructions may differ from domain to domain. Hence we focus on the portability of models trained on different corpora, and we also investigate the effect of simple domain adaptation techniques to reduce the gap between the domains. Our results show that in spite of domain specificities, out-domain data can also contribute to the successful LVC detection in all domains.
We investigate the impact of preextracting and tokenizing bigram collocations on topic models. Using extensive experiments on four different corpora, we show that incorporating bigram collocations in the document representation creates more parsimonious models and improves topic coherence. We point out some problems in interpreting test likelihood and test perplexity to compare model fit, and suggest an alternate measure that penalizes model complexity. We show how the Akaike information criterion is a more appropriate measure, which suggests that using a modest number (up to 1000) of top-ranked bigrams is the optimal topic modelling configuration. Using these 1000 bigrams also results in improved topic quality over unigram tokenization. Further increases in topic quality can be achieved by using up to 10,000 bigrams, but this is at the cost of a more complex model. We also show that multiword (bigram and longer) named entities give consistent results, indicating that they should be represented as single tokens. This is the first work to explicitly study the effect of n -gram tokenization on LDA topic models, and the first work to make empirical recommendations to topic modelling practitioners, challenging the standard practice of unigram-based tokenization.
The TAC 2010 summarization track initiated a new task—aspect-guided summarization—that centers on textual aspects embodied as particular kinds of information of a text. We observe that aspect-guided summaries not only address highly specific user need, but also facilitate content-level coherence by using aspect information. In this article, we present a full-fledged approach to aspect-guided summarization with a focus on summary coherence. Our summarization approach depends on two prerequisite subtasks: recognizing aspect-bearing sentences in order to do sentence extraction, and modeling aspect-based coherence with an HMM model in order to predict a coherent sentence ordering. Using the manually annotated TAC 2010 and 2010 datasets, we validated the effectiveness of our proposed methods for those subtasks. Drawing on the empirical results, we proceed to develop an aspect-guided summarizer based on a simple but robust base summarizer. With sentence selection guided by aspect information, our system is one of the best on TAC 2011. With sentence ordering predicted by the aspect-based HMM model, the summaries achieve good coherence.
With the growing interest in statistical parsing, special attention has recently been devoted to the problem of comparing different treebanks to assess which languages or domains are more difficult to parse relative to a given model. A common methodology for comparing parsing difficulty across treebanks is based on the use of the standard labeled precision and recall measures. As an alternative, in this article we propose an information-theoretic measure, called the expected conditional cross-entropy (ECC). One important advantage with respect to standard performance measures is that ECC can be directly expressed as a function of the parameters of the model. We evaluate ECC across several treebanks for English, French, German, and Italian, and show that ECC is an effective measure of parsing difficulty, with an increase in ECC always accompanied by a degradation in parsing accuracy.
We explore the properties of support-verb and nominalization (SVN) pairs in English, a type of multiword expression in which a semantically impoverished verb combines with a complement nominalization sharing an unexpressed role with the verb. This study follows others in seeking syntactic or lexical semantic factors correlated with the acceptability of these constructions. In particular, following recent work showing certain semantic verb class features to improve SVN classification [Tu and Roth 2011], we explore the possibility that support verbs and the verbal roots of nominalizations in acceptable SVN pairs are clustered according to the classes of Levin [1993]. We compare the compatibility correlation of these results with those of the Aktionsart-class-based proposal of Barrett and Davis [2002]. We find the evidence that Levin classes are a factor in the acceptability of SVN constructions to be equivocal, and conclude with a discussion of the reasons for this finding.
Affect interpretation from open-ended drama improvisation is a challenging task. This article describes experiments in using latent semantic analysis to identify discussion themes and potential target audiences for those improvisational inputs without strong affect indicators. A context-based affect-detection is also implemented using a supervised neural network with the consideration of emotional contexts of most intended audiences, sentence types, and interpersonal relationships. In order to go beyond the constraints of predefined scenarios and improve the system's robustness, min-margin-based active learning is implemented. This active learning algorithm also shows great potential in dealing with imbalanced affect classifications. Evaluation results indicated that the context-based affect detection achieved an averaged precision of 0.826 and an averaged recall of 0.813 for affect detection of the test inputs from the Crohn's disease scenario using three emotion labels: positive, negative, and neutral, and an averaged precision of 0.868 and an average recall of 0.876 for the test inputs from the school bullying scenario. Moreover, experimental evaluation on a benchmark data set for active learning demonstrated that active learning was able to greatly reduce human annotation efforts for the training of affect detection, and also showed promising robustness in dealing with open-ended example inputs beyond the improvisation of the chosen scenarios.
The issue of internal variability of multiword expressions (MWEs) is crucial towards their identification and extraction in running text. We present a corpus-supported and computational study on Italian MWEs, aimed at defining an automatic method for modeling internal variation, exploiting frequency and part-of-speech (POS) information. We do so by deriving an XML-encoded lexicon of MWEs based on a manually compiled dictionary, which is then projected onto a a large corpus. Since a search for fixed forms suffers from low recall, while an unconstrained flexible search for lemmas yields a loss in precision, we suggest a procedure aimed at maximizing precision in the identification of MWEs within a flexible search. Our method builds on the idea that internal variability can be modelled via the novel introduction ofvariation patterns, which work over POS patterns, and can be used as working tools for controlling precision. We also compare the performance of variation patterns to that of association measures, and explore the possibility of using variation patterns in MWE extraction in addition to identification. Finally, we suggest that corpus-derived, pattern-related information can be included in the original MWE lexicon by means of an enriched coding and the creation of an XML-based repository of patterns.
We are in 2013, and multiword expressions have been around for a while in the computational linguistics research community. Since the first ACL workshop on MWEs 12 years ago in Sapporo, Japan, much has been discussed, proposed, experimented, evaluated and argued about MWEs. And yet, they deserve the publication of a whole special issue of the ACM Transactions on Speech and Language Processing . But what is it about multiword expressions that keeps them in fashion? Who are the people and the institutions who perform and publish groundbreaking fundamental and applied research in this field? What is the place and the relevance of our lively research community in the bigger picture of computational linguistics? Where do we come from as a community, and most importantly, where are we heading? In this introductory article, we share our point of view about the answers to these questions and introduce the articles that compose the current special issue.
Our aim is to build listening agents that attentively listen to their users and satisfy their desire to speak and have themselves heard. This article investigates how to automatically create a dialogue control component of such a listening agent. We collected a large number of listening-oriented dialogues with their user satisfaction ratings and used them to create a dialogue control component that satisfies users by means of Partially Observable Markov Decision Processes (POMDPs). Using a hybrid dialog controller where high-level dialog acts are chosen with a statistical policy and low-level slot values are populated by a wizard, we evaluated our dialogue control method in a Wizard-of-Oz experiment. The experimental results show that our POMDP-based method achieves significantly higher user satisfaction than other stochastic models, confirming the validity of our approach. This article is the first to verify, by using human users, the usefulness of POMDP-based dialogue control for improving user satisfaction in nontask-oriented dialogue systems.
Information disparity is a major challenge with multilingual document collections. When documents are dynamically updated in a distributed fashion, information content among different language editions may gradually diverge. We propose a framework for assisting human editors to manage this information disparity, using tools from machine translation and machine learning. Given source and target documents in two different languages, our system automatically identifies information nuggets that are new with respect to the target and suggests positions to place their translations. We perform both real-world experiments and large-scale simulations on Wikipedia documents and conclude our system is effective in a variety of scenarios.
The integration of compounds in a parsing procedure has been shown to improve accuracy in an artificial context where such expressions have been perfectly preidentified. This article evaluates two empirical strategies to incorporate such multiword units in a real PCFG-LA parsing context: (1) the use of a grammar including compound recognition, thanks to specialized annotation schemes for compounds; (2) the use of a state-of-the-art discriminative compound prerecognizer integrating endogenous and exogenous features. We show how these two strategies can be combined with word lattices representing possible lexical analyses generated by the recognizer. The proposed systems display significant gains in terms of multiword recognition and often in terms of standard parsing accuracy. Moreover, we show through an Oracle analysis that this combined strategy opens promising new research directions.
