Early programming methodologies proved inadequate for the reliable implementation of the large programs enabled by rapidly growing computer memories, resulting in prohibitively long times for software development. Attempts to solve this problem by limiting control structure primitives within programming languages or by imposing external design methodologies to enforce ill-defined notions of modularity proved unsuccessful. Barbara Liskov's definition of abstract data types in 1973 and her embodiment of this notion in the programming language CLU provided a major step toward solving this problem. The key idea was to define the fundamental building blocks of programs as a set of abstractions, each of which resulted from the combination of a data type with all the operations that needed to be performed on the data type. Data abstraction is now fundamental to all modern programming languages. A second key innovation of Liskov's was what she called “behavioral subtyping” for object-oriented programming, now known as the Liskov Substitution Principle (LSP). The LSP formally defines when objects of subclasses can be safely substituted for the parent class, a crucial requirement of bug-free programming within this programming paradigm. In addition, Liskov has made several key contributions to distributed computer systems, including the first use of atomic actions for successful concurrency and providing a practical solution to the problem of Byzantine faults.

Fine-tuning the pre-trained language mod-001 els (PLMs) on downstream tasks is the de-002 facto paradigm in NLP. Despite the superior 003 performance on sequence labeling, the fine-004 tuning requires large-scale parameters and time-005 consuming deployment for each task, which 006 limits its application in real-world scenarios. 007 To alleviate these problems, we propose a 008 pluggable sequence labeling framework, plug-009 tagger. By switching the task-specific plugin 010 on the input, plug-tagger allows a frozen PLM 011 to perform different sequence labeling tasks 012 without redeployment. Specifically, the plugin 013 on the input are a few continuous vectors, 014 which manipulates the PLM without modifying 015 its parameters, and each task only needs to store 016 the lightweight vectors rather than a full copy 017 of PLM. To avoid redeployment, we propose 018 the label word mechanism, which reuses the 019 language model head to prevent task-specific 020 classifiers from modifying model structures. 021 Experimental results on three sequence la-022 beling tasks show that the proposed method 023 achieves comparable performance with fine-024 tuning by using 0.1% task-specific parameters. 025 Experiments show that our method is faster 026 than other lightweight methods under limited 027 computational resources 028

Author(s): Gabbard, Ryan; Lichtefeld, Jacob A; Beser, Deniz; Cecil, Joe; Marcus, Mitch; Payne, Sarah R B; Yang, Charles; Freedman, Marjorie | Abstract: Word learning models are typically evaluated as the problem of observing words together with sets of atomic objects and learn-ing an alignment between them. We use ADAM, a Python software platform for modeling grounded language acquisition, to evaluate a particular word learning model, Pursuit (Stevens, Gleitman, Trueswell, a Yang, 2017),under more realistic learning conditions (see e.g. Gleitman and Trueswell (2020) for review). In particular, we manipulate the degree of referential ambiguity and the salience of attentional cues available to the learner, and we present extensions to Pursuit which address the challenges of non-atomic meanings and exploiting attentional cues.

We introduce and implement a cognitively plausible model for learning from generic language, statements that express generalizations about members of a category and are an important aspect of concept development in language acquisition (Carlson&Pelletier, 1995; Gelman, 2009). We extend a computational framework designed to model grounded language acquisition by introducing the concept network. This new layer of abstraction enables the system to encode knowledge learned from generic statements and represent the associations between concepts learned by the system. Through three tasks that utilize the concept network, we demonstrate that our extensions to ADAM can acquire generic information and provide an example of how ADAM can be used to model language acquisition.

Previous studies have made great advances in 001 RST discourse parsing through neural frame-002 works or efficient features, but they split the 003 parsing process into two subtasks and heavily 004 depended on gold segmentation. In this pa-005 per, we introduce an end-to-end method for 006 sentence-level RST discourse parsing via trans-007 forming it into a text-to-text generation task. 008 Our method unifies the traditional two-stage 009 parsing and generates the parsing tree directly 010 from the input text without requiring a compli-011 cated model. Moreover, the EDU segmentation 012 can be simultaneously generated and extracted 013 from the parsing tree. Experimental results 014 on the RST Discourse Treebank demonstrate 015 that our proposed method outperforms existing 016 methods in both tasks of sentence-level RST 017 parsing and discourse segmentation. Consid-018 ering the lack of annotated data in RST pars-019 ing, we also create high-quality augmented data 020 based on several filtering strategies, which fur-021 ther improves the performance. 022

As Abstract Meaning Representation (AMR) 001 implicitly involves compound semantic anno-002 tations, we hypothesize auxiliary tasks which 003 are semantically or formally related can bet-004 ter enhance AMR parsing. With carefully 005 designed control experiments, we find that 006 1) Semantic role labeling (SRL) and depen-007 dency parsing (DP), would bring much more 008 significant performance gain than unrelated 009 tasks in the text-to-AMR transition. 2) To 010 make a better fit for AMR, data from auxil-011 iary tasks should be properly “AMRized” to 012 PseudoAMR before training. 3) Intermediate-013 task training paradigm outperforms multitask 014 learning when introducing auxiliary tasks to 015 AMR parsing. From an empirical perspective, 016 we propose a principled method to choose, re-017 form, and train auxiliary tasks to boost AMR 018 parsing. Extensive experiments show that our 019 method achieves new state-of-the-art perfor-020 mance on in-distribution, out-of-distribution, 021 and few-shots benchmarks of AMR parsing. 022

Dependency parsing algorithms capable of producing the types of crossing dependencies seen in natural language sentences have traditionally been orders of magnitude slower than algorithms for projective trees. For 95.8–99.8% of dependency parses in various natural language treebanks, whenever an edge is crossed, the edges that cross it all have a common vertex. The optimal dependency tree that satisfies this 1-Endpoint-Crossing property can be found with an O( n 4 ) parsing algorithm that recursively combines forests over intervals with one exterior point. 1-Endpoint-Crossing trees also have natural connections to linguistics and another class of graphs that has been studied in NLP.

By the early 1970s, work by Noam Chomsky (Benjamin Franklin Medal, 1999) and his colleagues provided strong evidence for a formal characterization of the syntax of human languages. While most linguists believed that a similar characterization of the semantics of language was possible, there was little consensus about its form. Two conflicting views had emerged; to quote a contemporaneous paper by Partee, "One approach, generative semantics, was founded in part on the conviction that semantic and syntactic rules could not be separated in any principled way.... The other, interpretive semantics, maintains the distinction between syntactic rules as formation rules and semantic rules as interpretive rules, but does not posit any systematic relation between them." In parallel, a third approach, a novel formal account of semantics based on strict compositionality and model theory had emerged in the work of the logician Richard Montague, even though it violated commonsense notions of meaning in key instances and provided only the weakest mechanisms to handle purely syntactic phenomena. In this context, Barbara Partee first interpreted Montague's ideas to the community of linguists, finding a way to get linguists past its considerable mathematical difficulty, and then fused Montague's work with interpretive semantics to propose a specific architecture for the syntax-semantics system that has supported accounts of a remarkably wide range of semantic phenomena. In the past 50 years, Partee's approach has become the dominant account of meaning within linguistics. Partee has also trained two generations of students in this new field of formal semantics, and these students collectively provide major intellectual leadership across this field today. The fundamental ideas of formal semantics as envisioned by Partee, particularly the notion of a compositional, declarative semantics running on the output of a deep syntactic analysis, have also played an important although indirect role in the development of natural language technology. Given the deep and continuing impact of Partee's work on the scientific study of language and on cognitive science, we hereby recommend to the Committee on Science and the Arts of The Franklin Institute that the Benjamin Franklin Medal in Computer and Cognitive Science be awarded to Barbara Partee.

The biaffine model is a strong and efficient 001 model for graph-based dependency parsing. 002 However, previous work only used the biaffine 003 method in single-layer form. In this paper, we 004 propose a multi-layer biaffine model for neural 005 dependency parsing. In this model, we modify 006 the biaffine method so that it can be utilized in 007 multi-layer form. We evaluate our model on 008 PTB and CTB and show our model achieves 009 state-of-the-art results on both datasets. Further 010 experiments show the benefits of introducing 011 multi-layer form into the biaffine method with 012 low efficiency loss. 013

We present ADAM, a software system for designing and running child language learning experiments in Python. The system uses a virtual world to simulate a grounded language acquisition process in which the language learner utilizes cognitively plausible learning algorithms to form perceptual and linguistic representations of the observed world. The modular nature of ADAM makes it easy to design and test different language learning curricula as well as learning algorithms. In this report, we describe the architecture of the ADAM system in detail, and illustrate its components with examples. We provide our code.

................................................................................................................................................ ...1

From its very inception, all traffic on what we now know as the Internet has been conveyed using a suite of protocols collectively called Transmission Control Protocol/Internet Protocol (TCP/IP). Remarkably, these protocols, designed in 1974 by Vinton Cerf and Robert Kahn to allow individual computer networks to internetwork, have remained essentially unchanged since they were created. TCP/IP was originally adopted to provide the protocols of the newly emerging Internet in the early 1980s and has scaled without change to power the global Internet of today. The goal of TCP/IP was to create a single protocol that could power networks sitting on very different communications systems, from point-to-point computer connection, satellite-connected networks with very high latency, and mobile packet-switch networks with ever changing communications reliability. This effort succeeded remarkably.

This paper describes a language-independent model for fully unsupervised morphological analysis that exploits a universal framework leveraging morphological typology. By modeling morphological processes including suffixation, prefixation, infixation, and full and partial reduplication with constrained stem change rules, our system effectively constrains the search space and offers a wide coverage in terms of morphological typology. The system is tested on nine typologically and genetically diverse languages, and shows superior performance over leading systems. We also investigate the effect of an oracle that provides only a handful of bits per language to signal morphological type.

This paper describes a new morphology resource created by Linguistic Data Consortium and the University of Pennsylvania for the DARPA LORELEI Program. The data consists of approximately 2000 tokens annotated for morphological segmentation in each of 9 low resource languages, along with root information for 7 of the languages. The languages annotated show a broad diversity of typological features. A minimal annotation scheme for segmentation was developed such that it could capture the patterns of a wide range of languages and also be performed reliably by non-linguist annotators. The basic annotation guidelines were designed to be language-independent, but included language-specific morphological paradigms and other specifications. The resulting annotated corpus is designed to support and stimulate the development of unsupervised morphological segmenters and analyzers by providing a gold standard for their evaluation on a more typologically diverse set of languages than has previously been available. By providing root annotation, this corpus is also a step toward supporting research in identifying richer morphological structures than simple morpheme boundaries.

This paper describes an unsupervised model for morphological segmentation that exploits the notion of paradigms, which are sets of morphological categories (e.g., suffixes) that can be applied to a homogeneous set of words (e.g., nouns or verbs). Our algorithm identifies statistically reliable paradigms from the morphological segmentation result of a probabilistic model, and chooses reliable suffixes from them. The new suffixes can be fed back iteratively to improve the accuracy of the probabilistic model. Finally, the unreliable paradigms are subjected to pruning to eliminate unreliable morphological relations between words. The paradigm-based algorithm significantly improves segmentation accuracy. Our method achieves start-of-the-art results on experiments using the Morpho-Challenge data, including English, Turkish, and Finnish.

We develop a post-processing system to efficiently correct errors from noisy optical character recognition (OCR) in a 2.7 million word Faroese corpus. 7.6% of the words in the original OCR text contain an error; fully manual correction would take thousands of hours due to the size of the corpus. Instead, our post-processing method applied to the Faroese corpus is projected to reduce the word error rate to 1.3% with around 65 hours of human annotator work. The foundation for generating corrected text is an HMM that learns patterns of OCR error and decodes noisy OCR character sequences into hypothesised correct language. A dictionary augments the HMM by contributing additional language knowledge, and a human annotator provides judgements in a small subset of cases that are identified as otherwise most prone to inaccurate output. An interactive workstation facilitates quick and accurate input for annotation. The entire toolkit is written in Python and is being made available for use in other low-resource languages where standard OCR technology falls short of desirable text quality. Supplementary analyses explore the impact of variable language resource availability and annotator time limitations on the end quality achievable with our toolkit.

This paper describes a large multilingual richly annotated corpus which is being made available to the community. There is an emphasis on quality and consistency with interannotator agreement rates targeted at 90%. The data covers multiple genres in English, Chinese, and Arabic, including a significant amount of parallel data. The annotation, intended to capture a skeletal representation of literal meaning, includes parse trees, predicate argument structures , word senses localized in an ontology, coreference, and name types. The resource is delivered as an integrated database, supporting combined queries that access multiple annotation layers. Annual incremental releases are distributed via LDC. 1 Motivation, Goals, and Rationale Our goal is to provide data in multiple languages and multiple genres (newswire, broadcast news, broadcast conversation, and web text), richly annotated by a skeletal representation of the literal meaning of sentences, so that a new generation of language understanding would deliver new functional capability. Our inspiration has been the impact on research and on applications of two seminal annotation products: the UPenn Treebank for syntax (Marcus, et al., 1993) and PropBank for semantic role labeling (Palmer et al., 2005). As shown in Figure 1, to the baseline structure of parse trees and propositions, OntoNotes adds 1 This work was supported under the GALE program of the Defense Advanced Research Projects Agency, Contract No. HR0011-06-C-0022. • Referring expressions and the textual phrases they refer to • Terms disambiguated by word sense and localized in an ontology • Named Entities Based on our interpretation of the criteria for successfully applying learning algorithms, our guiding principle has been to find a “sweet spot” in the space of • Inter-tagger agreement, so that human agreement as a ceiling on algorithm performance is as high as possible. • Productivity, so that the amount of training data is maximized, given a budget, • Depth of representation, so that the added semantic features are as deep as possible. The methodology described here was tested prior to entering production mode, where pilot rounds of annotation were conducted to find the sweet spot above. In particular, only those classes of co-reference satisfying the methodology above during the pilot study are annotated. The methodology has been applied for each word; the sense inventory for a word is selected according to the criteria above. Another dimension of the OntoNotes product is the integration of all of the annotations in a dataText Co-reference Word Sense wrt Ontology Treebank