In this paper, we report our discovery on named entity distribution in a general word embedding space, which helps an open definition on multilingual named entity definition rather than previous closed and constraint definition on named entities through a named entity dictionary, which is usually derived from human labor and replies on schedule update. Our initial visualization of monolingual word embeddings indicates named entities tend to gather together despite of named entity types and language difference, which enable us to model all named entities using a specific geometric structure inside embedding space, namely, the named entity hypersphere. For monolingual cases, the proposed named entity model gives an open description of diverse named entity types and different languages. For cross-lingual cases, mapping the proposed named entity model provides a novel way to build a named entity dataset for resource-poor languages. At last, the proposed named entity model may be shown as a handy clue to enhance state-of-the-art named entity recognition systems generally.

Fine-tuning the pre-trained language mod-001 els (PLMs) on downstream tasks is the de-002 facto paradigm in NLP. Despite the superior 003 performance on sequence labeling, the fine-004 tuning requires large-scale parameters and time-005 consuming deployment for each task, which 006 limits its application in real-world scenarios. 007 To alleviate these problems, we propose a 008 pluggable sequence labeling framework, plug-009 tagger. By switching the task-specific plugin 010 on the input, plug-tagger allows a frozen PLM 011 to perform different sequence labeling tasks 012 without redeployment. Specifically, the plugin 013 on the input are a few continuous vectors, 014 which manipulates the PLM without modifying 015 its parameters, and each task only needs to store 016 the lightweight vectors rather than a full copy 017 of PLM. To avoid redeployment, we propose 018 the label word mechanism, which reuses the 019 language model head to prevent task-specific 020 classifiers from modifying model structures. 021 Experimental results on three sequence la-022 beling tasks show that the proposed method 023 achieves comparable performance with fine-024 tuning by using 0.1% task-specific parameters. 025 Experiments show that our method is faster 026 than other lightweight methods under limited 027 computational resources 028

Deep neural network models have helped named entity recognition achieve amazing performance without handcrafting features. However, existing systems require large amounts of human annotated training data. Efforts have been made to replace human annotations with external knowledge (e.g., NE dictionary, part-of-speech tags), while it is another challenge to obtain such effective resources. In this work, we propose a fully unsupervised NE recognition model which only needs to take informative clues from pre-trained word embeddings. We first apply Gaussian Hidden Markov Model and Deep Autoencoding Gaussian Mixture Model on word embeddings for entity span detection and type prediction, and then further design an instance selector based on reinforcement learning to distinguish positive sentences from noisy sentences and then refine these coarse-grained annotations through neural networks. Extensive experiments on two CoNLL benchmark NER datasets (CoNLL-2003 English dataset and CoNLL-2002 Spanish dataset) demonstrate that our proposed light NE recognition model achieves remarkable performance without using any annotated lexicon or corpus.

Named entity recognition (NER) models are typically based on the architecture of Bi-directional LSTM (BiLSTM). The constraints of sequential nature and the modeling of single input prevent the full utilization of global information from larger scope, not only in the entire sentence, but also in the entire document (dataset). In this paper, we address these two deficiencies and propose a model augmented with hierarchical contextualized representation: sentence-level representation and document-level representation. In sentence-level, we take different contributions of words in a single sentence into consideration to enhance the sentence representation learned from an independent BiLSTM via label embedding attention mechanism. In document-level, the key-value memory network is adopted to record the document-aware information for each unique word which is sensitive to similarity of context information. Our two-level hierarchical contextualized representations are fused with each input token embedding and corresponding hidden state of BiLSTM, respectively. The experimental results on three benchmark NER datasets (CoNLL-2003 and Ontonotes 5.0 English datasets, CoNLL-2002 Spanish dataset) show that we establish new state-of-the-art results.

In this paper, we propose a novel bipartite flat-graph network (BiFlaG) for nested named entity recognition (NER), which contains two subgraph modules: a flat NER module for outermost entities and a graph module for all the entities located in inner layers. Bidirectional LSTM (BiLSTM) and graph convolutional network (GCN) are adopted to jointly learn flat entities and their inner dependencies. Different from previous models, which only consider the unidirectional delivery of information from innermost layers to outer ones (or outside-to-inside), our model effectively captures the bidirectional interaction between them. We first use the entities recognized by the flat NER module to construct an entity graph, which is fed to the next graph module. The richer representation learned from graph module carries the dependencies of inner entities and can be exploited to improve outermost entity predictions. Experimental results on three standard nested NER datasets demonstrate that our BiFlaG outperforms previous state-of-the-art models.