Fine-tuning the pre-trained language mod-001 els (PLMs) on downstream tasks is the de-002 facto paradigm in NLP. Despite the superior 003 performance on sequence labeling, the fine-004 tuning requires large-scale parameters and time-005 consuming deployment for each task, which 006 limits its application in real-world scenarios. 007 To alleviate these problems, we propose a 008 pluggable sequence labeling framework, plug-009 tagger. By switching the task-specific plugin 010 on the input, plug-tagger allows a frozen PLM 011 to perform different sequence labeling tasks 012 without redeployment. Specifically, the plugin 013 on the input are a few continuous vectors, 014 which manipulates the PLM without modifying 015 its parameters, and each task only needs to store 016 the lightweight vectors rather than a full copy 017 of PLM. To avoid redeployment, we propose 018 the label word mechanism, which reuses the 019 language model head to prevent task-specific 020 classifiers from modifying model structures. 021 Experimental results on three sequence la-022 beling tasks show that the proposed method 023 achieves comparable performance with fine-024 tuning by using 0.1% task-specific parameters. 025 Experiments show that our method is faster 026 than other lightweight methods under limited 027 computational resources 028
分享
引用
