Learning from Patterns via Pre-trained Masked Language Model for Semi-supervised Automated Essay Scoring.

Automated essay scoring is one of the key applications of natural language processing technology in the field of education. Currently, pre-trained language models for automated essay scoring systems have no significant advantage over classical models, and pre-trained language models are underutilized for this task. Moreover, in real-world scenarios, supervised models that lack annotated data often perform poorly. To address the issue that the pre-trained language models are not fully applied, we propose a novel prompt tuning model PTAES in this paper, and we convert the essay scoring procedure into a cloze-style question, after which we design pairs of natural language pattern-verbalizer. Further, we propose a joint model that combines the prompt tuning based model and the pre-trained fine-tuning based model, attempting to address the issue of inadequately supervised models in low-resource situations. Experimental findings indicate that, in supervised, semi-supervised, and zero-shot scenarios, our model can achieve state-of-the-art results, and our method further increases the value of the pre-trained language model in automated essay scoring.