Keyword mask pre-training model for the review of standard declaration

The review of standard declaration of import and export commodities refers to the evaluation of whether the declaration information of the commodities filled in by enterprises conforms to the customs filling standards according to the standardized declaration catalogue The manual review method cannot meet the actual work needs. Therefore, using text classification algorithm to conduct this task is necessary. The classification method based on the pre-training model would rely on the keywords. However, for the review of standard declaration task, excessive reliance on keywords will reduce the classification effect because of ignoring the overall information of the text. Keyword mask pre-training model is proposed. In the fine-tuning stage, the model designs a keyword masking strategy and a keyword blinding strategy. The keyword masking strategy regularizes the BERT model with a mask mechanism, that is, the keywords are reconstructed with other text information except keywords, so that the model focuses on non-keywords information; Keyword blinding strategy makes the model unable to distinguish the categories of text information without contextual text information, which can eliminate the influence of keywords on model decision-making. The experimental results on the real customs declaration data set showing that the method in this thesis exceeds the baseline method in terms of precision and true negative rate.