How BERT Speaks Shakespearean English? Evaluating Historical Bias in Contextual Language Models
CoRR(2024)
摘要
In this paper, we explore the idea of analysing the historical bias of
contextual language models based on BERT by measuring their adequacy with
respect to Early Modern (EME) and Modern (ME) English. In our preliminary
experiments, we perform fill-in-the-blank tests with 60 masked sentences (20
EME-specific, 20 ME-specific and 20 generic) and three different models (i.e.,
BERT Base, MacBERTh, English HLM). We then rate the model predictions according
to a 5-point bipolar scale between the two language varieties and derive a
weighted score to measure the adequacy of each model to EME and ME varieties of
English.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要