Was it "said" or was it "claimed"? How linguistic bias affects generative language models

People use language in subtle and nuanced ways to convey their beliefs. For instance, saying claimed instead of said casts doubt on the truthfulness of the underlying proposition, thus representing the author's opinion on the matter. Several works have identified classes of words that induce such framing effects. In this paper, we test whether generative language models are sensitive to these linguistic cues. In particular, we test whether prompts that contain linguistic markers of author bias (e.g., hedges, implicatives, subjective intensifiers, assertives) influence the distribution of the generated text. Although these framing effects are subtle and stylistic, we find qualitative and quantitative evidence that they lead to measurable style and topic differences in the generated text, leading to language that is more polarised (both positively and negatively) and, anecdotally, appears more skewed towards controversial entities and events.