Measuring Hate: Does a Definition Affect Self-Reported Levels of Perpetration and Exposure to Online Hate in Surveys?

The purpose of this research is to test the validity of commonly used measures of exposure to and production of online extremism. Specifically, we investigate if a definition of hate influences survey responses about the production of and exposure to online hate. To explore the effects of a definition, we used a split experimental design on a sample of 18 to 25-year-old Americans where half of the respondents were exposed to the European Union's definition of hate speech and the other half were not. Then, all respondents completed a survey with commonly used items measuring exposure to and perpetration of online hate. The results reveal that providing a definition affects self-reported levels of exposure and perpetration, but the effects are dependent on race. The findings provide evidence that survey responses about online hate may be conditioned by social desirability and framing biases. The findings that group differences exist in how questions about hate are interpreted when definitions of it are not provided mean we must be careful when using measures that try to capture exposure to and the production of hate. While more research is needed, we recommend providing a clear, unambiguous definition when using surveys to measure online hate.