Revisiting Privacy Scales: an Investigation into the Ability of Privacy Scales to Capture and Distinguish Granular Privacy Constructs

PurposePrivacy scales are scientific instruments to measure user attitudes, perspectives, preferences and concerns. However, their ability to reliably capture underlying constructs depends on the individual's interpretation of the scales' statements. For this purpose, this study aims to explore users' understanding of statements of commonly used privacy scales with a representative sample in the UK and compare the results with a similar study conducted with a representative sample from the USA.Design/methodology/approachIn this paper, the authors reproduced the work by Colnago et al. by targeting a representative from the UK (n = 390) instead of the US. They distributed an online survey and asked about the participants' understanding of commonly used privacy constructs such as behaviour, perception, and the like. Participants had to indicate the extent to which they associated each scale statement with each construct.FindingsThis study confirmed that no statement uniquely measured only one construct, and there is a misalignment of scale constructs and users' understanding between the USA and the UK. These findings highlight the need to revisit construct definitions and conduct cross-cultural scale validation studies. This study discusses the possible underlying reasons and best practices for future scale development studies.Research limitations/implicationsLike all online surveys, the results might suffer from response biases, such as wrong self-assessments. Furthermore, since online questionnaires do not involve interaction with an examiner, participants might have misinterpreted the sentiment behind a question. However, this study and the original study by Colnago et al. aimed to investigate precisely that.Practical implicationsThe findings highlight the need to revisit construct definitions and conduct cross-cultural scale validation studies. This study discusses the possible underlying reasons and best practices for future scale development studies. The authors publicly make the data collection and analysis code available to allow for replication and further investigations.Originality/valueThis paper makes the following contributions: (1) Investigation of understanding of scale statements: The authors replicate and extend the findings about the users' understanding of four common privacy scales from Colnago et al. with a representative sample from the UK (n = 390). They provide evidence of a significant misalignment between what each construct is expected to measure and what it is observed to be measuring. While the authors partly confirm the results from Colnago et al., they also provide new insights closer to the UK sample and compare both populations. (2) Recommendations for scale development and validation studies: The results show significant differences between the USA and UK samples even though they share one native language. Based on the original study and related work, the authors present recommendations and lessons learned for scale development and validation. (3) Replication Package: The authors make the data collected and analysis code publicly available to allow for replication and further investigations of further communities, such as speakers of other languages or non-(Western, educated, industrialised, rich and democratic) communities.