Auditing Socio-Demographic and Cross-Societal Fairness in LLM-simulated Public Opinion

Abstract Large Language Models (LLMs) are increasingly used to simulate public opinion and societal behavior as a scalable complement to traditional surveys. Yet their deployment in this role raises concerns about fairness, social representation, socio-demographic bias, and cross-cultural validity. In this study, we conduct a fairness audit of LLM-simulated public opinion, examining both cross-national differences (Chile vs. the United States) and disparities across socio-demographic groups within each country using fairness metrics. Using nationally representative survey data from both countries, we evaluate patterns of predictive accuracy and group-level bias. We find substantial disparities. LLMs reproduce U.S. survey responses more faithfully than Chilean ones, consistent with their predominantly U.S.-centric training data. Moreover, the structure of bias varies across contexts: in the United States, disparities are most pronounced along race and political identity, whereas in Chile, gender, education, and religion emerge as more salient axes of inequalities. These findings reveal the uneven social grounding of LLMs and the risk of epistemic injustice when globally trained models are applied to underrepresented regions. Our results serve as a cautionary tale for researchers using LLMs to simulate public opinion, particularly in underrepresented and cross-cultural contexts.