This study investigates researcher variability in computational reproduction, an activity for which it is least expected. Eighty-five independent teams attempted numerical replication of results from an original study of policy preferences and immigration. Reproduction teams were randomly grouped into a ‘transparent group’ receiving original study and code or ‘opaque group’ receiving only a method and results description and no code. The transparent group mostly verified original results (95.7% same sign and p-value cutoff), while the opaque group had less success (89.3%). Second-decimal place exact numerical reproductions were less common (76.9 and 48.1%). Qualitative investigation of the workflows revealed many causes of error, including mistakes and procedural variations. When curating mistakes, we still find that only the transparent group was reliably successful. Our findings imply a need for transparency, but also more. Institutional checks and less subjective difficulty for researchers ‘doing reproduction’ would help, implying a need for better training. We also urge increased awareness of complexity in the research process and in ‘push button’ replications.

Der Beitrag diskutiert den Stellenwert ethnografischer Perspektiven in der Hochschulforschung anhand von Pionierstudien und Schlüsseltexten. Ausgehend von Phänomenen aus dem Alltag von Studierenden, des Hochschulpersonals und der Wissensproduktion skizziert er Anknüpfungspunkte einer Ethnografie der Hochschule, die sich mithilfe des Konzepts des Settings neue Themenfelder und methodische Zugänge zur universitären Praxis erschließt und sich damit auch programmatisch zur gängigen Hochschulforschung positioniert.

This study explores how researchers' analytical choices affect the reliability of scientific findings. Most discussions of reliability problems in science focus on systematic biases. We broaden the lens to emphasize the idiosyncrasy of conscious and unconscious decisions that researchers make during data analysis. We coordinated 161 researchers in 73 research teams and observed their research decisions as they used the same data to independently test the same prominent social science hypothesis: that greater immigration reduces support for social policies among the public. In this typical case of social science research, research teams reported both widely diverging numerical findings and substantive conclusions despite identical start conditions. Researchers' expertise, prior beliefs, and expectations barely predict the wide variation in research outcomes. More than 95% of the total variance in numerical results remains unexplained even after qualitative coding of all identifiable decisions in each team's workflow. This reveals a universe of uncertainty that remains hidden when considering a single study in isolation. The idiosyncratic nature of how researchers' results and conclusions varied is a previously underappreciated explanation for why many scientific hypotheses remain contested. These results call for greater epistemic humility and clarity in reporting scientific findings.

Der Beitrag gibt einen Überblick über die Vielfalt studentischen Engagements in Deutschland und skizziert Eckpunkte einer Ethnografie studentischen Vereinslebens. Am Fallbeispiel des Kölner Organisationsforums Wirtschaftskongress (OFW) zeigt er auf, wie sich Studierendeninitiativen ethnografisch erforschen lassen und welche Bedeutung ihnen im Kontext der Hochschulethnografie zukommt. Hierzu erkundet er Praktiken der Vergemeinschaftung, Prozesse der beruflichen Sozialisation und Elemente der materiellen Kultur. Darauf aufbauend arbeitet der Beitrag methodische Besonderheiten einer Ethnografie studentischen Vereinslebens heraus und diskutiert die damit verbundenen Herausforderungen und Potenziale, darunter auch Fragen des Feldzugangs, der Fallauswahl und des Verhältnisses von Nähe und Distanz.

An abstract is not available for this content. As you have access to this content, full HTML content is provided on this page. A PDF of this content is also available in through the 'Save PDF' action button.

In an era of mass migration, social scientists, populist parties and social movements raise concerns over the future of immigration-destination societies. What impacts does this have on policy and social solidarity? Comparative cross-national research, relying mostly on secondary data, has findings in different directions. There is a threat of selective model reporting and lack of replicability. The heterogeneity of countries obscures attempts to clearly define data-generating models. P-hacking and HARKing lurk among standard research practices in this area.This project employs crowdsourcing to address these issues. It draws on replication, deliberation, meta-analysis and harnessing the power of many minds at once. The Crowdsourced Replication Initiative carries two main goals, (a) to better investigate the linkage between immigration and social policy preferences across countries, and (b) to develop crowdsourcing as a social science method. The Executive Report provides short reviews of the area of social policy preferences and immigration, and the methods and impetus behind crowdsourcing plus a description of the entire project. Three main areas of findings will appear in three papers, that are registered as PAPs or in process.

Die WSI-Mitteilungen sind eine wissenschaftliche Fachzeitschrift mit politik- und praxisorientierter Ausrichtung. Sie informieren über neue wissenschaftliche Erkenntnisse im Themenspektrum »Arbeit - Wirtschaft - Soziales«. Die Zeitschrift ist 1948 erstmals erschienen.