We analyse focus group discussions and free-text survey responses from a multi-stakeholder consultation conducted after the October 2023 publication of the proposal Towards Responsible Publishing by cOAlition S. The proposal calls for a systemic reform of scholarly communication by reducing barriers to knowledge dissemination, promoting early sharing of outputs through preprints, and shifting peer review to an open, post-publication model. We focus on how different stakeholder groups –such as researchers, infrastructure providers, academic institutions, and publishers –perceive obstacles to the large-scale, coordinated reform envisioned in the proposal. We interpret these accounts as articulations of collective action problems, shaped by entrenchment of many actors in existing academic reward systems and established commercial revenue models that make transitions toward a more economically sustainable scholarly communication system difficult, even where many actors see the principal need for change. This approach highlights the extent to which stakeholder perspectives align or conflict. It also underscores the performative nature of discourse about collective action problems in scholarly communication: by articulating challenges to reform, participants simultaneously construct, reinforce, or contest their own roles within the system, which directly influences their collective capacity to act.
Background Discourses of ‘Open Science’ have been criticised for privileging the quantitative and positivist methodologies of science, technology, engineering and mathematics (STEM) research at the expense of qualitative, interpretive, critical-theoretical, practice-based, and other forms of research more common in the arts, humanities and social sciences (AHSS). Methods In this article, which emerged from a work component of the MORPHSS (Materialising Open Research Practices in the Humanities and Social Sciences) project, we document the process, outcomes and practical implications of work to develop a catalogue of open research practices in these disciplines, which proceeded via a wide-ranging literature review followed by targeted desk research and a conceptual mapping exercise. Results Key findings include the fact that open research practices in AHSS are diverse, extending beyond the suite of practices emphasised within dominant accounts of Open Science. We identify among these practices a range of forms of openness including those focused on mobilising the involvement and expert knowledge of diverse participants and communities. Conclusions Presenting a typology of forms of openness in AHSS that is consistent with the epistemic logics of these disciplines, we conclude that openness in AHSS is highly situated and context-dependent, as well as resisting quantification and binary measurement. Drawing on these conclusions, we offer a series of recommendations for institutions, open research monitoring initiatives, funders, publishers, learned societies and researchers to enhance the inclusivity of their policy and practice around openness.
Preprinting has become an increasingly important component of the scholarly communication system, facilitating rapid open dissemination of scientific knowledge. This study investigates the adoption of preprinting over time, focusing on how it varies across scientific disciplines and geographical regions. We analyzed bibliometric data on 4M preprints and 105M peer-reviewed outputs in the period 1991-2023. Peer-reviewed outputs were linked to preprints using data from Dimensions, OpenAlex, and Crossref, resulting in 2.2M peer-reviewed outputs linked to a preprint. Our findings indicate a strong growth in preprinting, with a nearly threefold increase in the number of preprints published between 2017 and 2022. The adoption of preprinting is highest in the physical and mathematical sciences, particularly among researchers in the Americas and Europe. In recent years, preprinting has also increased notably in the information and computing sciences and the life and medical sciences, driven primarily by researchers in North America and Western and Northern Europe. Preprinting remains relatively uncommon in the humanities and the social and behavioral sciences. Asia demonstrates low preprint adoption, with Eastern Asia showing a modest increase in recent years. Preprint adoption in specific disciplines varies significantly across regions, showing that preprint adoption is shaped by the interplay between disciplines and regions.
Background Peer review-based funding evaluation systems are under increasing pressure. To help alleviate this, innovative approaches like Distributed Peer Review (DPR) – where applicants review proposals for the same funding call – are being explored. The Volkswagen Foundation trialled DPR alongside conventional panel review for an interdisciplinary call. Evaluation of this trial provides insights into the dynamics of (distributed) peer review. Methods All proposals (N=140) underwent both DPR and panel review. In DPR, applicants (N= 323) reviewed four or five proposals, providing an overall score, scores for each funding criteria, and a comment. Proposals were ranked using a trimmed mean score. Expectation and feedback surveys were administered. Panel review involved shortlisting by VWS staff, panellist review (N=8), and a final panel discussion. Semi-structured interviews with applicants, panellists, and VWS staff (N=20) explored perceptions and experiences of DPR, using thematic analysis. Quantitative analysis of review data and surveys utilised bootstrapping to assess decision stability and mixed effects models for consistency between reviewers. Qualitative and quantitative results were integrated in a convergent parallel design. Results DPR recommended ten proposals for funding, while the panel recommended eleven; three overlapped. Funding criteria strongly predicted overall score; ‘Originality’ was given most weight by both sets of reviewers. Stability of DPR funding decisions was modest; across 500 bootstrapped samples a mean of 5.49 out of ten previously funded proposals would still be funded. This may be explained by high inter-reviewer variability: more variance was attributable to between-reviewer differences (20.04%) than between-proposal differences (8.87%). Advantages of DPR included increased efficiency, faster decision times, detailed feedback to applicants, and democratisation of funding evaluation. Noted disadvantages were burden on applicants, absence of discussion, and perceived lack of reviewer expertise. Funders implementing DPR must consider the high initial effort, potential for ‘gaming’, and reduced direct influence. Conclusions This trial demonstrates feasibility of DPR for an interdisciplinary funding programme. Implementing a DPR process requires funders to accept trade-offs, but can offer valuable benefits, including shortening time to funding decision. Building on established peer review principles, it has the potential to contribute to the democratisation of funding evaluation.
As preprint servers proliferate across scientific fields, moderation has emerged as a lightly structured form of gatekeeping that precedes—or bypasses—peer review. This article examines moderation as a site of precautionary valuation and boundary-making in the early circulation of research. Drawing on interviews with 14 individuals responsible for screening and oversight across 13 preprint servers, we analyze both the evaluative logics that organize moderation and the ways moderators themselves understand and frame their work. We find that credibility is enacted through precaution-driven threshold judgments—centering harm, trust, and community expectations—which determine whether a submission can responsibly circulate as scholarship. While moderators often reject the label of “gatekeeper,” our findings show that their work performs a subtle but consequential filtering that shapes what and who enters the scholarly record—even as servers work to distance these decisions from the domain of peer review. By tracing how precautionary judgments and boundary-making structure decisions about what enters circulation at the preprint stage, we contribute to STS debates on valuation, peer review, and the governance of scholarly communication in open science.
Background Distributed peer review (DPR) of grant proposals, where applicants review the proposals of others applying to the same funding call, offers advantages, including increased feedback to applicants. With the reviewer pool being determined by the applicants to a particular call, there are, however, concerns that reviewers may not possess the necessary expertise and experience to provide quality reviews and feedback. We analysed review comments from recent trials of DPR for funding evaluation at the Volkswagen Foundation to obtain insights into reviewing and the impact of reviewer characteristics on review content. Method Review comments across two years (N=2950) were classified (content and sentiment) using existing, fine-tuned machine learning classifier models. Mean prevalence of content categories and sentiment was calculated, both overall and stratified by reviewer/proposal characteristics. Results Review comments were consistent with review scores and overall ranking of proposals. Minor differences in content and sentiment according to reviewer gender and research experience (years working in research) were detected. There was minimal variation according to funding peer review experience and disciplinary alignment between reviewer and proposal. Conclusion Existing classifier models can be used to analyse grant review comments and to obtain insights into reviewer consistency and adherence to funding criteria. Concerns about the influence of reviewer experience and expertise on reviews may be unfounded. Results from analyses of review comments can be used to audit and refine review processes.
Background:Discourses of 'Open Science' have been criticised for privileging the quantitative and positivist methodologies of science, technology, engineering and mathematics (STEM) research at the expense of qualitative, interpretive, critical-theoretical, practice-based, and other forms of research more common in the arts, humanities and social sciences (AHSS). Methods:In this article, which emerged from a work component of the MORPHSS (Materialising Open Research Practices in the Humanities and Social Sciences) project, we document the process, outcomes and practical implications of work to develop a catalogue of open research practices in these disciplines, which proceeded via a wide-ranging literature review followed by targeted desk research and a conceptual mapping exercise. Results:Key findings include the fact that open research practices in AHSS are diverse, extending beyond the suite of practices emphasised within dominant accounts of Open Science. We identify among these practices a range of forms of openness including those focused on mobilising the involvement and expert knowledge of diverse participants and communities. Conclusions:Presenting a typology of forms of openness in AHSS that is consistent with the epistemic logics of these disciplines, we conclude that openness in AHSS is highly situated and context-dependent, as well as resisting quantification and binary measurement. Drawing on these conclusions, we offer a series of recommendations for institutions, open research monitoring initiatives, funders, publishers, learned societies and researchers to enhance the inclusivity of their policy and practice around openness.
A preprint is a research article that is made openly available on a preprint server, typically before submission to a peer-reviewed journal. Preprinting enables new scientific knowledge to be shared in a timely and open way. It often takes many months or even years for an article to pass through the peer review process of a journal.
Preprinting has gained considerable momentum, and in some fields it has turned into a well-established way to share new scientific findings. The possibility to organise quality control and peer review for preprints is also increasingly highlighted, leading to the development of preprint review services. We report a descriptive study of preprint review services with the aim of developing a systematic understanding of the main characteristics of these services, evaluating how they manage preprint review, and positioning them in the broader scholarly communication landscape. Our study shows that preprint review services have the potential to turn peer review into a more transparent and rewarding experience and to improve publishing and peer review workflows. We are witnessing the growth of a mixed system in which preprint servers, preprint review services and journals operate mostly in complementary ways. In the longer term, however, preprint review services may disrupt the scholarly communication landscape in a more radical way.
Purpose This study explores the evolving role of preprint servers within the scholarly communication system, focusing on their relationship with peer-reviewed journals. As preprints become more common, questioning and understanding their future role is critical for maintaining a healthy scholarly communication ecosystem. By examining the values, concerns and goals of preprint server managers, this study highlights the significant influence these individuals have in shaping the future of preprints. Design/methodology/approach A qualitative, interview-based approach was used to gather insights from preprint server managers on their roles, challenges and visions for the future of preprints within the broader scholarly communication system. Findings The findings point to a lack of consensus on how preprint servers and journals should interact and to diverging views on how the certification and curation functions are best performed and by whom. Concerns about credibility and long-term financial sustainability are increasingly driving independent and community-run preprint servers to align more closely with journals, potentially undermining the disruptive and emancipatory potential of preprints. Originality/value This study is the first to examine the relationship between preprints and journals from the perspective of preprint server managers in the later stages of the COVID-19 pandemic. It sheds light on how preprint servers are navigating external pressures and market dynamics, how they are seeking to establish credibility and trust, and how, in doing so, they are reshaping the core functions of scholarly communication.
The COVID-19 pandemic accelerated the global adoption of open science (OS) practices. However, as the pandemic subsides, the debate around OS continues to evolve. This study investigates how the pandemic has shaped the OS discourse and identifies key issues and challenges. Interviews were conducted with influential actors across the research and publishing communities. The findings show that while many areas of debate remained constant, the ways in which they were discussed exposed underlying systemic challenges, which must be addressed if OS is to progress. These issues included the scope and definition of OS; regional variations in its implementation; the relationship between OS and fundamental questions of the purpose and practice of science; and the need to reform incentives and reward structures within research systems. A more complex understanding of OS is required, which takes into account the importance of equity and diversity and the challenges of implementing OS in different cultural and geographical contexts. The study emphasises the importance of shifting scientific culture to prioritise values such as quality, integrity, and openness, and reforming rewards structures to incentivise open practices.
Although fraud and misconduct have always existed in research and scholarly communication, the rise of paper mills over the past decade has led to an unprecedented volume of fake or manipulated research being published. A 2022 report jointly published by the Committee on Publication Ethics (COPE) and the STM Association suggests that between 2 and 46% of submissions to journals in the time between 2019 and 2021 were produced by paper mills.
Background: In this commentary, we argue that it is time for communication scholars to turn their attention to how and why data journalists engage with the increasing amount of open research and government data available online. Analysis: We review the limited scholarship that has investigated data journalists’ engagement with open data and suggest directions for future research. Conclusions and implications: Research that explicitly examines data journalists’ use of open data is sorely needed, especially research that attends to the varied forms and practices that can emerge in different national and institutional contexts.
National, international and organizational Open Science (OS) policies are being formulated to improve and accelerate research through increased transparency, collaboration and better access to scientific knowledge. Yet, there is mounting concern that OS policies do not effectively capture the ethos of OS, and particularly its goal of making science more collaborative, inclusive and socially engaged. This study explores how OS is conceptualized in emerging OS policies and to what extent notions of equity, diversity and inclusion (EDI), as well as public participation are reflected in policy guidelines and recommendations. We use a qualitative document research approach to critically analyse 52 OS policy documents published between January 2020 and December 2022 in Europe and the Americas. Our results show that OS policies overwhelmingly focus on making research outputs publicly accessible, neglecting to advance the two aspects of OS that hold the key to achieving an equitable and inclusive scientific culture—namely, EDI and public participation. While these concepts are often mentioned and even embraced in OS policy documents, concrete guidance on how they can be promoted in practice is overwhelmingly lacking. Rather than advancing the openness of scientific findings first and promoting EDI and public participation efforts second, we argue that incentives and guidelines must be provided and implemented concurrently to advance the OS movement’s stated goal of making science open to all.
Excellence is omnipresent in the research ecosystem but the narrow focus on excellence is increasingly controversial. One of the key actors in the research ecosystems are research funding organizations, yet their activities are comparatively little studied in relation to excellence. This paper aims to contribute to the excellence debate through an empirical study of how notions of excellence are used, and what functions they serve, in eight research funding organizations. Our study shows research funding organizations have recognized critical voices and are taking steps to address some of the problematic aspects of excellence. However, because research funding organizations are shaped by the excellence regime, and constrained by both governmental policy and scientific elites, research funding organizations cannot simply resort to a debunking critique and do away with excellence altogether. In their efforts to navigate their ambiguous relationship to excellence we show that in many of our case study sites the approach to excellence has shifted from it being taken as a ‘matter of fact’, that is rather taken for granted, to a ‘matter of concern’, that needs to be unpacked and reconfigured. We find funders resort to three mitigation strategies, patching, pluralizing and transforming, in their attempts to reconfigure excellence. Our findings suggest, however, that the current mitigation strategies adopted by funders aimed at reconfiguring excellence leave underlying assumptions about competition and meritocratic ideals largely unquestioned. A transformation of the research ecosystem is unlikely to happen when these ideals are not also problematized.
Classification schemes are a key way of organizing bibliographic knowledge, yet the way that classification schemes communicate their information to classifiers receives little attention. This article takes a novel approach by exploring the visual aspects contained within classification schemes. The research uses a classification scheme analysis methodology. Three different classification scheme phenomena are discussed in terms of their visualization: hierarchy, notation, and notes. Indentation is found to be a significant—and implicit—method of communicating hierarchy to classifiers and offers intriguing solutions to the issues of transmuting from two dimensions into one. The visual elements of notation reveal a strong separation between notation and class, while the visual elements of notes illuminate a varying narrative around the position of notes in the classification scheme . A categorization system for visual elements in classification schemes is presented. Model 1 proffers visual elements as a fourth plane of classification, which extends and remodels Ranganathan's Three Planes of Work . Model 2 shows how visual elements could fit into classification scheme versioning. Ultimately, looking at visual aspects of classification schemes is a novel way of thinking about knowledge organization and can help us to better understand—and ultimately, to better use—classification schemes.
It has long been recognized that there are issues with the appropriateness of citations in the academic literature. Citations of sources that do not support the statement they are cited against are known as quotation errors, and there have been many previous studies of their prevalence. The vast majority of these studies rely on researchers evaluating the accuracy of citations in a small sample of the literature, and show large variation in quotation error rates. In this article we report a novel approach to assessing quotation accuracy via an online survey in which 2648 corresponding authors of articles evaluated a real‐world citation of their work. Respondents were also asked to categorize the perceived purpose of the citation, and what action, if any, they take when encountering inaccurate citations of their work. We found a quotation error rate of 16.6%, with no significant difference across academic disciplines, suggesting that variation in previous studies may be a result of methodological differences. Only 11.3% of respondents indicated they had taken action after encountering an inaccurate citation of their work. This work reveals reasons contributing to inaccurate quotations and issues with citation practices, and offers suggestions of areas for future research.
The domination of scientific publishing in the Global North by major commercial publishers is harmful to science. We need the most powerful members of the research community, funders, governments and Universities, to lead the drive to re-communalise publishing to serve science not the market.