Peer review is an important part of the scientific process, but traditional peer review at journals is coming under increased scrutiny for its inefficiency and lack of transparency. As preprints become more widely used and accepted, they raise the possibility of rethinking the peer-review process. Preprints are enabling new forms of peer review that have the potential to be more thorough, inclusive, and collegial than traditional journal peer review, and to thus fundamentally shift the culture of peer review toward constructive collaboration. In this Consensus View, we make a call to action to stakeholders in the community to accelerate the growing momentum of preprint sharing and provide recommendations to empower researchers to provide open and constructive peer review for preprints.

Many argue that swift and fundamental interventions in the system of scholarly communication are needed. However, there are substantial disagreements over the short- and long-term benefits of most proposed approaches to changing the practice of science communication, and the lack of systematic, empirically based research in this area makes these controversies difficult to resolve. We argue that experience within public health can be usefully applied to scholarly communication. Starting with the history of DDT (Dichlorodiphenyltrichloroethane) application, we illustrate four ways complex human systems threaten reliable predictions and blunt ad-hoc interventions. We then show how these apply to interventions in scholarly publication – open access based on the article processing charge (APC), and preprints – to yield surprising results. Finally, we offer approaches to help guide the design of future interventions: identifying measures and outcomes, developing infrastructure, incorporating assessment, and contributing to theories of systemic change.

Growth of preprint review over time. Preprints reviewed per month on Sciety as of 2023-10-30, excluding reviews conducted by automated tools (ScreenIT) and reviews by journals posted after publication of the journal version. The original Google Sheets file may be viewed here: https://docs.google.com/spreadsheets/d/1rjH2weFpeXjY5cGAU3s7wDzjKPuAkf_opjpcEo-_4wg/edit?usp=sharing

Preprints, manuscripts posted online prior to journal-organized peer-review, are an alternative to the traditional, slow, expensive, and inequitable journal publication system. They enable earlier sharing of research outcomes and can even be used to obtain early feedback on work in progress. We aim to identify such alternative uses of preprints, including works-in-progress that are deposited and updated as preprints (iterative preprints). We rely first on a computational approach to identify alternative preprints that we then qualitatively assess. We aim to communicate our approach and results to the community as an iterative preprint itself. In this version, we present our computational approach and initial exploratory results as we seek feedback on our methodology.

Preprints enable new forms of peer review that have the potential to be more thorough, inclusive, and collegial. In December 2022, 80 researchers and representatives of funders, institutions, preprint servers, journals, indexers, and review services were invited to gather online and at the Janelia Research Campus for a workshop on Recognizing Preprint Peer Review. Sponsored by HHMI, ASAPbio, and EMBO, this meeting aimed to catalyze community consensus and support for preprint peer review and to create model funder, institutional, and journal policies that recognize both preprints with reviews, and reviews of preprints. Here, we make a call to action to stakeholders in the community to help capture the growing momentum of preprint sharing and empower researchers to provide open and constructive peer review for preprints.

These outputs were produced in connection with the Recognizing Preprint Peer Review workshop held at Janelia Research Campus & Online on December 1-2 by EMBO, HHMI, and ASAPbio. Original versions of some of these files, retaining original formatting and features, can be found on Google Docs: Survey results Slides

In 2016, Crossref launched a metadata schema for ‘posted content’ with the aim of better accommodating the metadata needs for preprints, working papers, theses, reports, and related outputs, which are not typically peer-reviewed. In 2021, Crossref formed an Advisory Group (AG) to discuss how the quality of metadata for preprints using this schema could be improved. The group focused on four topics: 1) preprints as an article type (rather than subtype of posted-content); 2) relationships to/from preprints; 3) versioning; and 4) withdrawal/removal of preprints. The recommendations that resulted from the group’s deliberations have been published previously. Here we provide further background to the recommendations and report more details of the discussions.

The use of preprints for the dissemination of research in some life sciences branches has increased substantially over the last few years. In this document, we discuss preprint publishing and use in the life sciences, from initial experiments back in the 1960s to the current landscape. We explore the perspectives, advantages and perceived concerns that different stakeholders associate with preprints, and where preprints stand in the context of research assessment frameworks. We also discuss the role of preprints in the publishing ecosystem and within open science more broadly, before outlining some remaining open questions and considerations for the future evolution of preprints.

Amid the Coronavirus Disease 2019 (COVID-19) pandemic, preprints in the biomedical sciences are being posted and accessed at unprecedented rates, drawing widespread attention from the general public, press, and policymakers for the first time. This phenomenon has sharpened long-standing questions about the reliability of information shared prior to journal peer review. Does the information shared in preprints typically withstand the scrutiny of peer review, or are conclusions likely to change in the version of record? We assessed preprints from bioRxiv and medRxiv that had been posted and subsequently published in a journal through April 30, 2020, representing the initial phase of the pandemic response. We utilised a combination of automatic and manual annotations to quantify how an article changed between the preprinted and published version. We found that the total number of figure panels and tables changed little between preprint and published articles. Moreover, the conclusions of 7.2% of non-COVID-19-related and 17.2% of COVID-19-related abstracts undergo a discrete change by the time of publication, but the majority of these changes do not qualitatively change the conclusions of the paper.

Ensuring that public feedback on preprints is focused, appropriate, specific and transparent (or FAST) will help to develop a thriving culture for reviewing and commenting on preprints.

Preprints catalyze rapid and open communication of research. A frequent criticism of preprints, however, is their lack of peer review. In recent years, myriad new initiatives have enabled review of preprinted research to be coordinated, collected, and displayed alongside preprints. This provides evaluation and context for readers, as well as feedback for the authors. The processes behind preprint review are diverse and may differ from journal peer review, which can be a challenge for readers seeking to compare and interpret the reviews. To address this, the ASAPbio organized a working group that set out to define key features of preprint review processes. Here, we describe Preprint Review Features (PReF) as descriptors and provide an implementation guide. PReF captures the key elements of preprint review processes using 8 standard key-value pairs. PReF can serve within the descriptions of individual preprint review processes, and act as search filters on indexing services. Widespread adoption of PReF will promote understanding and categorization of preprint review and improve its discoverability.

The Crossref Preprint Advisory Group (AG) was formed in June 2021 to discuss metadata issues related to preprints. It has the following aims...

There has been strong interest in preprint commenting and review activities in recent years. Public preprint feedback can bring benefits to authors, readers and others in scholarly communication, however, the level of public commenting on preprints is still low. This is likely due to cultural barriers, such as fear by authors that criticisms on their paper will bias readers, editors and evaluators, and concerns by commenters that posting a public critique on a preprint by a more senior colleague may lead to retribution. In order to help address these cultural barriers and foster positive and constructive participation in public preprint feedback, we have developed a set of 14 principles for creating, responding to, and interpreting preprint feedback. The principles are clustered around four broad themes: Focused, Appropriate, Specific, Transparent (FAST). We describe each of the FAST principles and designate which actors (authors, reviewers and the community) each of the principles applies to. We discuss the possible implementation of the FAST principles by different stakeholders in science communication, and explore what opportunities and challenges lie ahead in the path towards a thriving preprint feedback ecosystem.

s represent the first port of call for most readers, usually being freely available, brief, relatively 264 jargon-free, and machine-readable. Importantly, abstracts contain the key findings and conclusions 265 from an article. To analyse differences in abstracts between preprint and paper, we employed multiple 266 approaches. We first objectively compared textual changes between abstract pairs using a 267 computational approach before manually annotating abstracts (Fig. 3). Both approaches 268 demonstrated that COVID-19 articles underwent greater textual changes in their abstracts compared 269 to non-COVID-19 articles. However, in determining the type of changes, we discovered that 6% of non270 COVID-related abstracts and 15% of COVID-related abstracts had discrete, “major” changes in their 271 conclusions. Indeed, 42% of non-COVID-19 abstracts underwent no meaningful change between 272 preprint and published versions, though only 34% of COVID-19 abstracts were similarly unchanged. 273 The majority of changes were “minor” textual alterations that lead to a minor change or strengthening 274 or softening of conclusions. Of note, about 1/3 of changes were additions of new data (Fig 3F). While 275 previous works have focused their attention on the automatic processing of many other aspects of 276 . CC-BY 4.0 International license perpetuity. It is made available under a preprint (which was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in The copyright holder for this this version posted February 20, 2021. ; https://doi.org/10.1101/2021.02.20.432090 doi: bioRxiv preprint

Peer review of a research product varies widely depending on the publishers and platforms involved in the process. As scholarly publishing is disrupted by new innovations, peer review processes become more heterogeneous, placing an increasing burden on the researcher in understanding how they can communicate their scholarship. New ways to model such processes, and increase transparency, trust, and experimentation in scholarly publishing are needed. Many are emerging but can tend to focus on the needs of creators, and not those of readers, funders, and the whole scholarly publishing ecosystem. They may not place focus on representing editorial practices in ways that can be reliably aggregated, surfaced, and queried; are often limited to traditional peer review processes; and cannot capture the full range of editorial practices and events needed to accommodate alternative publication, review, and curation models. To support researchers in a world of experimentation in scholarly publishing, we propose a machine-readable, extensible, and discoverable framework for representing and surfacing review and editorial processes. Working with a Technical Committee composed of interested parties by employing a modified Delphi Method, we developed initial guiding principles and proposals towards an object-level editorial metadata framework compatible with a broad range of possible futures for scholarly publishing. We present the results of this process with a proposal and example use cases for DocMaps , a framework for representing object-level assertions.