This study examines whether there is any evidence of bias in two areas of common critique of open, non-anonymous peer review - and used in the post-publication, peer review system operated by the open-access scholarly publishing platform F1000Research. First, is there evidence of bias where a reviewer based in a specific country assesses the work of an author also based in the same country? Second, are reviewers influenced by being able to see the comments and know the origins of previous reviewer? Methods: Scrutinising the open peer review comments published on F1000Research, we assess the extent of two frequently cited potential influences on reviewers that may be the result of the transparency offered by a fully attributable, open peer review publishing model: the national affiliations of authors and reviewers, and the ability of reviewers to view previously-published reviewer reports before submitting their own. The effects of these potential influences were investigated for all first versions of articles published by 8 July 2019 to F1000Research. In 16 out of the 20 countries with the most articles, there was a tendency for reviewers based in the same country to give a more positive review. The difference was statistically significant in one. Only 3 countries had the reverse tendency. Second, there is no evidence of a conformity bias. When reviewers mentioned a previous review in their peer review report, they were not more likely to give the same overall judgement. Although reviewers who had longer to potentially read a previously published reviewer reports were slightly less likely to agree with previous reviewer judgements, this could be due to these articles being difficult to judge rather than deliberate non-conformity.

Primary data collected during a research study is often shared and may be reused for new studies. To assess the extent of data sharing in favourable circumstances and whether data sharing checks can be automated, this article investigates summary statistics from primary human genome-wide association studies (GWAS). This type of data is highly suitable for sharing because it is a standard research output, is straightforward to use in future studies (e.g., for secondary analysis), and may be already stored in a standard format for internal sharing within multi-site research projects. Manual checks of 1799 articles from 2010 and 2017 matching a simple PubMed query for molecular epidemiology GWAS were used to identify 314 primary human GWAS papers. Of these, only 13% reported the location of a complete set of GWAS summary data, increasing from 3% in 2010 to 23% in 2017. Whilst information about whether data was shared was typically located clearly within a data availability statement, the exact nature of the shared data was usually unspecified. Thus, data sharing is the exception even in suitable research fields with relatively strong data sharing norms. Moreover, the lack of clear data descriptions within data sharing statements greatly complicates the task of automatically characterising shared data sets.

PurposePeer reviewer evaluations of academic papers are known to be variable in content and overall judgements but are important academic publishing safeguards. This article introduces a sentiment analysis program, PeerJudge, to detect praise and criticism in peer evaluations. It is designed to support editorial management decisions and reviewers in the scholarly publishing process and for grant funding decision workflows. The initial version of PeerJudge is tailored for reviews from F1000Research's open peer review publishing platform.Design/methodology/approachPeerJudge uses a lexical sentiment analysis approach with a human-coded initial sentiment lexicon and machine learning adjustments and additions. It was built with an F1000Research development corpus and evaluated on a different F1000Research test corpus using reviewer ratings.FindingsPeerJudge can predict F1000Research judgements from negative evaluations in reviewers' comments more accurately than baseline approaches, although not from positive reviewer comments, which seem to be largely unrelated to reviewer decisions. Within the F1000Research mode of post-publication peer review, the absence of any detected negative comments is a reliable indicator that an article will be ‘approved’, but the presence of moderately negative comments could lead to either an approved or approved with reservations decision.Originality/valuePeerJudge is the first transparent AI approach to peer review sentiment detection. It may be used to identify anomalous reviews with text potentially not matching judgements for individual checks or systematic bias assessments.

Today’s publishing environment is evolving. New University Presses (NUPs) and scholarly publishing in the library are increasingly playing an important role in the shift of scholarly communications. The US-based Library Publishing Coalition defines these new library-led presses as a ‘…set of activities led by college and university libraries to support the creation, dissemination, and curation of scholarly, creative, and/or educational works’. They typically embrace open access, digital first, new business models, enable universities to meet strategic goals including outreach and impact, and facilitate researchers in publishing research outputs. In 2016, Jisc and the Northern Collaboration, a group of 25 higher education libraries in the north of England embarked on a research study to identify, evaluate and benchmark NUPs and library-led initiatives. Informed by a desk top review of current library publishing ventures in the US, Europe and Australia, the study will provide an overview of Universities’ existing and future plans and directions regarding NUPs or library publishing ventures in the UK. The data gathered will: • Identify and classify existing and future NUPs / library led ventures in the UK • Learn of the motivations behind their establishment and their missions, visions and goals • Determine the types of output being published, e.g. monographs, journals, grey literature etc. and the service level, e.g. hosting, full publishing services • Gather information on governance and policies, such as peer review processes, contracts and licensing • Identify the publishing platforms being utilised – such as OJS/OMS, repositories, or commercial solutions • Ascertain what business models and distribution methods are being applied to formats, such as open access, print on demand, freemium etc. • Review the marketing and metadata workflows adopted to support end user discovery – such as DOAJ, DOAB, and library web scale discovery systems • Identify workarounds, gaps and frustrations in the workflows A number of follow up interviews will enrich the data gathered in order to gain a snapshot of library publishing trends in the UK in 2016. The research has been designed with a number of goals in mind; taking forward recommendations from the Jisc and AHRC OAPEN-UK final report on open access monographs that pushes for collaboration and best practice through sharing, and providing an evidence base to feed into the development of Jisc’s work on a shared publishing platform. It is also envisaged that this research will facilitate libraries and their institutions working together at a European level by establishing common goals and encouraging best practice and shared services across library publishers in Europe. The study runs from February to June 2016. The presentation at LIBER 2016 will be the first opportunity to present and discuss the findings of this research.This study forms part of a larger Jisc research project focused on institutional publishing initiatives which includes academic led publishing ventures.