Abstract Animal methods bias is a peer review bias characterized by a preference for animal-based biomedical research methods or lack of expertise to adequately evaluate nonanimal methods, like complex in vitro models. Preliminary evidence indicates that some manuscript reviewers request animal experiments, which authors often feel are not scientifically justified. We explored this through a cross-sectional survey of 189 biomedical researchers and bibliometric analyses. Fifty-two percent of survey respondents had been asked to add animal experiments to nonanimal studies during manuscript review. On average, authors refused 79% of these requests, citing reasons like lack of necessity or feasibility. Moreover, 46% felt their nonanimal studies received unfair reviews due to absence of animals. Most respondents (84%) reported negative consequences, like publishing in lower-impact journals (65%), rejection (49%), and publication delays (42%). To triangulate self-reported findings, we investigated over 1.1 million publications (2009–2023), identifying significantly higher journal impact factors for studies using animal versus in vitro methods, although differences varied by field. Additionally, we created a natural language processing pipeline to analyze 304,849 animal-based publications and compare species used in 3,159 corresponding preprints. We identified 17 studies where an animal experiment was introduced after the first preprint version was posted, suggesting that authors occasionally add them after original study completion. Animal methods bias presents challenges for researchers using animal-free methods and addressing it is essential as funding bodies increasingly invest in nonanimal approaches to improve translatability. These findings help elucidate animal methods bias and highlight a critical challenge for biomedical research evaluation.
Abstract When a public official proposes direct employment as the test of whether publicly funded research is worth its cost, the remark is easy to dismiss as political rhetoric. Read from inside the evaluation community, however, it is a proposal about evaluation: it nominates a single, legible indicator as the measure of public value. Writing as a research-institute director, principal investigator and evaluator in Chile’s national funding system, I take that proposal seriously to show what an assessment system would do if it adopted it. I argue that the temptation toward single-proxy assessment is strongest, and most damaging, in low-investment systems, where the cost of misreading what research produces is highest. Drawing on field experience, I distinguish five kinds of return–scientific, formative, institutional-territorial, democratic, and slow-maturing–that an employment figure cannot capture, and that are especially characteristic of the social sciences and humanities. Read against the ‘productive interactions’ approach, the typology also counts the collective capacity built by institutional continuity as a return in itself. I then translate this critique into three practical design moves for assessment: eliciting a plural, structured value proposition at the application stage; crediting public presence and uptake as evidence of return; and timing evaluation to the return through periodic ex-post review. The aim is not to reject accountability but to make our instruments of accountability intelligent enough to register what research actually contributes. The piece offers situated, transferable guidance for funders and panels, particularly in lower-investment and Global South research systems.
Abstract Large-scale bibliometric databases generally provide a selective and geographically-biased representation of scientific output. Given that these indexing biases reduce the likelihood of retrieving certain types of scientific production, it has been historically hard to assess the extent and characteristics of those excluded scholarly communications. Based on the Lattes platform, a unique data source with comprehensive coverage of Brazilian scholarly output, this paper assesses the coverage of two large-scale bibliometric databases: OpenAlex and Web of Science. Our findings identify the specific research areas underrepresented in both databases and assess the relation between publication language and indexing. Additionally, we examine gender-based differences in topic coverage, language, and indexing rates. Our results suggest that indexing biases disproportionately affect researchers focusing on topics of national relevance for Brazil through articles written in Portuguese. Given women’s overrepresentation in this group, our findings illustrate how indexing biases contribute to gender inequalities in science.
Abstract Global inequalities in knowledge production raise concerns that doctoral training outside the Anglo-American core may yield weaker research returns because peripheral universities depend on core journals and networks. The centre–periphery model suggests that graduates trained in core systems may retain lasting research advantages. However, Japan has emerged as a regional hub whose doctoral programs may challenge this prediction. This study examines whether doctoral training location is associated with the subsequent research performance of Chinese engineering PhDs trained in the global core, represented by the USA, and a regional core, represented by Japan. Two cohorts of graduates from 2010 to 2011 were linked to publication records in SciVal, and regression analyses examined three dimensions of research performance: quantity, citation-based quality, and international collaboration. The analysis detects no statistically significant difference in publication counts or the number of internationally co-authored publications. However, US-trained graduates show higher citation impact and more publications in the top 1% by citation percentile. These findings indicate no statistically detectable difference in overall publication output or internationally co-authored publication output in the main specification, while differences remain in several citation-based indicators. The study refines centre–periphery theory by highlighting Japan’s capacity to narrow, though not eliminate, disparities in highly cited research and points to a differentiated geography of doctoral advantages in an increasingly multipolar academic landscape.
Abstract This paper examines the research assessment in Uruguay as a paradigmatic case of multi-level governance and overlapping evaluation burn-out. Focusing on the interaction between the National System of Researchers (SNI) and the Total Dedication (TD) regime at the major University of the country, the study analyzes how national and institutional evaluation systems coexist and conflictingly shape academic careers. Drawing on 55 semi-structured interviews, focus groups with evaluation bodies, and a prosopography analysis of researchers’ trajectories, the paper explores how quality, productivity, and recognition are symbolically constructed and applied. The findings show that, despite an explicit commitment to qualitative evaluation, both systems reproduce quantitative pressures through requirements such as publication regularity, doctoral training, and international visibility. The SNI promotes an ‘academicist’ model centered on individual performance and global recognition, contributing to stagnation in career advancement, gender inequalities, and the devaluation of locally oriented research. In contrast, the TD regime emphasizes integrality across teaching, research, and service, enabling more diverse academic profiles but generating high evaluative workloads due to its coexistence with other systems. The Uruguayan case highlights the tensions between global standards and local standards, as well as the unintended consequences of the autonomous governance of several coexisting value regimes. The paper argues that responsible research assessment requires not only revising evaluation criteria but also addressing the necessary coordination of the institutions with the R&D system. These findings contribute to broader debates on how governance shapes evaluation practices and the possibilities for reforming research assessment in contexts with a long-standing tradition of university autonomy.
