Methods to Assess Trustworthiness of Data from Peer-Reviewed, Published Randomised Controlled Trials

Untrustworthy randomised controlled trials (RCTs) and other study types are an increasingly recognised problem in health and medical research. Assessing and responding to this problem is essential for ensuring the reliability of scientific evidence informing clinical practice. We consider theory, methods and tools for assessing research integrity and the trustworthiness of both aggregate and individual-level data from peer-reviewed, published RCTs. We review approaches to the assessment of trustworthiness based on published checklists, the statistical analysis of aggregate and individual participant data, and the use of AI tools. We found checklists that examine the trustworthiness of data considering questions about the timeframe, authors, governance, and plausibility of a published, peer-reviewed paper. These checks of authenticity are complemented by statistical techniques that identify unusual patterns in aggregate or individual data, including new procedures for simulating data to determine if any of the possible distributions are realistic. The unique character of RCTs provides additional opportunities for checks, specifically for the baseline data. The approaches presented offer a series of novel, quantitative tools for assessing research integrity and data trustworthiness across published RCTs. These techniques can detect instances of data duplication, questionable research practices, and check if any aspects of the study or results are unrealistic. They work best when employed beyond isolated, single-trial analyses and, when embedded in AI platforms, should scale to meet the volume of corrupted papers already published and the stream of fake research from paper mills.