Impact of the P-Value Threshold on Interpretation of Trial Outcomes in Obstetrics and Gynecology.

Objective Randomized controlled trials (RCTs) are considered the highest level of evidence to inform clinical practice. However, the reproducibility crisis has raised concerns about the scientific rigor of published RCT findings. Some advocate for a lower p-value threshold. We aimed to review published OB/Gyn topical RCTs in three representative OB/Gyn journals and three high impact non-OB/Gyn journals to determine if their interpretations would change with adoption of a p-value threshold for significance of 0.005. Secondarily, we evaluated if there were differences in methodologic characteristics between those that did and did not lose significance. Study Design A manual search was performed to identify all OB/Gyn RCTs published in the selected journals between July 2017 and June 2019. Data were collected on primary outcome(s), methodology, and p-values. We determined the proportion of primary outcomes that would remain statistically significant with adoption of a p-value significance threshold of 0.005 versus be reinterpreted as "suggestive" (defined as p-value between 0.005 and 0.05). Chi-square or Fisher's exact test were used to compare study characteristics. Results Overall, 202 RCTs met inclusion criteria; 52% in obstetrics and 48% in gynecology. Of 90 studies considered significant with p <0.05 at the time of publication, 54.4% ( n = 49) would maintain significant ( p < 0.005), while 45.6% ( n = 41) would become suggestive using the lower threshold. Most RCTs utilized a single (90.1%) versus composite (8.9%) primary outcome type, used an intent-to-treat analysis (73.3%), and studied a drug intervention (46.5%). Methodologically, 23.7% did not prespecify analysis type, 28.2% did not meet the pre-determined sample size, and 9.4% did not report an a priori sample size calculation. Studies maintaining significance were more likely to be international and report a funding source. Conclusion Adopting a p-value significance threshold of 0.005 would require reinterpretation of almost half of RCT results in the OB/Gyn literature. Highly variable methodological quality was identified.