Randomized Controlled Trials Evaluating Large Language Models in Digestive Diseases: a Scoping Review

This scoping review summarizes the current landscape of randomized controlled trials (RCTs) evaluating the use of large language models (LLMs) in digestive diseases. We conducted a systematic search of PubMed, Web of Science, Scopus, and CENTRAL for published RCTs, and of ClinicalTrials.gov and the International Clinical Trials Registry Platform (ICTRP) for ongoing trials. We included RCTs on digestive diseases in which LLMs were the primary intervention. A total of four published and ten ongoing RCTs were identified. Most trials were conducted in China and the United States, primarily as single-country, single-center studies. Gastrointestinal diseases were the main focus, followed by hepatobiliary conditions. The assessed LLM applications predominantly supported clinical decision-making and patient education, with question answering emerging as the most common natural language processing task. Notably, the trials were relatively balanced in their use of general-purpose versus domain-specific LLMs, reflecting diverse strategies in model deployment. The most common study design involved comparisons between LLM-assisted and unassisted approaches, with primary outcomes centered on various aspects of care management. The main limitation of this review is the relatively small number of available studies. Despite this, the identified trials highlight the promising potential of LLM applications in digestive diseases. Further international, multicenter, well-reported RCTs with a focus on real patient outcomes are urgently needed to confirm the usefulness of LLMs in clinical practice.