Protocol for the AutoRayValid-RBfracture Study: Evaluating the efficacy of an AI fracture detection system

Background Rapidly diagnosing fractures in appendicular skeletons is vital in the ED, where junior physicians often interpret initial radiographs. However, missed fractures remain a concern, prompting AI-assisted detection exploration. Yet, existing studies lack clinical context. We propose a multi-center retrospective study evaluating the AI aid RBfracture™ v.1, aiming to assess AI’s impact on diagnostic thinking by analyzing consecutive cases with clinical data, providing insights into fracture detection and clinical decision-making. Objectives To provide new insights on the potential value of AI tools across borders and different healthcare systems. We will evaluate the performance of the AI aid to detect fractures on conventional x-ray images and how its use could affect handling of these cases in a healthcare setting. In order to explore if the use of a trained and certified AI tool on clinical data exposes new challenges, a daily practice clinical scenario will be approached by minimising selection criteria and using consecutive cases. A multicenter, retrospective, diagnostic accuracy cross-sectional design incorporates clinical context. Methods The multicenter study spans three European sites without onsite hardware. AI system RBfracture™ v.1 maintains consistent sensitivity and specificity thresholds. Eligibility involves age ≥21 with x-ray indications for appendicular fractures. Exclusions include casts, follow-up x-rays, nearby hardware. AI aids retrospective fracture detection. Reader sessions include radiology and emergency care residents and trainees reading with and without AI. Fractures are marked, rated, with expert-established reference standards. Data Sequential patient studies at three sites yield 500 cases per site. Data includes anatomy, referral notes, radiology reports, and radiographic images. Expert readers use annotations, clinical context for standards. Statistical methods include dichotomized confidence ratings, sensitivity, specificity calculations, site-based analysis and subgroup considerations. Reference Standard Two experienced readers annotate fractures; if their annotations overlap by 25% or more, the common area is the reference. Discrepancies are resolved by a local expert. Individual fractures are labelled. ### Competing Interest Statement The Departments of Radiology at Bispebjerg and Frederiksberg Hospital, Charite Universitatsmedizin, Berlin, and Erasmus Medical Center, Rotterdam are subcontractors of Radiobotics ApS as part of the EU sponsored AutoRayValid project for which Radiobotics is the primary applicant. M. Boesen is a shareholder and medical adviser in Radiobotics ApS. ### Funding Statement This project has received funding from the European Union’s Horizon 2020 research and innovation programme under grant agreement No 954221 for the EIC SME Instrument project AutoRay. The work only reflects the authors’ view and the European Commission is not responsible for any use that may be made from the information it contains. ### Author Declarations I confirm all relevant ethical guidelines have been followed, and any necessary IRB and/or ethics committee approvals have been obtained. Yes The details of the IRB/oversight body that provided approval or exemption for the research described are given below: Ethics committee of Danish Patient Safety Authority waived ethical approvals for this work. IRB of Charite Universitatmedizin Berlin waived ethical approvals for this work. IRB of Erasmus Medical Center waived ethical approvals for this work. I confirm that all necessary patient/participant consent has been obtained and the appropriate institutional forms have been archived, and that any patient/participant/sample identifiers included were not known to anyone (e.g., hospital staff, patients or participants themselves) outside the research group so cannot be used to identify individuals. Yes I understand that all clinical trials and any other prospective interventional studies must be registered with an ICMJE-approved registry, such as ClinicalTrials.gov. I confirm that any such study reported in the manuscript has been registered and the trial registration ID is provided (note: if posting a prospective study registered retrospectively, please provide a statement in the trial ID field explaining why the study was not registered in advance). Yes I have followed all appropriate research reporting guidelines, such as any relevant EQUATOR Network research reporting checklist(s) and other pertinent material, if applicable. Yes All data produced in the present study are available upon reasonable request to the authors. * AI : Artificial Intelligence AUC : Area under the receiver operating characteristics curve BFH : Bispebjerg and Frederiksberg Hospital CUB : Charité - Universitätsmedizin Berlin ED : Emergency Department EMC : Erasmus Medical Center IRB : Institutional Review Board PACS : Picture Archiving and Communication System REC : Research Ethics Committee