PURPOSE Radiologists differ in how strongly they hedge. We tested whether large language models can grade linguistic uncertainty in chest radiograph reports on a 0-3 scale and whether aggregate agreement conceals category-specific differences. METHODS We used 3,927 public chest radiograph reports (Open-I, Indiana University). Three radiologists independently scored 100 reports from 0 (definitive wording) to 3 (extensive ambiguity). Majority vote, with prespecified attending adjudication for seven complete three-way disagreements, formed the reference standard. Claude Opus 4.5 and Gemini 2.5 Pro scored the same reports from a fixed prompt without task-specific training. We measured agreement using Cohen's quadratic weighted kappa (QWK). We also recorded per-score recall, ordinal error, prompt and reference-standard robustness, and comparison with a keyword-counting baseline. We then scored the remaining 3,827 reports with both models. RESULTS Human pairwise QWK ranged from 0.421 to 0.659. Claude and Gemini had similar aggregate agreement: QWK: 0.807 [95% confidence interval (CI), 0.722-0.876] and 0.814 (95% CI, 0.735-0.877), respectively; paired difference: 0.007 (95% CI, -0.062 to 0.079). In this validation sample, Claude identified 5/8 reports exactly as Score 3 and Gemini 1/8; Score-3 precision was 5/11 and 1/3, respectively. Of Gemini's seven exact Score-3 misses, six were assigned the adjacent Score 2 category. Reference-standard sensitivities yielded QWK: 0.810-0.841; prompt-variant pairwise agreement ranged from 0.818 to 0.913; repeat runs were identical for 29/30 Claude and 28/30 Gemini reports. All 3,827 Gemini discovery requests completed successfully, with token accounting recorded for the discovery cohort. CONCLUSION The approach is workable at the corpus scale, but similar aggregate agreement concealed differences in exact Score-3 classification, largely involving the Score-2/3 boundary. The score reflects wording, not clinical risk, and must not be used for risk triage. Clinical or large-scale use requires external validation enriched for ambiguous cases and prior radiologist calibration. CLINICAL SIGNIFICANCE These scores may help radiologists audit reporting language and build balanced research evaluation sets, but downstream benefit must be demonstrated rather than assumed.
PURPOSE To synthesize histology-referenced diagnostic performance at study-selected thresholds and the longitudinal responsiveness of magnetic resonance imaging proton density fat fraction (MRI-PDFF) and selected quantitative ultrasound methods in adults with non-alcoholic fatty liver disease/metabolic dysfunction-associated steatotic liver disease. METHODS Four databases were searched from database inception through July 1, 2026. Sensitivity, specificity, and odds ratios were pooled with 95% confidence intervals using heterogeneity-selected fixed- or random-effects models. Small-study effects were assessed using funnel plots and Begg tests; longitudinal changes were pooled when change-score standard deviations were reported. RESULTS In total, 44 articles were included: 34 contributed histology-referenced diagnostic data, and 11 were assessed for longitudinal responsiveness, with 1 article contributing to both. At steatosis grade (S) ≥ 1/S ≥ 2/S ≥3, MRI-PDFF sensitivity and specificity was 93.6%/81.5%/83.1% and 87.1%/86.7%/83.2%, respectively; the controlled attenuation parameter (CAP) was 86.8%/81.2%/86.4% and 98.9%/71.0%/64.6%, respectively. The attenuation coefficient/attenuation imaging sensitivity/specificity was 83.2%/98.1%, 86.4%/79.9%, and 99.3%/76.4%, respectively. Ultrasound-guided attenuation parameter and ultrasound-derived fat fraction evidence was limited. Three head-to-head longitudinal studies included 298 participants, of whom only 158 had analyzable change-score data; in these, pooled changes were -4.53 percentage points for MRI-PDFF and -12.71 dB/m for the CAP. Thirty-two of the 34 diagnostic studies were at high-risk of bias from cohort-selected thresholds. Begg tests did not detect significant small-study effects, although their power was very limited in analyses that included only 2 or 3 studies. CONCLUSION At study-selected thresholds, MRI-PDFF showed balanced sensitivity and specificity across steatosis grades, whereas quantitative ultrasound varied by method, platform, and threshold. These findings support comparative synthesis but do not establish validated cut-offs or superiority of either modality. CLINICAL SIGNIFICANCE Study-selected thresholds limit clinical translation. MRI-PDFF proton density fat fraction and quantitative ultrasound are complementary, with use depending on clinical purpose, platform, and precision needs.
Artificial intelligence (AI) in radiology remains predominantly examination-centric, whereas clinical interpretation frequently depends on comparison across time or with relevant prior cases. This narrative review examines comparative reasoning in radiology AI through two related but clinically distinct forms: longitudinal reasoning, which compares examinations from the same patient over time, and reference-case reasoning, which uses clinically analogous cases from other patients to support interpretation. We propose a capability hierarchy that distinguishes simple access to comparative information from higher-level capabilities, including selection of relevant historical context, anatomical and lesion-level correspondence, characterization of interval change, temporal-state interpretation, multi-interval trajectory reasoning, and entity-specific assessment of clinical similarity. This distinction matters because providing prior examinations or retrieving visually similar cases does not itself demonstrate meaningful comparative reasoning. We review developments in prior-examination selection, registration, lesion tracking, temporal vision-language modeling, treatment-response assessment, analogous-case retrieval, and longitudinal report generation. Particular emphasis is placed on evaluation strategies that determine whether model performance depends appropriately on clinically relevant comparative information rather than on additional input volume, global image similarity, acquisition characteristics, or other shortcuts. Key challenges include incorrect prior selection, lesion mismatching, temporal inconsistency, clinically irrelevant reference-case retrieval, data leakage, misleading comparative language, and propagation of historical errors. Future progress will require models that preserve entity identity, reason coherently across multi-visit imaging histories, identify clinically informative reference cases, and demonstrate benefit under realistic clinical conditions. Comparative reasoning is therefore a necessary step in the transition from examination-level toward patient-level radiology AI.
