Pearls and pitfalls of structured staging and reporting of rectal cancer on MRI: an international multireader study

Objectives: To investigate uniformity and pitfalls in structured radiological staging of rectal cancer.Methods: Twenty-one radiologists (12 countries) staged 75 rectal cancers on MRI using a structured reporting template. Interobserver agreement (IOA) was calculated as the percentage agreement between readers (categorical variables) and Krippendorff's alpha (continuous variables). Agreement with an expert consensus served as a surrogate standard of reference to estimate diagnostic accuracy. Polychoric correlation coefficients were used to assess correlations between diagnostic confidence and accuracy (=agreement with expert consensus).Results: Uniformity to diagnose high-risk (>= cT3 ab) versus low-risk (<= cT3 cd) cT-stage, cN0 versus cN+, lateral nodes and tumour deposits, MRF and sphincter involvement, and solid versus mucinous tumours was high with IOA > 80% in the majority of cases (and >80% agreement with expert consensus). Results for assessing extramural vascular invasion, cT-stage (cT1-2/cT3/cT4a/cT4b), cN-stage (cN0/N1/N2), relation to the peritoneal reflection, extent of sphincter involvement (internal/intersphincteric/external) and morphology (solid/annular/semi-annular) were considerably poorer. IOA was high (alpha = 0.72-0.84) for tumour height/length and extramural invasion depth, but low for tumour-MRF distance and number of (suspicious) nodes (alpha = 0.05-0.55). There was a significant positive correlation between diagnostic confidence and accuracy (=agreement with expert consensus) (p < 0.001-p = 0.003).Conclusions: - Several staging items lacked sufficient reproducibility.- Results for cT- and N-staging g improved when using a dichotomized stratification.- Considering the significant correlation between diagnostic confidence and accuracy, a confidence level may be incorporated into structured reporting for specific items with low reproducibility.Advances in knowledge: Although structured reporting aims to achieve uniformity in reporting, several items lack sufficient reproducibility and might benefit from dichotomized assessment and incorporating confidence levels.