
AIM Despite RapidAI’s widescale adoption for acute stroke management, gaps remain in validating its real-world diagnostic impact. Given local observations of RapidAI overestimating infarct size for equivalent images, we evaluated the reliability of RapidAI versus neuroradiologists at a Canadian comprehensive stroke centre. MATERIALS AND METHODS Computed tomography (CT) head and carotid studies (Jan 2020–Aug 2021) from acute stroke activations were retrospectively reviewed. Repeat presentations, no follow-up imaging within 10 days, bilateral infarcts, non–middle cerebral artery (MCA) strokes, or haemorrhage were excluded. Alberta Stroke Program Early Computed Tomography Scores (ASPECTS) from RapidAI and neuroradiologists were compared to follow-up magnetic resonance (MR) or CT imaging using intraclass correlation coefficient (ICC). A linear mixed-effect model was adjusted for age and sex. Subgroup analyses examined infarct burden and infarct location (cortical vs deep). RESULTS In total, 481 patients (49% female; median age: 74 years) were included. Neuroradiologists had higher agreement with the reference standard than RapidAI (ICC: 0.627 vs 0.546). RapidAI had a smaller adjusted difference (β = −0.10; P = 0.198) than neuroradiologists (β = 0.74; P < .001). Although agreement was poor across both groups, RapidAI showed higher agreement in extensive ischaemia (ASPECTS: 0-4: 0.194 vs 0.115; ASPECTS: 0-2: 0.083 vs 0.059), whereas neuroradiologists had higher agreement in threshold, ASPECTS: 5-7 cases (0.141 vs 0.068). Neuroradiologists showed superior agreement in deep infarcts (ICC: 0.612 vs 0.452). CONCLUSION Neuroradiologists maintain higher overall agreement in mid-range ASPECTS cases and deep infarcts, but RapidAI exhibited higher agreement in lower ASPECTS cases. RapidAI demonstrates context-dependent accuracy, serving as an adjunct requiring cautious use alongside neuroradiologist interpretation for clinical judgement.