Evaluation of an automated phenotyping algorithm for rheumatoid arthritis

•A phenotyping algorithm for RA showed lower performance on new patient population.•Clinically important features and patient subtypes were missed by algorithm.•Missing data and EHR design limited usability of the phenotyping algorithm.•Design choices during algorithm development reduces transportability.•Feature design and training cohort impacts algorithm generalizability.