Initial Validity Analysis of the American Board of Emergency Medicine Enhanced Oral Examination.

ObjectivesThe American Board of Emergency Medicine (ABEM) has introduced a new testing format for the oral certification examination (OCE): the enhanced oral or eOral format. The purpose of this study was to perform initial validity analyses of the eOral format. The two hypotheses were: 1) the case content in the eOral format was sufficiently similar to clinical practice and 2) the eOral case materials were sufficiently similar to clinical practice. The eOral and traditional formats were compared for these characteristics. MethodsThis was a prospective survey study. The survey was administered as a voluntary postexamination activity at the end of the 2015 spring (April 25-27) and fall (October 10-13) ABEM OCEs. The survey is a routine part of the ABEM oral examination experience. For 2015, two additional questions were added to gauge the similarity of the eOral format to clinical practice. Validity was defined by content and substantive elements within Messick's model of construct validity as well as portions of Kane's validity model. ResultsOf the 1,746 physicians who took the oral examination, 1,380 physicians (79.0%) completed all or part of the study survey questions. The majority of respondents agreed the patient presentations in the cases were similar (strongly agreed or agreed) to cases seen in clinical practice, in both the traditional cases (95.1%) and the eOral cases (90.1%). Likewise, the majority of respondents answered that the case materials (e.g., laboratory, radiographs) were similar (strongly agreed or agreed) to what they encounter in clinical practice, both in the traditional format (85.8%) and in the eOral cases (93.7%). ConclusionsMost emergency physicians reported that the types of cases tested in the traditional and eOral formats were similar to cases encountered in clinical practice. In addition, most physicians found the case materials to be similar to what is seen in clinical practice. This study provides early validity evidence for the eOral format.