The Value of Psychometric Analysis of the Advanced Trauma Life Support Cognitive Test: Outcome of an ACS-Accredited Educational Institute Multisite Study.

BACKGROUND:The Advanced Trauma Life Support® (ATLS®) course provides a standard approach to trauma. Participants must pass the ATLS® post-test. We deployed the test online to allow ongoing psychometric item analysis and potential objective refinement.METHODS:A two-phase study was undertaken with the ACS COT permission. In the first phase, ATLS® post-test #2 was computerized and deployed using Qualtrics©. Data were collected from fourteen courses conducted between 2014 and 2015 (n = 306) at one ACS AEI site. In the second phase, the same post-test was administered to 238 trainees in 10 courses via secured computers at four ACS AEI sites in 2016.RESULTS:Phase 1 item analyses showed two items with very low percentages correct, and one of these also showed a low discrimination index. Phase 2 item analyses suggested four items as candidates for review and possible revision. We also found differences by learner background and by instructional site.CONCLUSIONS:This study demonstrates computerized delivery of the ATLS post-test is feasible, promotes psychometric analysis, and could improve the quality of the test. Further collaboration between the ACS COT and ACS AEI would be beneficial.