Using Markov Chain Model to Evaluate Medical Students’ Trajectory on Progress Tests and Predict USMLE Step 1 Scores

Abstract Medical students must meet curricular expectations and pass national licensing examinations to become physicians. The Michigan State University College of Human Medicine implemented progress testing in place of discipline-specific examinations as its primary assessment of knowledge in 2016. Ideally this innovative assessment strategy will characterize students’ growth in basic science knowledge over time and predict licensing examination performance.Markov chain method was employed to: 1) identify latent states of acquiring scientific knowledge based on progress tests, 2) estimate students’ transition probabilities between states, and 3) predict United States Medical Licensing Examination Step 1 results based on the students’ predicted probabilities in each state. A total of 358 students were included in the analysis. Four latent states were identified based on students’ progress test results: Novice, Advanced Beginner I, Advanced Beginner II and Competent States. At the end of the first year, students predicted to remain in the Novice state had lower mean Step 1 scores compared to those in the Competent state (209, SD = 14.8 versus 255, SD = 10.8 respectively) and had more first attempt failures (11.5% versus 0%). On regression analysis, it is found that at the end of the first year, if there was 10% higher chance staying in Novice State, Step 1 scores will be predicted 2.0 points lower (P< .01); while 10% higher chance in Competent State, Step 1scores will be predicted 4.3 points higher (P< .01). Similar findings were also found at the end of second year medical school.Using the Markov chain model to analyze longitudinal progress test performance offers a flexible and effective estimation method to identify students’ transitions across latent stages for acquiring scientific knowledge. The results can help identify students who are at-risk for licensing examination failure and may benefit from targeted academic support.