[The following notes were generated by Mark Slivkoff, PhD]
IAMSE Webinar Series, Spring 2019
Speakers: Michelle Daniel, MD, MHPE, FACEP
Assistant Dean for Curriculum and Associate Professor of Emergency Medicine and Learning Health Sciences
University of Michigan School of Medicine, Ann Arbor, MI USA
Title: Moving USMLE Step 1 After Core Clerkships: Rationale, Challenges, and Early Outcomes
Series: The Role of Basic Science in 21st Century Medical Education
The focus this week was once again on USMLE Step 1. Dr. Michelle Daniel of the University of Michigan School of Medicine presented this final seminar of our series which focused on the placement of Step 1 in the curriculum, a topic that has generated debate recently.
Currently, the ideal placement for Step 1 is unknown. Its current placement, usually at the end of the second year, is mainly based on tradition rather than an alignment of learning goals or outcomes. In her presentation, Dr. Daniel explained the rationale for, breaking with tradition from and altering the timing of Step 1. She also addressed early outcomes associated with the change and discussed potential challengesâand how to overcome them.
She started with a poll in which she asked the audience to choose the stage of change that describes your institution concerning the timing of Step 1:
⢠Pre-contemplative stage (Students take Step 1 following the preclinical curricula before entering clinical clerkships and currently, there are no thoughts of changing)
⢠Contemplative stage (Students take Step 1 following the preclinical curricula, but the institution is currently considering moving Step 1 after clerkships)
⢠Determined stage (Students take Step 1 following the preclinical curricula, but plans are currently underway to move Step 1 after clerkships)
⢠Action stage (Students currently take Step 1 after the core clerkships)
⢠N/A (Students do not take Step 1)
The results were similar to what Dr. Daniel has seen nationally. Over half (55%) of schools are in the pre-contemplative stage, about a third (32%) are in the contemplative stage, 1% are in the determined stage, while 8% are in the action stage.
As is the case with a majority of medical schools, Step 1 is given after years 1 and 2, the basic science years. Students memorize and purge the information during the examination, then promptly forget it. Schools who desire to move Step 1 usually argue that moving the timing of Step 1 after the core clinical year helps promote better retention of foundational science knowledge and basic and clinical science integration. Most schools who alter the timing of Step 1 do so as just one part of the process of determining the means by which science is delivered across the curriculum.
AnecdotallyâDr. Daniel continuedâstudents report that when they study for Step 1 after core clerkships, the basic science information âfeels stickierâ since they are able to recall real patients they have encountered with certain conditions and it becomes easier to remember basic science concepts when they can create linkages to clinical care. The long question stems encountered on the USMLE are therefore easier to understand.
Another reason that schools move Step 1 is to boost their scores. As Dr. Daniel points out (and as last weekâs poll also demonstrated), a majority of individuals desire making Step 1 pass/fail but the reality is that the score matters to residency directors. However, Dr. Daniel believes that this reason for change should be lowest on the list. What is critically important is that when schools have changed the timing, their scores have NOT been negatively impacted. Schools, therefore, can feel free to make sweeping changes to their curriculum, and this seems to be the trend since the list of schools who are making the change is increasing.
Dr. Daniel then took a dive into some of the data which she has published with her colleagues concerning the characteristics of schools which have moved Step 1 (Daniel et al. 2017). She then presented outcome data from a follow-up paper (Jurich et al. 2018). Specifically, they were interested in what happened to Step 1 scores and failure rates after moving the examination. Results indicated a slight increase in scores and a reduction in failure rates. Dr. Daniel additionally pointed out a highlight from the failure data in that the number of failures went down from 48 to 6. She also currently has a paper in review which focuses on Step 2 outcomes.
The conclusion of her webinar focused on the challenges (which she has published as well with her colleagues) (Pock et al. 2019). Since they lack a Step 1 study period to consolidate knowledge, students may not seem as prepared to start clerkships. Communication with clinical faculty must emphasize that learners entering the clerkships are âdifferentâ – not better or worse than historical students, but different. Curricula that foster the integration of basic and clinical sciences, as well as those that emphasize active learning in the pre-clerkship phase can help smooth this transition.
Another challenge to moving Step 1 is that weaker students may not be identified early and provided with appropriate academic support. Supplementing institutional exams with NBME basic science subject exams, the comprehensive basic science exam or comprehensive basic science self-assessment can help identify such learners before they get to Step 1.
Shelf exam performance weaknesses are another concern. Many institutions who have moved Step 1 after core clerkships report a decline in shelf exam performance, particularly in the early clerkships. The declines in shelf scores, as Dr. Daniel explained, are not surprising when one considers that students have not had as much practice with timed tests and NBME style questions.
Other challenges include students extending the Step 1 study period if there is not a firm âdeadlineâ of when to take the exam, students becoming overly concerned about what residency specialty choices may be open to them until they know their Step 1 scores, and lastly, learners may be concerned about having enough time to take and pass Step 1, Step 2 CK, and 2 CS in rapid succession. Dr. Daniel suggested that all these things can be easily managed, however.
Dr. Daniel concluded her presentation just as she did at the beginning: with a poll. She asked now that you have learned a bit more, what would you advise your institution to do?
⢠Keep Step 1 after the pre-clerkship phase
⢠Move Step 1 after core clerkships
⢠It depends on the needs of curricular re-design, but I would be comfortable advocating either position
Approximately half of the respondents chose the latter, and the remaining responses were split between keeping Step 1 as is or moving it. Dr. Daniel once again emphasized the large drop in failure rates.
Similar to the previous four webinars, this one was also followed by numerous questions, too many to address before the hour concluded.
We wish to thank again all presenters and the audience who made this series such a success!
References:
⢠Daniel M., A. Fleming, C. OâConner Grochowski, V. Harnik, S. Klimstra, G. Morrison, A. Pock, M. L. Schwartz, and S. Santen. Why Not Wait? Eight Institutions Share Their Experiences Moving United States Medical Licensing Examination Step 1 After Core Clinical Clerkships. Acad Med 2017;92:1515-1524.
⢠Jurich, D., M. Daniel; M. Paniagua, A. Fleming, V. Harnik, A. Pock, A. Swan-Sein, M.A. Barone, and S. Santen. Moving the United States Medical Licensing Examination Step 1 After Core Clerkships: An Outcomes Analysis. Acad Med 2019;94:371-377.
⢠Pock, A. M. Daniel, S.A. Santen, A. Swan-Sein, A. Fleming, and V. Harnik. Challenges Associated With Moving the United States Medical Licensing Examination (USMLE) Step 1 to After the Core Clerkships and How to Approach Them. Acad Med 2019. DOI: 10.1097/ACM.0000000000002651