The original intention of the USMLE was to serve as the primary assessment tool for state medical board physician licensing. Gradually, USMLE (in particular reliance on Step 1 scores) has been adopted as the primary screening and selection tool for the transition of candidates from UME to GME by residency directors and selection committees. Consensus has been developing that the current UME-GME transition reliance on USMLE Step 1 is inherently flawed since the results of a high stakes exam designed to qualify physicians for state licensing is not relevant in either holistically evaluating residency candidates or in providing an equitable means fairly distributing residency positions. This culminated in the Invitational Conference on USMLE Scoring (InCUS) conference in March 2019, where it became clear that USMLE Step 1 was no longer serving the stakeholders in what has become a flawed UME-GME transition system. Although the was general consensus of InCUS was that changes were needed, USMLE alone would not be the only component requiring a “fix”. However, several suggestions included implementing a “pass/fail” , composite, or categorical USMLE scoring system, as well as minimizing racial demographic difference affecting USMLE performance, among others.
From the program director’s perspective, it is unclear that changes in USMLE scoring alone would provide significant additional guidance in residency selection. Equitable selection of residency applicants has become major challenge, in view of the absence of hard data to reliably predict residency performance. In addition to USMLE part 1, our traditional tools have included letters of reference, medical school grades, election to AOA, Deans letter, and clinical course evaluations. Unfortunately, notwithstanding the MSPE and Dean’s letter, critical factors such as professionalism, accountability, social responsibility, team performance, peer interactions, and clinical skills cannot adequately be assessed. For many candidates, the most important determinants of future resident performance are observed during clinical rotations in the chosen subspecialty, either at the home school or as away rotations. Unfortunately, most students are faced with the challenge of a limited number of clinical elective rotations, and hence exposure to potential residency programs. As a consequence, most residency directors overly rely on the USMLE part 1 as a surrogate of clinical performance for students that have not rotated with them. During the COVID pandemic, the absence of in person away rotations has made clinical assessment of resident candidates even more challenging for programs.
Potential solutions could include incorporating a standardized residency assessment tool (RAT) utilized by all medical schools, that may include milestone – type assessments that correlate to the six core competencies, as well as evaluation of skills relevant to the type of residency applied to ( e.g. spatial coordination, technical ability for surgical specialties). Other tools such as the Jefferson Scale of Empathy and Crowd sourcing of clinical skills to assess potential candidates are being piloted in some programs.
This webinar will review the current challenges of resident candidate assessment and selection. The pandemic coming on the heels of the planned conversion of USMLE conversion to Pass/Fail has magnified the importance of developing alternative and viable candidate assessment tools and will be a major factor informing our discussion.
