Written by Mark Slivkoff, PhD

Title
Navigating the Effects of COVID-19 on the 20-2021 Residency Application Cycle

Presenters
Maya M. Hammoud, MD, MBA
Professor of Ob/GYN
University of Michigan

Versha Pleasant, MD, MPH
Fellow, Department of Ob/GYN
University of Michigan

Keli Santos-Parker, PhD
4th Year Medical Student
University of Michigan

In this final presentation of the mini-Fall series, the focus turned towards graduate medical admissions, towards residency applications. All three presenters were from Michigan Medicine, at the University of Michigan.

They discussed the implications of COVID-19 on the residency cycle and outlined some actions applicants and programs can take to mitigate harmful effects.

Dr. Hammoud began the webinar. She first discussed how COVID has affected and will potentially affect the applications, screening, interviews and post-interview processes. The Electronic Residency Application Service (ERAS) will still be utilized, but the timelines for submission have been altered with applications being accepted by program directors now on October 21.

The number of applications continues to rise as well, which poses additional challenges since applications outnumber residencies. Full applications will still be viewed, but Step 2 scores and/or letters of reference may be absent or delayed. Again, the reduced timeline and increased number of applicants could prove problematic. Program directors are very concerned about not being able to complete holistic reviews of each applicant.

Interviews will be done virtually, which presents more challenges due to technology that has to be learned, and to the lack of face-to-face interaction which makes it very difficult to rank candidates. There is concern that program directors will be more apt to rank their own students higher.

Overall, Dr. Hammoud emphasized that the current pandemic is amplifying problems already inherent in the residency application process. She and others have recently published a paper in the Journal of the American Medical Association (JAMA) which addresses the following proposed changes to the process:

• Adjust the residency application timeline
• Modify application requirements
• Encourage holistic review
• Limit the number of applications
• Expand program information available to applicants
• Improve the quality of information programs receive
• Temporarily make exceptions to the NRMP all-in policy
• Cap the number of interviews a student can accept
• Implement preference-signaling mechanisms

Dr. Hammoud concluded by calling for all interested stakeholders to check out the recent guidelines for their specialties, available via the American Association of Medical Colleges (AAMC): https://students-residents.aamc.org/applying-residency/article/specialty-response-covid-19/

Dr. Versha Pleasant continued the webinar with a discussion of virtual experiences and the impact on underrepresented medical students during COVID-19.

She first discussed the American Medical Association’s guidelines regarding underrepresented students and residents during COVID-19. They highlight in their report (https://www.ama-assn.org/delivering-care/public-health/protecting-underrepresented-students-and- residents-during-covid-19) the following:

• COVID-19 impacting all areas of society but not equally
• Additional stressors of structural racism
• Disruptions from COVID-19 amplify underlying inequalities
• Responsibility of AMA to advocate for all medical students; reverse the historic active exclusion of racially marginalized groups (specifically Blacks, Latinx, Native Americans)

Some of this education inequity, she continued, involves the shift to virtual platforms which requires access to technology and dedicated spaces. Loss of enrichment opportunities including shadowing, research, and global experiences is also concerning. Furthermore, there are geographical inconsistencies to consider as well, plus limitations in obtaining letters of recommendation and away rotations. She emphasized the potential weathering that takes its toll on underrepresented minorities.

To address these issues, the University of Michigan hosted a workshop in August of this year to highlight their culture. They had a 90-min session dedicated to sharing the culture with an emphasis on diversity and inclusion. Underrepresented minority (URM) members of departments were represented and experiences were shared. There were about 90 participants with 26 URMs (Black, Latinx, Native American). They plan on following up with a leadership conference.

Dr. Pleasant concluded her portion of her talk by reiterating that the new climate has required us to be flexible and creative, and despite these changes, we must remain committed to diversity and equity.

Dr. Santos-Parker, a current medical student at the University of Michigan who is currently applying to residencies, concluded the webinar with a focus on student perspectives.

Unfortunately for residency applicants this year, he explained, the pandemic has coincided with the normal process and timing of selecting residencies, identifying mentors, and attending conferences.

Dr. Santos-Parker explained the usual stressors of the application process and placed additional emphasis on the challenge in securing away rotations which are used to demonstrate an applicant’s preference for a residency. He spoke of innovations such as virtual events that are helping to fill this component of the application process. Other preference-signaling mechanisms include token systems and limiting the number of applications and interviews, all of which are currently being utilized by applicants.

He spoke of the advantages and disadvantages of virtual interviews. They save cost and time and are convenient, but they are much different than in-person social interaction, make it difficult for applicants to judge the local environment, and set the stage for the applicant and interview inflation.

Dr. Santos-Parker concluded by discussing that, ideally, applicants should apply only to the few programs of interest, but he recognized that there is always the dilemma of playing the odds and applying broadly. He noted that if everyone applies broadly, holistic review diminishes. He urged all applicants to apply intentionally, communicate with mentors, and promptly decline interview offers if not interested.

Written by Mark Slivkoff, PhD

Title
PA and PT Admissions in the Time of COVID-19: A Panel Discussion

Presenters

Erika Brooks
Manager, Application Services
Physician Assistant Education Association

Thomas O’Shea, PhD, MEd
Director of Administration and Student Services
University of Iowa Carver College of Medicine
Physician Assistant Graduate Program

Jeremy Turkall, MS
Academic Services Administrator – Admissions, Alumni, and Community Engagement
University of South Florida Morsani College of Medicine
School of Physical Therapy and Rehabilitation Sciences

Ms. Brooks began the webinar. She discussed how her national organization, the Physician Assistant Education Association (PAEA), is supporting its members during the pandemic. Specifically, she explained that the PAEA is providing members with resources to adapt to the admissions process and maintain its integrity, and creating awareness about implicit bias in the admissions process.

The PAEA has helped schools transition to new virtual environments by hosting virtual fairs for campus visits, managing the integrity of the interviews and the application process, and providing resources for matriculants to help create a sense of belonging. As the organization behind CASPA, the Centralized Application Service to Physician Assistants, Ms. Brooks went on to explain how this process has also morphed. There is now a COVID-19 impact essay requirement, and an allowance for unofficial transcripts from applicants running into difficulty getting official transcripts from academic institutions.

The PAEA has been actively surveying its member schools since the pandemic began. They have found that 74% have eliminated in-person interviews, 64% are offering virtual tours, and 27% are paying particular attention to the impact essay. Approximately 13% of programs have not made any changes at all to their admissions process.

The webinar was passed over to Dr. O’Shea of the University of Iowa Physician Assistant Graduate Program. He addressed how the admissions process of this program was specifically altered during the pandemic. Dr. O’Shea first discussed the mission of the University of Iowa’s PA program, then summarized the admission and interview criteria before and after COVID-19.

He described the program’s Pre-COVID interview process which is a two-day experience for applicants. When the pandemic surfaced, the admissions committee pondered questions related to academic issues such as GPA requirements, acceptance of pass/fail courses and of online coursework, GRE requirements and recommendation letters. There were concerns about applicants not obtaining the required 1000 hours of volunteer work, and about the online process of submitting their applications through CASPA. Virtual interviews were also concerning.

The final question which Dr. O’Shea presented was, with all concerns addressed, “Does anything really need to change?” The answer was “no”, but they did change some of the variables for admission. These included allowing courses that were graded pass/fail to be counted towards requirements, allowing GRE scores to be self-reported, and lowering the number of volunteer healthcare hours from 1000 to 500. The school did move interviews to Zoom (virtual), but they remained two-day events.

To conclude the panel discussion, Jeremy Turkall of the University of Florida Morsani discussed the updates and changes to the DPT program at his school.

He first began with describing the changes related to PTCAS, the Physical Therapy Centralized Application Service. These changes included launching the application service three weeks earlier to allow applicants more time to prepare, and increasing the number of maximum recommendation letters from 4 to 5 in order to capture additional insight from individuals who know the applicants. Similar to CASPA, a COVID-19 impact question was also added to the application (although this was optional).

At the University of South Florida, Mr. Turkall explained that they accepted pass/fail courses and waived the requirement for observation hours and for two letters of recommendation from licensed physical therapists. They also accepted online courses, but still required the GRE with scores due by November. Interviews were not affected since the school does not carry out interviews as part of the application process.

Written by Mark Slivkoff, PhD

Title
Medical School Admissions in the Time of COVID-19: Maintaining the Integrity of Holistic Review

Presenters
Christina J. Grabowski, PhD, MSA
Associate Dean for Admissions and Enrollment Management
University of Alabama at Birmingham School of Medicine

Leila E. Harrison, PhD, MA, MED
Senior Associate Dean for Admissions and Student Affairs
Washington State University Elson L. Floyd College of Medicine

This presentation by Drs. Grabowski and Harrison was the first of three webinars in IAMSE’s abbreviated Fall webinar series on admissions during the COVID-19 pandemic.

They began by detailing the impact of COVID-19 on medical school applicants whose premedical experiences were altered when the pandemic began back in March 2020. Lectures and labs were transitioned from in-person to remote environments, access to medically-related experiences was restricted, and there were delays and cancellations of the Medical College Admissions Test (MCAT). In addition to its effects on academics, the pandemic also impacted the personal lives of applicants. Many suffered job (and health insurance) losses, decreased health and wellness, and forced moves and relocation due to university closures.

Schools had to consider these impacts when preparing for upcoming interviews. Similar to courses, interviews were also transitioned to virtual processes since students could not travel, and the health and safety of everyone involved took priority. Interview day events had to be restructured using new tools and technology, and virtual tours and videos had to be produced in order to showcase campuses. These changes required additional training of interviewers and everyone involved with the admissions process.

Every student has been impacted by the pandemic, but students from disadvantaged and marginalized populations have been subjected to additional challenges, as Dr. Grabowski explained. Many students do not have access to technology and fast internet connections, nor do they have access to a private space in their homes to study or interview. She also addressed concerns that the virtual environment has led to a loss of accommodations for disabilities. Masks, for example, make it impossible for the hearing impaired to read lips.

Mitigating these unintended consequences on applicants and school employees has involved changes to the application and screening processes, and committee and interview training. As Dr. Harrison explained, secondary applications now include a separate essay which addresses impacts of the pandemic which allows applicants to share their experiences, and further allows reviewers to gain further insight into applicants. She also delved a bit deeper into the training of interviewers and focused on the importance of not biasing applicants based on where they are interviewing (in their cars, for example) or what they are wearing during the interviews. Better communication with applicants and pre-health advisors has also been implemented with the goal of alleviating anxieties and remaining as transparent as possible during the admissions cycle.

Dr. Grawbowski continued the discussion by pointing out the importance of doing a more holistic review of applicants, of a more individualized consideration of experiences, attributes, and metrics in the context of the applicant’s lived experience. She emphasized the benefits of focussing more on competencies rather than on the time and duration of experiences.

She closed the session by pointing out the positive side, the opportunities which have been highlighted during the pandemic including:

• Applicants may reveal attributes such as resilience and adaptability
• Reduced expenses/burden for travel
• Cost savings for schools (travel, events)
• Heightened awareness of bias, impact on disadvantaged or marginalized populations
• Use of new technologies
• Engage stakeholders from remote campuses/locations in the process
• Creative thinking on how we do what we do

[The following notes were generated by Rebecca Rowe, PhD.]

IAMSE Webinar Series, Spring 2020

Speakers: Jed Gonzalo, MD MSc, Penn State College of Medicine and Stephanie Starr, MD, Mayo Clinic Alix School of Medicine
Title: “Health Systems Science is the Broccoli of US Medical Education: Tackling the Key Challenges of Implementation”
Series: Evolution and Revolution in Medical Education: Health Systems Sciences

Objectives

• Describe the phases of implementation of HSS in US medical schools
• Identify the “TOP 7” – selected key challenges to HSS education
• Explore three vignettes related to HSS education in US medical schools, with articulation of potential solutions or “take-aways”
• Commit to one action to address challenges at your school

The Phases of Change: Where is your Medical School?

• Exploration Phase – Just beginning to think about HSS
• Decision and Planning Phase – Have decided to incorporate HSS and are now in the planning phase.
• Implementation Phase – We launched! Now What?
• Maintenance and Continuous Improvement Phase – Have been doing this for a while but are still experiences challenges that exist along the continuum.

The “TOP” 7 Challenge Areas for HSS Education (There are many more but these are the top challenges across some of the phases.

Issue 1: The Nomenclature and Language

• The evidence around any one of the 12 different areas of HSS have been around for decades, for example Patient Safety has been around for decades, Quality Improvement has been around for decades as well.
• With this said, the last 7 years have seen all of these areas coalescing into 1 strategic and comprehensive framework.
• The definition that we typically assign to HSS includes “The methods, the processes and the principals involved with improving outcomes, the quality and cost of care of patients of populations within a larger context of systems of care.”
• The nomenclature issues involves some schools stating they may integrate the Social determinants of health or quality care, but these are only two components of HSS. In order to fully integrate HSS in to medical education all 12 components must be integrated.
• This led to a study (Gonzalo, et. al. HSS in Med Ed: Unifying the Components to Catalyze Transformation. Acad Med, 2020) that mapped the 12 different areas of HSS with the different resources, expectancies, competencies, curriculum recommendations that exists and milestones that are used in medical education, that would show and demonstrate the areas of overlap and where there may be gaps across the different components of HSS.
• HSS is not equivalent to some of the finite pieces but is more comprehensive framework balancing all 3 pillars of medical education.

Issue 2: Curriculum Timing, Sequence, Integration

• Includes the total footprint of incorporating HSS
• Lack of consensus of HSS competencies (pieces in AAMC core EPAs, LCME, DC)
• Curriculomegaly – where can we find more space to add content?
• Drip vs bolus method— Drip method where add in several HSS topics into the preexisting system blocks with the basic and clinical sciences. Bolus method have several stand-alone intensive 2 week courses
• Development sequence – what is the right order of business for folks to learn these topics?
• Integrating well with the basic and clinical sciences. Want the students to experience a seamless three science strands across all four years. To be able to do this is a challenge!

Issue 3: Student Perceptions

Adding HSS into existing curricular is not an easy thing to do, especially in the area of student perceptions and student engagement.  In a study that was done prior to implementing HSS into the curriculum to determine the pedagogical challenges, which are the issues that need to be identified before starting the work of adding HSS to make sure you are able to overcome these issues in order to have everyone engaged in your educational program.

Current medical student priorities (includes two pillars of medical science, basic and clinical science) vs alternative medical students priorities that includes the HSS along with basic and clinical sciences.

Current Medical Student                                           Alternative Medical Student

Best Residency Program                                             Best Doctor Possible

Grades and Board Exams        ←AT ODDS→            Patient-Centered Skills

Basic and Clinical Science Courses                            Balance of Basic, Clinical and HSS

• Look at student perceptions of the two-pillar model to medical education to the three pillar approach to medical education. And these two areas are at ODDS.
• In another study (Gonzalo, JD, et. al. Unpacking Medical Students’ Mixed Engagement in HSS. Teach Learn Med, 2019), where a qualitative analysis was done from all of their students’ comments at the end of sessions and courses. These are the themes that emerged:

Issue 4: Faculty Role Modeling and Skills

Comes from the study (Gonzalo, et.al. Concerns and Recommendations for Integrating HSS into Medical Student Education, Acad Med, 2017) that looked at faculty comments and issues shown below:

Importance of Learning HSS

1. “If medical education isn’t broke, don’t fix it.”
2. “HSS is too complex and best learned in residency or practice.”
3. “Early students do not have skills to contribute to health care, and the value added roles already exist.” The roles of medical education is not for the future. The roles of medical education is now and can make a difference today!
4. “Health Systems Science is not yet a true science.”  All of the components of HSS have been around for a while. What is new is bringing them all together in a comprehensive framework. Basic and Clinical Sciences are not enough.

Practical Concerns

1. “There is limited space in an already packed curriculum.” Most of HSS may be already present in our curriculum. It just needs to be relabeled.
2. “Few faculty have the knowledge and skills to teach HSS.”
3. “Accreditation agencies and licensing boards do not support medical education transformation.”
4. “Evolving health systems are not ready to partner with schools with HSS curricula.”

• May be that some of our pre-clinical and clinical faculty are already teaching components of HSS but the topics are not labeled as such.
• May need to develop new educator roles for HSS as documented in the paper by Gonzalo, et. al. New Educator Roles for HSS: Implications for US Medical School Faculty: Acad Med, 2019, where three new or reimagined types of educator roles in HSS were discussed. These are:

1. Classroom or Zoom Instructors
2. Clinical Supervisors or Educators
3. Curriculum leader/evaluator
4. Mentor/advisor (projects, scholarship, career path)

In Summary:

• “New” educators are here.
• Need to advance skills based on education science with attention to learning environments (classroom, online or clinical learning environments).
• Need to acknowledge and reward these roles and work on faculty development.

Issue 5: Assessment of Learners

How do you assess? This is not always an easy thing to do. Start by looking at Miller’s pyramid.

• Learner triangle has four components at the base are cognition and at the top or apex is behavior.
• Base of the triangle is KNOWS: Fact Gathering
• Next level is KNOWS HOW: Interpretation/Application
• These two areas make up ~ 80% of where medical students are.
• Third level is SHOWS: Demonstration of Learning
• Top level or apex is DOES: Performance Integrated into Practice
• Top two levels is where 20% of medical students are.
• The bottom two levels make up the Cognition portion of the triangle and the top two levels make up the Behavior portion of the triangle.
• The percentages are the opinion of Dr. Gonzalo.

Where does HSS appear on the USMLE Board Examinations?

1. Behavioral Health

• • Patient Adherence

2. Epidemiology/Population Health

• • Epidemiology/population health

3. Social Sciences

• • Communication/cultural competence
  • Death/dying and palliative care

4. Systems-Based Practice

• • Complexity/systems thinking
  • Quality improvement
  • Patient safety
  • Health care policy and economics

Content Analysis of HSS Content on NBME USMLE Examinations: (Please note most of this data is anecdotal and should not be quoted!)

• Before 2020 ~7%
• After 2020 estimated that Step 1 and Step 2 would include more HSS ~10-15%
• It is possible that Step 3 could contain as much as 20%.
• What this is showing is HSS is appearing on the Board Exams more frequently.

Issue 6: Clinical Learning Environment

“The learning environment refers to the social interactions, organizational culture and structures, and physical and virtual spaces that surround and shape the learners’ experiences, perceptions, and learning.” (Macy Foundation Conference on the Clinical Learning Environment (Gruppen, Irby, Durning, Maggio, van Schaik)

• Closer gradient of HSS concept knowledge, skills between faculty and learners. Therefore, the students are teaching the faculty, as well as, faculty teaching students
• Practice and education silos
• Practice pressures
• Faculty development
• Insufficient UME structures, processes to ensure reliable ongoing horizontal and vertical integration of HSS

Issue 7: Program Evaluation

This is looking beyond level of the individual learner, but from also the programmatic standpoint.

• Level 1 Reaction: Satisfaction Engagement Relevance (Surveys and Course Evaluations)
• Level 2 Learning: Knowledge, Skills, Attitude Confidence Commitment (NBME HSS Exam Grad. Questionnaire)
• Level 3 Behavior: Application Drivers (Course Assessment, Clinical Assessment, AMA-GME Milestones)
• Level 4 Results: Outcomes Indicators (Patient Outcomes Big Data)

Completed the TOP 7 areas of challenges of the different phases of where institutions might be!

At the end of the Webinar, the speakers took us through a few vignettes from institutions at various stages of bringing HSS into their curriculum that reached out to them for consultation. The title of the vignettes and the specific phase of change are:

Vignette 1: “We’re Thinking about HSS!” – Exploration Phase
Vignette 2: “Our Student Satisfaction Scores are So Low!” – Implementation Phase
Vignette 3: “How Do We Integrate with Our Health System?” – Implementation Phase

Please refer to the archive to listen to the discussion of these vignettes about these implementation challenges.

Lastly, HSS is not just an education framework. It is actually a framework that unifies all health care.