[The following notes were generated by Michele Haight, PhD.]

Shani Fleming, MS, MPH, PA-C, Associate Professor, University of Maryland Baltimore Graduate School, Norma Iris Poll-Hunter, PhD, Senior Director Workforce, Diversity, Equity, Inclusion, Association of American Medical Colleges, Rick McGee PhD, Associate Dean for Professional Development, Professor of Medical Education, Northwestern University School of Medicine

Strategies for Promoting Inclusivity in Health Sciences Education:

• Pathways and Pipelines: Approaches to Increasing Diversity in the Health Professions
• Diversity in Medicine, the Impact of Pre-Med Programs
• Diversity in Research Training and Careers

Physician Assistant Programs:
Pathways and Pipelines: Approaches to Increasing Diversity in the Health Professions
Shani Fleming, MS, MPH, PA-C

• The Physician Assistant (PA) program was created in the 1960’s to address physician shortages and promote health equity.
• Physician Assistants work prominently in underserved communities and have the potential to significantly impact the care of racial and ethnic patients of color.
• Demographically, the PA profession is predominantly white and female; current trends show a troubling decrease in diversity and highlight the need for innovative pathways and initiatives to increase PA diversity.
• Health Equity is the ultimate goal and requires equal opportunities for underrepresented students in order to achieve educational excellence within the PA profession.
• The University of Southern California (USC) serves as a model PA pipeline program that bridges the gaps between parents, students, schools and the community to guide and support potential PA students.
• There are several Bachelor/Master PA programs allowing students to have an accelerated pathway for completion of a Bachelor’s degree and entry into the PA program. The University of Maryland Baltimore has partnered with local community colleges to create more affordable options for an accelerated Bachelor/Master PA program.
• Project Access is a grassroots, outreach program which visits schools and communities to inspire and motivate underrepresented students to pursue a career as a PA.
• The PAC (Physician Assistants of Color) program is a grassroots group which began on Facebook and is dedicated to increasing diversity in the PA profession through mentoring, increasing awareness and providing networking opportunities.
• The PA profession is well positioned to collaborate with existing pipeline programs such as STEM, Myerhoff Scholars, historically Black, Hispanic and Tribal Colleges, etc. to increase exposure and opportunities for underrepresented students in the health professions.
• In order to effect real change, we need to systematically examine the health professions institutional culture through an anti-racist lens to address racial disparities.
• Instead of preparing students to enter a flawed system, we need to work on dismantling policies, practices and procedures that are exclusionary for students of color.
• It is time to re-imagine admissions’ polices, recruitment strategies, and institutional culture. Now is our opportunity to move beyond “best practices” to innovative pathways for underrepresented students.

Medical School
Diversity in Medicine, the Impact of Pre-Med Programs
Norma Iris Poll-Hunter, PhD

• This presentation examines the medical education workforce pipeline and the impact of these pipelines along the physician-training continuum.
• Underrepresentation is pervasive across the health professions.
• Underrepresentation in medical schools has evolved to looking at how medical schools define this by looking at local communities.
• As the number of seats for medical students has increased over the years, the number of students who are underrepresented in medicine (UIM) has remained relatively stagnant.
• Over the past 30 years, the majority (80%) of entering medical students come families whose incomes are at the top socioeconomic quintiles. Educational equity needs to look at both underrepresentation of race and ethnicity as well as intersections of income and how income impacts communities with less economic advantage.
• In order to better understand underrepresentation, we must consider the system in which many inequities are embedded. There are systemic issues that need to be addressed in order to enact sustainable equity solutions.
• Pathway/pipeline programs are essential, but cannot exist in isolation. Pathways/pipelines and broader systemic, socio-political issues go hand in hand to impact the trajectory of diversity across the health professions.
• LCME accreditation requirements are an important lever for developing equity programs.
• There are many different pathway/pipeline programs, starting with K-12 and extending to post –baccalaureate. The focus of these programs is to build social capital, clinical exposure and research.
• Pathway/Pipeline programs make a difference in attracting non-typical medical students.

Successful Pathway/Pipeline Characteristics:

• A bundle of activities, not just one.
• Leadership and Faculty engagement. (This had the greatest impact on student outcomes.)
• Student participants pursue professional careers across disciplines: law, business, health professions, public health etc.
• Programs promote and increase diversity in the national pool and provide the ability to track participants.

Lessons Learned from Pre-Med Programs:

• Pre-med programs make a difference; they increase diversity as a part of the solution.
• Pre-med programs provide an emphasis on basic science study and learning skills through academic enrichment.
• Bundling activities makes a big difference.
• Faculty engagement is critical; it predicts success
• National and local partnerships are critical to sustainability and helping to spread the word.
• Grow local talent. Go to the community. Use zip code data; reach out to diverse talent.
• The pandemic has taught us to integrate well-being programs into pipeline initiatives to better prepare underrepresented students for the unique experiences they might encounter over the trajectory of their careers.

PhD Programs in Biomedical Research:
Diversity in Research Training and Careers
Rick McGee, PhD

• Compared to medical students and PAs, PhD students are paid to do their PhD training. After their second year, PhD students are more like residents because they enter the work world.
• There has been a huge decline in the role of the standardized test (GRE) for PhD admission.
• NIH funding drives PhD training.
• Diversity efforts began in 1972 as an outgrowth of the Civil Rights Movement and focused on minority-serving institutions.
• In the 1990s, this focus changed to research-intensive universities to develop diversity initiatives and to produce a much more intensive evaluation of program outcomes as a requirement to continue funding.
• Over the past 30 years the number of students completing STEM and Bachelor of Science (BS) programs has risen substantially; the number of students entering PhD studies has more than doubled.
• Programs such as SACNAS (Society for the Advancement of Chicanos/Hispanic and Native Americans in Science) and ABRCMS (Annual Biomedical Research Conference for Minority Students) support students and link underrepresented minority graduates to PhD programs. The AAMC GREAT (Group on Graduate Research, Education and Training) program made diversity efforts more prominent.
• Data suggested that, despite these programs, the number of NIH funded PIs had not changed at all for those who had been supported by these programs.
• Underrepresented diversity of faculty has changed very slowly over 40 years.
• Simply getting underrepresented minority students into PhD programs is not sufficient. “Mass action” did not work, especially at the faculty level.
• The dismal lack of impact of these programs prompted a concerted effort to create different ways to support PhD students beyond their PhD work.

These efforts included the following:

• Diversity Supplements
• Mentored Career Development Awards (K)
• More emphasis on what goes on during training: T-32, NIGM Grants
• Efforts to increase the skills of mentors and focus on inclusion.
• MOSAIC Program (K99/R00) which includes integration transition into faculty positions.
• Focused support through scientific societies.

Recent New Programs include:

• FIRST (Fellowship in Research and Science Teaching) Program, which supports cluster hires.
• NIH UNITE Program designed to end structural racism and achieve racial equity.

Contributors to African-American disparities in funding appear to be connected to the types of research African-American students are pursuing; these types of research are traditionally lower funded and undervalued by the NIH.

Efforts need to go beyond “numbers only” in terms of diversity and focus on inclusion and equity. These efforts need to understand and eliminate bias in review.

[The following notes were generated by Michele Haight, PhD.]

Arianne Teherani PhD, Professor of Medicine and Education Scientist, Director for Program Evaluation and Education Continuous Quality Improvement,  Karen Hauer, MD, PhD, Clinical Professor, Associate Dean for Assessment,  University of California San Francisco School of Medicine

Achieving Equity in Assessment for Clinical Learners

Equity in assessment  occurs when all learners have fair and impartial opportunities to learn, be coached and receive feedback, be assessed and graded, be advanced and graduated, and be selected for subsequent training and job opportunities. (Lucey et al. Academic Med 2020)

Equity in clinical education is a relatively unexplored area. There is not a singular definition of equity. Equity is built upon the following principles:

• Acknowledgment that differences are not deficits
• Supporting the success of all learners
• Implementation of multiple types of assessments
• Holding all learners to the same high standards
• Moral, legal, political, economic and social dimensions
• What constitutes an equitable outcome?

Conceptual framework for thinking about equity

• Closing achievement gaps (key outcomes we desire to achieve)
• Equal treatment (adequacy and opportunity)
• Fairness (personal or social status are not obstacles to achieving educational potential) and Inclusion (ensuring a standard of education for all)
• Justice

What does it mean to have equity in assessment?

Core ideas of two prevailing equity in assessment  frameworks (Gibbs and Lucey) have been organized into one framework that is focused on macro and micro level solutions. Both levels of solution need to be addressed to propel change.

• Macro level of solution (procedures/policies, top down mandate)
• Micro level of solution (occurs in micro settings/one on one interaction)

MACRO

• Intrinsic Equity (assessment practices are just and fair for all groups and minimize bias)
• Instrumental Equity (results are shared and used to create opportunity; interpretation of results is just and fair)
• Definition of Achievement (takes into account how achievement is defined; what does the institution and the teacher consider achievement; what are the knowledge and skills that are equated with achievement? )

MICRO

• Contextual Equity (learning environment factors, teachers’ beliefs, fairness in the learning environment, bias-free learning environment)
• Multifactorial (learner identity, self-esteem, motivation, expectations for success)

Key Evidence about Equity and Clinical Performance Assessment

• Differential attainment addresses how educational outcomes diverge for different demographic groups which are assessed in the same way
• Studies show differences in clinical performance assessments favor non-UIM (underrepresented in medicine) learners.
• Amplification Cascade shows how a small difference in assessed performance leads to larger differences in grades and selection for awards.
• A UCSF study of its own learners showed evidence of an amplification cascade for UIM learners and demonstrated that academic outcomes differences favor non UIM learners throughout their medical education experience.
• UCSF conducted a Root Cause Analysis to assess what happened, why it happened and how it can be prevented. Results indicated the following:

1. Factors that caused the differences were personal, interpersonal, cultural and structural.

2. Personal/Interpersonal factors that included stereotyping, microaggressions, unconscious biases, etc. disproportionately impacted UIM learners.

3. Cultural/Structural factors embedded in institutional policies (e.g., normative vs. criterion-based grading) and present in the clinical learning environment (e.g., rapidly changing team assignments) disproportionately impacted UIM learners.
4. These impacts resulted in fewer honors grades and awards for UIM learners and affected UIM residency specialty choices and the potential for academic career choices.

To address equity in clinical performance, there needs to be an integrated, systems approach to change which focuses on the following:

• redesign of medical school curricula
• redesign of learning and assessment methods
• faculty development and training programs to identify and avoid bias
• use of frequent observations and feedback as a best practice for assessment
• use of narrative assessments over time
• avoidance of comparing  learners

Countermeasures and Solutions implemented by UCSF

• Advocacy and Health Equities curriculum in Internal Medicine clerkship
• Altered clerkship assessment practices (e.g., grading committees)
• Re-envisioned honors societies’ membership processes
• Faculty training to recognize microaggressions and bias, especially in the learning environment
• Reinforce Educational CQI habit of mind; establish and implement Equity Evaluation Plan.

The clinical learning environment is complex and raises questions about equity.

Concerns about bias found UIM bias affects all aspects of assessment. Non UIM learners were rated slightly higher than UIM.

• Bias is not merely an individual problem; it is a systemic problem.
• Many biases are harmful.
• It is not possible to remove all bias.
• Bring awareness to biases so we can make a choice to avoid acting with harmful discrimination.

We need to design systems that allow all students to learn and become best. This requires a cultural change in medicine.

Factors to consider in determining a clerkship grade:

• Did team members observe the student contributing to patient care?
• Did the learning environment allow learners to perform at their best?
• Do learners and team members share a common understanding of expectations?
• What structural factors in learners’ educational experiences before and during medical school might lead to differences in grades?
• What process does clerkship director use to determine final grade?

There are three components of equity in assessment:

• Intrinsic equity : the way one designs and conducts assessment practices.
• Contextual equity: addresses the environment in which learning and assessments occur.
• Instrumental equity: how use assessment information such as  grades, to determine selection for next opportunity or honors and awards.

Intrinsic Equity:

Older Model of Assessment Practices
Purpose: Classify and rank learners; some students learn more and better. There is  little feedback; promotes fixed mindset
Learner ability: Some will learn more and be better
Feedback:  infrequent, correct/incorrect
Progress: Time-based
Context:  De-contextualized
Mindset: Fixed

Current Model of Mastery Assessment Practices
Purpose: Promote and shape learning (all learners have capacity to learn and succeed)
Learner ability: All learners have the capacity to learn
Feedback:  Frequent, immediate and actionable
Progress: Based on progress map/competency-driven
Context: Real world/interact with patients
Mindset: Growth

We need to challenge faculty to think differently to overcome old assumptions and adopt a growth mindset.

Characteristics of a Growth Mindset in Assessment

• Recognizes and rewards growth and improvement.
• Rethinks the cultural affinity for excellence; the view of excellence in medicine is that a few learners and a few physicians are truly the top and we can identify those. This is a faculty myth based on working with learners for a few days or a week.
• Reframes meritocracy. Not all learners start at same point or have the same experiences. In the learning environment, with attention all learners can be highly successful.
• Establishes a non-competitive learning environment; changes the learner focus from keeping achievement higher than peers to individual focus on how to be better tomorrow than today.

UCSF Interventions:

• Faculty Diversity, Equity and Inclusion Champion Training
• Website for Assessment Guidelines and Equity Checklist for Assessment  https://meded.ucsf.edu/faculty-educators/equity-assessment-guidelines-and-checklist Faculty are encouraged to continue to reflect upon their own biases, and observe learners to better understand all the ways they are contributing to patient care.
• Changes to clerkship assessments: Establish grading committees, eliminate core clerkship honors grades in third year so learners have a year to learn, grow and receive feedback.

Contextual Equity: (Learning and Assessment Environment)

• Environment offers opportunities for all students participate and learn.
• Faculty preparation including allyship against stereotype threat and microaggressions.

Factors influencing performance of UIM learners:

• Stereotype threat: phenomenon in which performance is impaired due to fear of confirming negative stereotypes.
• Microaggressions (can emanate from patients, team members, observations and images)
• Microaggressions can appear as:
  • Microassaults
  • Microinsults
  • Microinvalidations
  • Environmental Images

Bystander Intervention Guide

• Pre brief : anticipate that microaggressions and will occur; engage learners in how they would like a response.
• Recognize and analyze microaggression.
• Consider type of response, e.g., bear witness, intervene with patient, pause, exit room.
• Respond in real time.
• Check in with learner immediately.
• Follow up (based on learner preference).

Instrumental Uses of Assessments: Use of assessments for purposes for which they are not designed. For example, clerkship exams are intended to assess and ensure minimum knowledge competency at the end of the clerkship, but are used to determine clerkship grades, AOA selection or residency selection.

UCSF changed its selection practice for AOA away from metrics to  a holistic process in which learners submit an application and describe how they feel they meet the criteria for AOA, which center on serving the sick.  UCSF no longer participates in AOA due to a conflict of values.

Changing assessments in medical schools raises questions about residency selection. Many national organizations are working to improve the residency application process to ensure it promotes learning across the continuum rather than maintaining a focus on a high stakes selection process.

Recommendations for achieving equity in assessments:

• Examine your own data.
• Design your assessment to fit your purpose.
• Continuously improve assessment.
• Think of assessment as a system.
• Consider the intrinsic, contextual and instrumental aspects of equity in assessment.

[The following notes were generated by Michele Haight, PhD.]

Karen Eley Sanders, MS, EdD, Vice Provost for College Access, Charlotte Baker DrPH, MPH, CPH, Richard Vari, PhD, Senior Dean for Academic Affairs, Joanne Greenawald MD, Director of Problem-Based Learning, Emily M. Holt Forest, MA, Director Counseling and Academic Enrichment Services, Virginia Tech Carilion School of Medicine

Strategies to Recognize and Address Implicit or Explicit Bias in Small Group Teaching

Dr. Eley Sanders presented a video of a small group learning session in which an administrator and medical student actors portrayed a small group learning facilitator and group participants respectively. The video showed group introductions for a problem-based learning  opening session. Dr. Eley Sanders encouraged the webinar participants to consciously reflect upon their impressions, thoughts and feelings during the video presentation.

Upon completion of the video, Dr. Eley Sanders conducted a debriefing session in which she initiated a discussion on “microaggressions.”  She described a system of bias. She explained how stereotypes can lead to prejudice. Acting upon prejudice results in discrimination. Microaggressions are an extension of discrimination, prejudice and bias.

• Bias is an inclination for or against something.
• Stereotypes are generalizations about a person which do not permit for individual differences or variations.
• Prejudice is an opinion, pre-judgment or attitude about a group or its members.
• Discrimination is behavior that treats people unequally because of their group memberships. This often begins with negative stereotypes and prejudices.
• D. Wing Sue classifies microaggressions as “ brief, everyday exchanges that send denigrating messages to certain individuals because of their group membership.”

Dr. Eley Sanders described eight different categories of microaggressions:

• Alien in One’s Own Land
• Second Class Citizen
• Ascription of Intelligence
• Pathologizing Values/Communications Styles
• Color Blindness
• Assumption of Criminal Status
• Denial of Racism/Sexism/Homophobia/Transphobia
• Myth of Meritocracy

Microaggressions can have short term and long terms impacts such as the following:

• Feeling of “onlyness” and loneliness
• Physical avoidance
• Disengaging
• Physiological stress
• Anxiety and/or Depression
• Sleep Difficulties
• Inability to Focus
• Dropping Out
• Patients who experience microaggressions are less likely to follow through on treatment plans, and less likely to return to a medical professional who shows bias.

Dr. Eley Sanders noted that the persons committing microaggressions might be otherwise well-intentioned and unaware of the impact of their words, but this does not minimize the impact of their words.

Dr. Charlotte Baker guided the group to view the video again, pausing to analyze each example of a microaggression. She provided alternative comments and strategies to avoid inappropriate comments. She noted that some people might not be offended by certain comments, but she cautioned that this might not always be the case.

Dr. Baker reviewed other situations that commonly occur in medical education and can lend themselves to microaggressions.

Drs. Eley Sanders and Baker concluded the session by responding to questions posed by the webinar participants.

1. How do you recommend a bystander respond to a microaggression that has been observed in the classroom? Simply say “Ouch” or “Wow.” These responses provide the opportunity for discussion. Give everyone the opportunity to learn from the incident.
2. Is it appropriate to acknowledge that you might have difficulty pronouncing someone’s name, but you will make every effort to learn how to pronounce it? Yes, acknowledge this and you might work with the student to perfect your pronunciation.
3. Participants noted that the examples in the video were more “macro” than “micro” aggressions. Eley Sanders stated that this is a correct observation and clarified that microaggressions can be further broken down to micro-assaults, micro-insults and micro-invalidations.
4. Baker noted that it is important to establish relationships with students, so we need to be constantly aware of “how” we ask questions.
5. What are your recommendations for navigating the student/faculty power differential in conversations? Create a “safe space” in the learning environment for learners to have these types of discussions. Acknowledge and apologize to the learner for any missteps and apologize in the group (if this is where the misstep occurred) in order to role model the behavior we seek from our learners.
6. Should students be called out for committing microaggressions? Absolutely yes. We have to help them learn appropriate behaviors and engagement.
7. Baker and Eley Sanders recommended reading the following article: “My Name is Not Interpreter,” Montenegro, R. JAMA, 2020, 323(17):1700-1701. https://doi.org/10.1001/jama.2020.2976