[The following notes were generated by Sandra Haudek, PhD.]

The Winter 2022 IAMSE Webinar Seminar Series, titled “How Science Educators Still Matter. Leveraging The Basic Sciences for Student Success” ended with its fifth seminar on Thursday February 3, 2022, titled “Rethinking Assessment Strategies in the Basic Sciences as Step 1 Goes Pass/Fail”. This seminar was presented by Dr. David Harris, Associate Professor of Physiology at the University of Central Florida (UCF), College of Medicine. Dr. Harris discussed alternative strategies to assess basic science content, specifically the new assessment strategies recently implemented at his institution.

Dr. Harris started with describing UCF’s current medical school curriculum (120 students per class). The first year focuses on the normal basic sciences; the second year builds on the pathology, pathophysiology, pharmacology, and management of different organ systems. Over the first 7 months, two courses cover most of the traditional basic sciences. He stated that the herein described changes targeted the assessment strategies during this first 7-month period. He cautioned that these changes were still work in progress as they continuously adapt to feedback. He also mentioned upfront that a particular challenge they faced was UCF’s use of letter grades.

Dr. Harris cited external recourses frequently used by students during their early study of basic sciences, such as Anki, Aquifer, and Boards & Beyond. While many of these resources are well constructed with practice questions, feedback, and links to reliable sources, he stated that students struggle with how to effectively include these resources into their studies. Nevertheless and probably further driven by the COVID-19 pandemic, students accelerated their use of such outside resources. Thus, the role of faculty also changed from traditional textbook teaching to include other teaching modalities. Citing work by Simpson (Simpson, JGME 2018), Dr. Harris listed expected roles of a medical educator in 2025. Dr. Harris proposed that, due to the COVID-19-induced drastic changes in pedagogy, these role changes will happen more quickly than originally expected.

Dr. Harris then specifically discussed the role of the medical educator as “assessor” using his own experience at UCF. He reviewed that most assessments to test basic science content were multiple-choice questions (MCQ). However, he argued that, while MCQ work well to assess content knowledge, they do not assess how learners apply what they memorized. He emphasized on other skills and attributes needed for being a ”good doctor”. Such attributes include critical thinking, good communication skills, and life-long learning. In addition, he mentioned that students with disabilities and specific learning difficulties should not be disadvantaged by the assessment method (disability inclusion). Dr. Harris specifically outlined how faculty will have to adapt to roles of being an “assessor” and “coach” to provide feedback for skills that cannot be fostered by outside resources or tested in MCQs.

Dr. Harris continued with stating UCF’s goals that guided their changes in assessment strategies: (1) USMLE STEP 1 going pass/fail, (2) emphasis on communication, critical thinking, and self-regulation, (3) limit use of MCQ, and (4) provide opportunities for cognitive integration early in the curriculum. With these goals in mind, UCF focused their efforts on four areas: (1) concept mapping, (2) high fidelity patient simulations, (3) team-based learning sessions, and (4) case-based learning. At this point, the fourth focus has not been implemented since this requires uniforming the use of case presentations (for delivering new content, highlighting applications, or reviewing old content) and thus will not be discussed in this seminar.

Dr. Harris explained how UCF integrated concept mapping exercises at various points (once a month) during the first 7 months of their preclinical curriculum. Some were used for formative assessments; others were designed as summative assessments. For each exercise, students received the following: The overall objective of the activity (“terminal” objective), several specific objectives (“enabling” objectives) each of which had to be addressed, a set of instructions including rules for which resources can be used, a clinical vignette, a list of facts they memorized from text books (e.g., Ohm’s Law), a starter map, tools to build and change the map, and the rubric by which they will be assessed (number grading). Students worked in groups of 6, applying their knowledge of foundational content and using available technology; faculty acted as facilitators. The final concept maps (created with “Cmap” software) were due within 2 hours. Dr. Harris noted that several of these steps were adjusted over time. Grading/assessment was performed by content and non-content experts. Several challenges arose: Non-content experts needed too long to finish grading, contributed to high variability in grading, and were unable to provide in-depth content feedback (solved by only involving content experts, using a speed-grading feature). The inability to comment directly in the Cmap file (solved by converting file to pdf files). Other challenges pertained to the grading mechanisms, faculty time, and reducing student stress.

Dr. Harris then discussed the “high fidelity patient simulations” in his basic science course. The goal was to highlight important physiology concepts underlying various diseases. He explained that students started in the classroom with an introduction (10 min), then entered the simulation center (20 min), then came back to the classroom for debriefing (30 min). To increase student engagement, students needed to answer (individually or in teams) specific essay questions on which they receive feedback from faculty. Students were then asked to self-reflect on their simulation experience on which faculty again provided narrative feedback (to foster self-regulation skills). Dr. Harris stated that the advantages (e.g., engaging, clinical application) and disadvantages (e.g., resource and time extensive) of this process were still under debate.

Lastly, Dr. Harris addressed team-based learning. After briefly explaining the general process of team-based learning, Dr. Harris pointed out the changes UCF made: all parts were made summative, MCQ questions were replaced with other question types, the grading scale was modified, and peer evaluation was added.

Dr. Harris concluded with his take home message: (1) Modify what you have or is available to you. (2) Recognize the “cultural” shift for students and faculty roles. (3) Focus on what you want the students to be able to do, what you value, and measure those.

The presentation lasted about 40 minutes and a rich discussion followed. Among other topics, questions from the audience addressed: How to prepare students on how to do concept mapping and using the software? What was their response to the mapping exercise? How (and why) can students be limited in their use of resources? How big is inter-rater variability? Recruitment and training of faculty? More information on the grading scale? Do student groups rotate?

[The following notes were generated by Sandra Haudek, PhD.]

The Winter 2022 IAMSE Webinar Seminar Series, titled “How Science Educators Still Matter. Leveraging The Basic Sciences for Student Success” continued with its fourth seminar on Thursday January 27, 2022, titled “Identity Shape Shifting: How basic science teaching practices can foster identity transformation from medical student to medical professional”. This seminar was presented by Dr. Michelle Lazarus and Dr. Shemona Rozario, both from Monash University in Melbourne, Australia. Dr. Lazarus is the Director of the Centre of Human Anatomy Education and leader of the Centre for Scholarship in Health Education Curriculum Integration Network. Dr. Shemona Rozario is a physician and PhD student in Dr. Lazarus’ research group. They presented an overview of what professional identity (PI) is, mechanisms of professional identity development (PID), ways foundational medical sciences can impact PID, and applied teaching practices basic science educators can implement to help support medical student PID.

Dr. Lazarus and Dr. Rozario introduced themselves by telling us their journey of PI formation from basic scientist / physician to medical researcher. Dr. Lazarus started the presentation with discussing the historical apprentice model, in which trainees watched and learned in a didactic fashion. In the present, medical curricula integrate simulation opportunities. Future curricula will be full with technology including artificial intelligence (AI) and PID must shift alongside this development.

Dr. Rozario quoted Rees & Monrouxe “How we perceive ourselves as professionals is based on our attributes, beliefs, values, motives, and experiences in relation to our profession, providing us with ethical frameworks and values”. She provided an example from the Australian Medical Association: https://www.ama-assn.org/residents-students/specialty-profiles/what-it-s-specialize-emergency-medicine-shadowing-dr-clem, concluding that PI comes from “What we do influences how we view ourselves”. She explained that our identity development includes the formation of personal and professional identities that both develop through social interactions over our life span. Pre-determined factors (genetics) are unchangeable. During childhood, we are most malleable to social construction of identity formation and the results are most resistant to changes. The dynamic social construction of identity occurs in adult life when we interact more formally with our environment, individuals, and community (religion, culture, socioeconomics, status, personal relationships).

Dr. Rozario quoted work by Monrouxe & Poole who compared the structure of identify with the structure of an onion (core = internal identity, inner layer = internal identity, outer layer = constructed identity). As we grow and interact more widely with others, we add more layers (complexity) to our identity. Identity dissonance occurs when professional and personal identities clash while growing. Pratt and colleagues described identity dissonance in a study investigating identity construction of medical residents in different specialties over a 6-year period. They found that radiology residents spent more time on attending lectures and thus developed a parallel identity as medical educators since their work did no match their expectations of the roles and responsibilities of a physician (identity splinting). Similarity, surgery residents spent more time on paperwork and thus developed a parallel identity as general physicians (identity patching). By contrast, primary care residents experienced only minor violations and thus were reinforced of their current identity (identity enriching).

Dr. Lazarus continued with discussing how basic science educators can help students develop their PI by providing the right environment and right people. She reviewed work from Dr. Nicole Woods, a cognitive psychologist specializing on identity formation https://staff.ki.se/kiprime-podcast-episode-6-nicole-n-woods. She explained that expert physicians have encapsulated knowledge that leads to quick clinical reasoning, yet trainees work with isolated basic blocks. One goal of basic science educators is to help students encapsulate these blocks and thus nurture their PID. Teaching psychosocial skills without the sciences will lead to “empty capsules” and ineffectual integration of skills and knowledge.

Dr. Lazarus used the field of anatomy as an example to explain the impact of technology on anatomy curricula. Her group performed a metanalysis showing that anatomy content was taught at every medical school using different modalities, yet all students learned it equally well. She suggested to integrate AI into the anatomy curricula to deliver content more effectively and use the gained time to enhance critical thinking and thus increase PID. Her work demonstrated that AI technology increased student engagement and learner monitoring, but also developed a view that healthcare has singular logic, that individuals fit into categories, and that psychological skills have limitations. In another study, her group identified six themes that were linked to PID: Process, peer relationships, tutor relationships, ethics, tolerance of uncertainty, and exposure. Of those, the tolerance of uncertainly (= managing novelty effectively) was the only theme that negatively impacting PID in students. Using a clinical case, she illustrated the process of uncertainly tolerance: the physician’s response to a patient’s symptom (the uncertain stimulus) is moderated by cognitive (doubt/confidence), emotional (fear/curiosity), and behavioral (avoidance/decisions) influences. Her group stated that basic science education is a powerful moderator of uncertain times that can increase PID; it can increase job satisfaction, independence, creative solutions, and curiosity, and it can decrease burnout, requirement of supervision, difficulty in solving problems (insecurity), and disengagement.

Dr. Lazarus then listed how educators can stimulate students’ uncertainty in the classroom: (1) Transferring knowledge to a real-world example. (2) Gray cases in which parameters are changed. (3) Providing multifaceted perspectives from different specialists. (4) Questioning preconceptions, e.g., show same experiment with different outcomes. She suggested to bring different moderators into the classroom once students have been stimulated, including educator- and student-sourced moderators (e.g., self-reflection, discuss career value, give assessments with more than 1 correct answers, clinical cases without a diagnosis). She explained that it was important to know which moderator(s) to use at what time and what level.  She also listed three cultural literacy pedagogical components: critical incidents (case studies), destabilization (role-play, simulation), iso-immersion (work integrated learning, placements).

Dr. Lazarus concluded that appropriately build-in of PID in the basic science curriculum has advantages: (1) It prepares students better for the realities of their future careers. (2) By creating a classroom fostering uncertainty tolerance, it helps students manage future transitions to work. (3) Uncertainty tolerance will increase students’ ability to detect novelty and promote PID.

The presentation lasted about 40 minutes and a rich discussion followed. Among other topics, questions from the audience addressed: What is the level of PID? How can uncertainty tolerance be coordinated with the necessity of correct answers in USMLE step exams? How do we train faculty to teach uncertainty without undermining their expertise? How does this impact the ratio students to instructors?

1. Rees, C.E. and Monrouxe, L.V., “Who are you and who do you want to be? Key considerations in developing professional identities in medicine.” Medical Journal of Australia, 2018. 209(5): p. 202-203.
2. Monrouxe, L.V and Poole, G., “An onion? Conceptualizing and researching identity.” Medical Education 47.4 (2013): 425-429.
3. Pratt, M.G., Rockmann K.W. and Kaufmann J.B., “Constructing Professional Identity: The Role of Work and Identity Learning Cycles in the Customization of Identity among Medical Residents.” Academy of Management Journal 49 (2006): 235-262.