2016 Spring: Curriculum Design and Evaluation

Curriculum development and evaluation in the modern health sciences environment poses many challenges from the perspective of course developers. Faculty members face an ever-increasing demand to develop integrated courses that incorporate active learning and that are also matched to assessment and program evaluation criteria. To meet this demand and provide quality education programs, faculty members must have a thorough understanding of course design. The IAMSE Spring Series will cover several key topics to help course directors design courses based on carefully planned objectives and expected student outcomes, assess student performance using several types of questions, and perform detailed program evaluation to help gauge course effectiveness and promote successful quality improvement. Session 1 will explore issues related to instructional design with emphasis on creating measurable learning objectives using Blooms Taxonomy, and utilizing a backwards design approach to course development. Session 2 will focus on how to use curriculum mapping to identify content gaps and undesired redundancy within programs. The next two sessions will demonstrate how to perform post-hoc multiple choice item analysis using psychometric data and how to design effective essay questions that assess student knowledge. The final session will concentrate on methods to connect program evaluation to continuous quality improvement.

Back to Archives

March 3, 2016 at 12:00 pm

Instructional Design – Learning Objectives; Backwards Design, Blooms Taxonomy

Presenter: Michelle Yoon, PhD

Michelle YoonMichelle Yoon, Ph.D. is Assistant Dean of Assessment, and Assistant Professor in the Department of Medicine at Uniformed Services University of the Health Sciences (USUHS) in Bethesda, MD. There, she directs program evaluation for UME, teaches Instructional Design/Curriculum Development courses in the Health Professions Education graduate program, chairs the Assessment Committee, and is on the Long Term Career Outcomes Study research team. Her eight years in Medical Education encompass medical school instructional design and curriculum development, program evaluation, student assessment, and educational research.

Dr. Yoon holds a Ph.D. from the University of Virginia in Educational Psychology with minors in 1) Gifted Education, and 2) Educational Research, Statistics, and Evaluation. Prior to joining faculty at USUHS, she was the Instructional Designer at the University of Virginia (UVA) School of Medicine, where she was a Co-PI on a project examining Biochemistry-related preparation and learning needs among first year students. Previous to that, she was a post-doctoral educational researcher and curriculum evaluator at UVA.

Dr. Yoon has published on team-based learning, medical student communication skills, and student learning and development through curriculum change. She has nationally and internationally on topics related to curriculum design, team-based learning, and the learning needs of incoming medical students, and talent development/giftedness in the context of young adults in the health professions.

Prior to receiving her Ph.D., Dr. Yoon was a college and university career counselor, and a vocational rehabilitation counselor, and holds her M. Ed. in Counseling and Higher Education Student Affairs Administration from the University of Virginia.

It is often said that medicine and healthcare is both a science and an art. Likewise, teaching is often referred to as both a science and an art, and the same can said specifically in reference to teaching in medical science education, or health professions education. But before good teaching can occur, it is important to examine the process of developing of learning experiences through systematic instructional design. “What do you want your students to know?” “What do you want them to be able to do?” “How do you gauge or calibrate the complexity of learning?” This web audioseminar will enable participants to thoughtfully answer these questions, by introducing them to the juxtaposed perspectives of the iterative 5-step Dick and Carey model, and Wiggins & McTighe’s Backward Design Model of instruction design (ID). The session will connect these instructional design paradigms to Bloom’s Taxonomy so that participants will be able to think globally about systematic instructional design in the context of cognition or learning domains. Participants will be introduced to designing instruction in a methodical, step-by-step, iterative way that allows them to calibrate instruction involving varying levels of complexity. This session will also allow participants to see how designing good instruction can also begin with using learning objectives and goals as a guiding map to help design specific pieces of instruction. Participants will learn how good instructional design can–and should–bridge, or align, learning objectives with assessment in the instructional process.

Seminar Archive
March 10, 2016 at 12:00 pm

Planning for Curriculum Mapping

Presenter: Dina Kurzweil, PhD

Dina KurzweilDr. Dina Kurzweil has worked in the field of education for over 20 years. Her expertise is in the areas of curriculum development, instructional design, educational technology and faculty support and development. She held a variety of teaching, research, technical consulting, instructional and management positions prior to joining Uniformed Services University of the Health Sciences (USU) where she currently serves as the Director of the Education and Technology Innovation (ETI) Support Office and teaches in the Health Professions Education (HPE) program.

She has co-authored publications and presented on faculty support and instructional design topics, educational technology themes and curriculum development issues. In her current position as the ETI Director she partners with USU faculty and staff to plan and implement the use of innovative technologies and educational techniques in ways that advance the educational mission of USU.

She holds a Master of Science degree from the Syracuse University in Education, a Masters or Arts degree from Syracuse University in Communication and her PhD in Education is from the University of Maryland.

Curriculum mapping acts as a catalyst for opening up lines of communication among all educators. The use of curriculum mapping encourages leadership, faculty and staff to have conversations within their own department, school, and university, increasing collegiality and breaking down silos. This session will introduce the audience to topics and conversations focusing on how curriculum mapping gets started. The session will provide some best practices, challenges, and things to consider before engaging in curriculum mapping. Finally we will discuss some of the hidden aspects and impacts of curriculum mapping.

Seminar Archive
March 17, 2016 at 12:00 pm

Testing your test: Assessing the Quality of Test Items

Presenter: Veronica Michaelsen, MD, PhD

Veronica MIchaelsenDr. Veronica Michaelsen holds an M.D. from the University Of Iowa College Of Medicine as well as an M.Ed. in Instructional Design and a Ph.D. in Education Research, Statistics and Evaluation both from the University Of Virginia. She served as an Assistant Professor of Research in Medical Education at the University Of Virginia for seven years before transitioning to Director of Curriculum Design and Development. In 2012, she took on her current role as Assistant Dean of Evaluation at the George Washington University School of Medicine and Health Sciences where she oversees program evaluation for the MD program. Her research interests include diffusion of innovations in an educational setting and student assessment as a program outcome. The former led her to conduct her dissertation work on the stages of concern of faculty during a curricular change while the latter has resulted in involvement in the clinical skills testing program at the University of Virginia and the implementation of computer based testing at GWUSMHS.

For an assessment to be an acceptable measure of student learning, it must be a valid and reliable and contain high quality test items. But how can you determine if your assessment makes the grade? Psychometric and item analysis data are often provided and allow for a post-hoc analysis of an assessment and yet few of us have formal training on how to interpret these data. This web seminar will review the types of statistics that are generally provided, how these numbers are derived and the ways in which they can be interpreted. Participants will learn about some general guidelines used in determining the quality of test items as well as hear about some of the exceptions to those rules.

Seminar Archive
March 24, 2016 at 12:00 pm

Essay Exams: Beyond Knowledge and Recall of Factual Information

Presenter: Amy Wilson-Delfosse, PhD and Klara Papp, PhD

AmyW-DAmy L. Wilson-Delfosse, PhD is the Associate Dean for Curriculum at Case Western Reserve University School of Medicine and is the immediate Past-President of the International Association of Medical Science Educators. She is published in the fields of pharmacology education, team-based learning and integration of basic science and clinical medicine. She also has particular interest in the identification of strategies to optimize learning within the context of problem-based learning teams. The American Association of Medical Colleges recognized Dr. Wilson-Delfosse’s accomplishments in medical education with the Alpha Omega Alpha Robert J. Glaser Distinguished Teacher Award in 2012.

 

KlaraPappKlara K. Papp, PhD, is Professor and Director of the Center for the Advancement of Medical Learning at Case Western Reserve University School of Medicine in Cleveland, Ohio, USA. The Center supports students and faculty in their efforts to pursue excellence in learning, teaching, and educational scholarship. Dr. Papp completed her PhD in Educational Psychology at the State University of New York at Buffalo. She provides expertise in educational testing and measurement which includes the construction and interpretation of performance-based measures of student performance, including multiple choice tests, essay examinations, behavior checklist rating scales, and patient-based clinical skills exams. She is working with a small and dedicated team of faculty on designing assessments for the curriculum redesign at CWRU that aligns educational goals, instructional methods, and assessments.

A firmly held belief in medical education is that assessment drives learning. Students generally learn what they need to learn to succeed on required assessments. As part of our curriculum redesign a decade ago at Case Western Reserve University SOM, we switched from almost exclusively multiple-choice to open-ended essay type questions. The switch occurred after vigorous debate. The leadership believed that constructed response-type questions promoted more desirable study methods and required conceptual organization and synthesis of information on the students’ part more so than multiple choice. This shift was supported by our faculty. During this webinar, we will review our experience with open-ended assessments as well as the lessons learned using open-ended essay type questions for the assignment of student grades during Foundations of Medicine and Health. We will share a sampling of our faculty’s comments and insights regarding the assessment of student performance using open-ended essay type questions. We will explore evidence behind the commonly held view that open-ended items require that students both search for and retrieve information whereas multiple choice test items require only that students recognize and pick the correct answer out from among a list of incorrect choices (ironically enough, called distractors), i.e., that different assessment formats place different cognitive demands on students.

Seminar Archive
March 31, 2016 at 12:00 pm

Supporting CQI through faculty development and instructional design initiatives

Presenter: Andrea Berry, Julie Bridges-Catalano, MS Ed, and Don Robinson, PhD

Andrea BerryAndrea Berry holds a Master’s degree in Public and Nonprofit Administration and majored in Health Sciences and Administration for her undergraduate degree. She has worked in medical education for over 15 years and has a strong perspective of both the student and faculty experience. Her background and expertise in work-based learning and assessment and program evaluation has allowed Ms. Berry to apply quality improvement principles to her work in faculty development in order to provide relevant and transformative opportunities for the faculty she serves.
As the director of faculty development and instructor in medical education, Ms. Berry has had the opportunity to work side-by-side with faculty as they design and deliver the innovative curriculum and assessment programs. She is particularly interested in the clinical curriculum and has created several regionally recognized programs designed to support physicians in their medical education roles. She serves as a faculty advisor for the Institute for Healthcare Improvement Open School Chapter at UCF and is a member of the AAMC Medical Education Scholarship, Research and Evaluation (MESRE) Faculty Development Committee and Program Evaluation and Faculty Development Special Interest Groups

 

JulieC-BJulie is an instructional designer with 17 years of experience in public school, higher education and medical education. She has 5 years of experience in working with the LCME and SACS accreditation process. As a doctoral candidate in the Instructional Design and Technology program at Old Dominion University, Julie’s research involves integrating technology into medical education.

 

 

 

 

DonRobisonDr. Robison is an instructional designer with 26 years of experience in analysis, design, development and evaluation of learning experiences. He holds a graduate certificate in Modeling and Simulation. Don has designed individual courses, broad curricula, as well as an enterprise distance learning system. He is currently a lead instructional designer in Eastern Virginia Medical School’s MD Program curriculum reform effort.

LCME has emphasized the importance of continuous quality improvement (CQI) to ensure medical schools have plans in place to meet programmatic goals and outcomes. In order to remain compliant, programs must utilize quality improvement principles, such as those used in the broader healthcare context, to evaluate and develop programming. Of particular interest to this talk, CQI must be applied to faculty professional development (LCME Element 4.5) and in giving feedback to faculty (Element 4.4). Inspired by the 2015 Institute for Healthcare Improvement Open School Chapter Congress, this presentation will use a rapid fire Plan-Do-Study-Act (PDSA) format to demonstrate progress made with two essential faculty development programs, Resident as Teacher and an Active Learning series, at the University of Central Florida College of Medicine. Participants will explore the types of data that were utilized to make decisions and changes in the faculty development programs while developing an understanding of the principles that drive CQI.

Seminar Archive