COMMENTARY: Techniques for Effective Teaching of a Medical School Course on Biostatistics and Epidemiology

INTRODUCTION

Faculty presenting curriculum in biostatistics and epidemiology (B&E) to medical students may initially feel that their task is quite a difficult one. After all, students come to medical school to become doctors, and the statistical material and population-based characteristics of B&E make these subjects unlikely candidates for pragmatic relevance to clinical practice, in the minds of some students. Also, many people do not see “statistics” as synonymous with “exciting,” and sometimes the teaching does little to help overcome this preconceived notion. Student attitudes and reactions are not the only barrier to the effective communication of B&E course material, however; faculty must be very accepting of the need to adapt their curriculum and presentation to reflect the settings in which their students will most likely be working, so that students come to see the material as practical and interesting.

This article presents the authors’ approach to teaching biostatistics and epidemiology based on more than two decades of experience, an approach which has resulted in a successful and popular course cited by the medical students for its clinical relevance. It is hoped that this advice will help colleagues. especially beginners, at other institutions.

ADJUSTING TO THE AUDIENCE

Nothing matters more in the teaching of biostatistics and epidemiology in medical schools than a keen awareness of the needs of the audience. When biostatisticians or epidemiologists are running a B&E course for medical students, a common mistake is to fall back upon faculty experiences in courses taken in degree programs designed for the preparation of such specialists. It is not adequate to update the scientific content of introductory courses from B&E degree programs while leaving the orientation unchanged. A course aimed at medical students must be quite different from a course whose audience foresees careers in biostatistics and epidemiology. Even physicians with substantial coursework in these fields sometimes fall prey to the tendency to focus on matters of technical interest or non-clinical examples, forgetting that most physicians don’t have, want, or need their depth in this area.

At our institution decades ago, a course was given which presented a solid traditional systematic survey of introductory biostatistics and epidemiology, with detailed hand calculations or computer analyses performed by students for nearly every statistical method in the curriculum, and often-fictitious examples contrived to match the development of statistical topics. This was typical of medical school coursework in our field at the time. Such a course might be a suitable foundation for introductory students in a school of public health, who have more class hours, some pre-existing interest in B&E, and an expectation of further study beyond the introductory level. We are well aware of this “cultural difference,” because we both teach M.P.H. students, in addition to our medical school students. We went on to change the medical school B&E course curriculum based on our belief that medical students should have a course that might be termed “What Every Physician Needs to Know About Biostatistics and Epidemiology” – which is actually rather different from what, say, M.P.H. candidates need to know. Remember, a minority of physicians will ever be responsible for using statistical packages to perform analyses during their careers in practice, and those who become part of a research team generally have statistically-sophisticated specialists to consult. On the other hand, all physicians need to be able to read and understand the presentation of statistics in articles, for example. Thus, our course exercises focus on real-world examples, almost exclusively excerpted from published literature, and involve discussion of why the statistical method was selected, and how it answers a clinical question.

COURSE GOALS AND OBJECTIVES

Clear goals and objectives are an essential guide for faculty deciding how to teach a course. As mentioned above, the goals and objectives derive from the needs of our audience.

Key concepts of statistical inference, experimental design, and epidemiology together make up a modest yet essential part of the physician’s intellectual toolkit. A firm grasp of these areas improves the ability to assess the strength of the evidence for recommended treatments, and is important for a competent discussion of that evidence with patients. Moreover, a physician might be called upon to explain the epidemiological concepts behind herd immunity or public health interventions, for example, to people unfamiliar with such ideas.

Because of the importance of these topics in the career of the practicing physician, biostatistics and epidemiology are part of medical school curricula in the United States and Canada. The National Board of Medical Examiners indicates expected coverage of these topics in the USMLE Step 1 content list shown in the “Quantitative Methods” subheading at www.nbme.org. We were guided by this NBME topics list in making our list of goals and objectives. We recommend that faculty periodically consult this topics list (or analogous lists in other licensing jurisdictions) in case of changes. Review books for licensing exams should also be consulted. While not necessarily authoritative summaries of statistics course curricula, some of these books reflect the recent experience of test-takers and provide insight into on-going changes in examination topics. A review book is not an appropriate source for a course outline, but if our students are ill-prepared for licensure then our efforts to train them have been inadequate by an important minimal standard. By reviewing USMLE topics as well as boards review books, we have observed such specific changes over the years as the addition of box plots and Kaplan-Meier curves to the topics that should be covered. In addition, our reading of the current medical literature, our long experience with physician-collaborators, and informal feedback from alumni, all contribute to our understanding of the relative importance of various areas of statistics and epidemiology from the clinician’s viewpoint.

Our course has these key goals, as stated in the syllabus distributed to the students:

“1. To promote an understanding of biological and random variability and how these are quantified; to promote an understanding of how statistical comparisons can be made despite these sources of variability using statistical tests, which serve as tools in clinical decision-making and in the interpretation of laboratory results.
2. To acquaint students with key principles and methods of biostatistics and epidemiology that are important for the understanding of published studies and for the assessment of their strengths and weaknesses.”

Based on these key goals, we developed and distribute the following specific course objectives:

“1. As students read the literature, they should be able to interpret graphical or tabular representations of distributions of biological measurements or patient outcomes or characteristics, and make judgments about the associated probabilities.
2. Students should be able to understand the meaning of evidence presented in a journal article in the form of confidence intervals or p-values.
3. Students should be familiar with the uses and interpretation of some of the most common statistical tools that they will encounter in publications.
4. Students should understand that published conclusions may be affected by errors due entirely to sampling fluctuation, and/or due to biases inherent in the design of a study or the sampling plan used.
5. Students should understand the impact of patient characteristics on clinical laboratory results, and be able to do simple calculations concerning the variability in predictive value that would be associated with changing prevalence of disease or risk factors.
6. Students should understand the principles of study design involved in epidemiological research and in clinical trials, and the strengths and weaknesses conferred by various designs.
7. Students should be aware of the ethical issues surrounding studies on people in such contexts as epidemiological studies and clinical trials. (This is a rather limited objective: abuses of the past, such as the Tuskegee Study and Nazi medical experimentation, are used as heinous examples of uncontrolled power over human subjects, which motivates a brief discussion of the origins, purposes, and responsibilities of IRBs, ethics boards, and similar bodies around the world. In other words, students are made aware of IRB approval as an essential step in the conduct of clinical research, and the reasons for this requirement.)
8. Students should be aware of the importance of disease surveillance systems and their relationship to public health and clinical decision-making.
9. Students should be familiar with the steps taken in the investigation of an outbreak or epidemic.
10. Students should be familiar with some of the principal causes of death and be able to account for these patterns of mortality.”

Each class session has specific identifiable objectives indicating what the student should be able to do once the session is complete. Students tend to like this feature of the syllabus, because they can use it as a checklist when they review for exams.

MATERIALS DISTRIBUTED

On the first day of class, a student manual is distributed, which contains the course schedule, including the objectives for each session. All the slides which will be shown in class are also distributed, sorted by session. There are some exceptions, due to copyright restrictions on particular images or the need to have students work through solutions “in real time” when problems are solved in class. The slides distributed in hard copy are available online at the course website as well. The hard-copy student manual also contains all the readings which can legally be duplicated; the student reading is largely journal articles which exemplify the statistical or epidemiological principles under discussion. The online version of the readings is provided in the form of links to the websites containing the articles and posted reading materials, whenever access is unrestricted or our library’s licensing agreement permits such linked access. There is also required reading material which we make available only online, such as information about John Snow’s investigation of a cholera epidemic (http://courses.sph.unc.edu/john_snow/), or the British Medical Journal’s comprehensive series on “Epidemiology for the Uninitiated” ( http://www.bmj.com/collections/epidem/epid.dtl). The student manual also includes previous exams, so the students know what to expect, and can test their knowledge.

There is no required textbook to buy, in view of the plethora of websites providing background reading. It is true that students often want a more fully developed explanation of topics discussed in class than might be provided by their own notes or the slides, so we lend each student a copy of Probability Without Equations: Concepts for Clinicians¹ free of charge. The free loan avoids the conflict of interest that would be posed by a professor in our course requiring and profiting from his own book, and allows us to provide a written exposition of topics at a pace and level that exactly matches many of the lectures. At other institutions, where this parallelism is not an issue, other books might be equally valuable. For example, two suitable, brief, well-written, and inexpensive books to use for such a course might be Dawson’s Easy Interpretation of Biostatistics² followed by Gordis’ Epidemiology.³ In our class, we individually tailor additional reading for those few students who want additional material at a higher level.

CLASS SESSIONS AND WORK EXPECTATIONS

The class schedule reflects about 50 hours of contact time for B&E, a number which has decreased just a bit over the years thanks to essential, consistent support from our university’s administrators. Their educational theories and expertise lead them to believe in the importance of these class hours, and also influenced the placement of the course in the curriculum. B&E is taught at the very end of the second year, after all other coursework has been completed. It was felt that this subject matter would be most relevant when students could foresee a need for it in the immediate future, with their clinical work in the offing. It is widely understood that residents, fellows, and attending physicians expect the students to read and explain articles, and such explanations include issues of study design and statistical significance.

The class sessions are divided between lectures and workshops, with a slight preponderance of lectures. Biostatistical topics are covered first, followed by epidemiological applications, but the division between the two types of material is not very strict. It is hard to learn either topic in isolation, and there is no reason to keep them rigidly separate. Incidentally, over the years we have sometimes put biostatistics first and sometimes epidemiology; student reaction to the order of topics is not strong and is very mixed, with perhaps a moderate preference for biostatistics first. We find the curriculum can work well in either order and have settled on doing biostatistics first because of faculty scheduling issues. In addition, topics seem to us a bit easier to develop in that order, and having epidemiology second allows students to pull up their grades if they are not satisfied with their mark in biostatistics. The reverse – countering a poor epidemiology grade with an outstanding performance in biostatistics – is less practical for many students.

Too much lecture can be stultifying, but lectures can be an effective and efficient way to explain statistical material which students are not likely to master on their own. For example, in our experience, more of our statistical novices are able to understand the principles and applications of logistic regression after a careful, well-practiced, and well-illustrated explanation than would be able to understand it by reading a textbook or website on their own. It is also highly unlikely for individuals or problem-solving teams to originate or derive a statistical test for a particular type of data when given data to analyze independently as part of an exercise. But it is equally true that passively witnessing an explanation of an existing statistical test is a poor learning experience compared to the experience of determining how best to analyze a set of data, and deciding what conclusions to draw from it. So we use a paradigm for our class sessions in which, generally, the first half is a lecture used to explain a statistical or epidemiological principle, and the second half is a “workshop” in which the students solve a problem under our direction. For example, a lecture on sensitivity, specificity, and predictive value is followed by a workshop in which each row of students in a large auditorium receives a worksheet with the same sensitivity and specificity but differing population prevalence rates. Students are then asked whether the test would be useful in their patient population. Since positive and negative predictive values vary according to prevalence, and vary in opposite directions, this makes for a gradient in the answers, along the rows. The students come to realize that the differing prevalence rates give each row a different perspective, and they can remember this point without studying, just by recalling what went on in class. The class gets experience in actually calculating predictive values and in interpreting them, with the motivation being a clinical decision.

For certain topics, we use online work outside of class in lieu of a workshop. For example, the CDC offers fine simulations of investigations of outbreaks. We usually use the one found at http://www2a.cdc.gov/epicasestudies/computerbased/botarg.htm as a homework assignment that the students do at their convenience, rather than as a workshop, following the lecture on infectious disease outbreaks.

We think that student participation is the key to an interesting and effective in-person class session. Even the lecture part should involve many questions posed to the class, with excitement, energy, and humor: What hypotheses are suggested by this map of disease? Does it imply that our region is better than others? Does the difference in illness rates coincide with differences in rates of eating pizza? Would you be willing to take this medicine? Would you give it to the person next to you? Why or why not? (Hey, I thought he was your friend!) Would you use an ELISA test in an AIDS-free population? Why or why not? Instructors can develop the knack for making the lecture develop out of a guided dialog that gives the students a chance to think, play, and be a part of an entertaining and appealing session.

Instructors should also have attention-getting tricks and examples available to make topics vivid and fun whenever possible. For example, the topic of hypothesis testing can be engaging to students, if you carry out the published demonstration involving a two-headed coin⁴. When discussing the importance of examining mortality distributions, we present some in class and have the students guess the source of the particular distribution – correct answers have included the Titanic disaster and a tsunami. Most of the workshops are reviews of published journal articles, and we do not give out the answers. The group has to develop them during the workshop time. One workshop involves review of an article presenting chi square and logistic regression results; another involves interpreting the beta coefficients and p-values of linear regression and correlation. Thus, from our experience we recommend a lecture explaining vividly how and why a method works, followed by a workshop where published results are presented based on that statistical method. The actual performance of calculations is secondary, although for some simple topics (e.g., chi-square, predictive values) it is a skill the students are expected to learn.

We would add that this approach requires a high energy and interest level on the part of the instructors. Selection of the course directors and participants must be uncompromising – for a successful course you cannot accept faculty whose presentations are monotonous. In addition, if the faculty running the course are academic biostatisticians and/or epidemiologists, it is a great help if guest speakers can be added who have experience as field epidemiologists. We are fortunate to have several former Epidemic Intelligence Service Officers from the Centers for Disease Control (CDC) as department members, and they give arresting lectures about their experiences, which the students enjoy very much.

In preparation for class sessions, students are expected to study the material from the previous day and come in with any questions they may have. (We have an open question period scheduled after each class, and students never feel that no one will help them. Some students come just to listen to other students’ questions and the responses.) In many cases, students are expected to read the articles or article excerpts before coming to the workshop. They are also expected to do a self-test quiz about half-way through the course, consisting of questions drawn from previous exams. They are told that their mark on this test is a good predictor of their mark on the actual exams, so that if they get a poor mark on this test, it indicates a need for extra help. We schedule class time for the administration and a thorough review of this exam; if they want to take it under exam conditions, they can do so.

The pace of any course with 175 students – a typical number for us – is inevitably unsatisfactory to some of them. If you teach to the upper echelons of the class, others will not assimilate the material, and you end up failing the latter group in both senses of the word “failure.” If you teach slowly enough to ensure that essentially everyone masters the material, some students are bored. We do not have the staff to teach several sections of the course for students varying in interest and preparation level. We feel that on balance, in a large and diverse class, it is better to bore the best students (who find the material easy) than to panic and inadequately prepare the students who find this material difficult. After all, everyone who has been admitted to medical school should potentially be able to master this material at a reasonable level of competence, and indeed the national board exams require them to do so. Students who come in with extensive directly-relevant coursework are exempt from our course, and if they fall just short of the exemption requirements, or choose not to seek an exemption, then they simply have an easy experience with our course, which we do not see as a tremendous problem. (It should be noted that we do not take attendance.) We would rather focus on “teaching for mastery,” i.e., making sure that all students meet a certain minimal standard of understanding of the material, rather than run a course catering to the needs of the more advanced students.

EVALUATION AND CONCLUSION

Student learning is evaluated by two written exams, one on biostatistics and one on epidemiology. The exams are mainly excerpts from journal articles, accompanied by multiple choice questions asking for conclusions about the study designs and statistical methods employed, and interpretive questions on confidence intervals, power, and other statistical issues. The students are asked to draw conclusions from the article excerpts, from which certain key passages have been excised. Calculations are rarely required on the exam, except for numerical examples involving simple tabular material such as chi square and predictive values.

It is important to offer a high-quality, appropriate exam. Before the exam, faculty proofread each other’s exam questions and independently check the answer keys. The examination is compared with the list of objectives distributed at the outset (see above), to confirm that evaluation is based on the list of things we expect students to be able to do. In this way we are meeting an LCME standard. In addition, our university administration periodically collects lists of competencies from all course directors and ensures that all LCME-mandated competencies are represented appropriately in the curriculum. In reaction to a few recent queries from our school’s faculty about the level and breadth of our course exams, the course director has circulated exams to faculty at other institutions, who confirmed the suitability of these tests. This process is useful in order to avoid the establishment of insular, idiosyncratic, or possibly outdated ideas about exam coverage in a small group of faculty members – a fresh “outsider’s” look at exams is a good idea. Similarly, it is also useful to examine the results of your students on the “Biostatistics and Epidemiology” segment of the USMLE Step 1 results, which are sent to medical school officials. In recent years our students’ results on the boards in our fields do match or exceed the national average.

Immediately after each exam we post the answer key in person and online, and in a classroom session we immediately answer questions about the reasoning by which the correct answer is derived. Typographical errors or ambiguities are resolved at that time, with changes to the list of accepted answers if necessary.

In keeping with the policy of our medical school, the course is graded on a fail/pass/high pass/honors scale. The two exams are averaged, and marks above 70 are passing; marks in the 80s are “high pass” grades; and marks of 90 and above are “honors” grades.

The course itself is evaluated by peer review within the institution, which has been favorable, and by students, who must fill out the online course evaluation in order to view their course grade upon completion. The students rate the course highly. We typically get ratings above 4.5 on a scale of 1 to 5, in which 5 is the topmost score. More important than this “popularity contest” type of feedback are later survey results from students indicating that they felt we prepared them well for boards questions and also prepared them well for residency and clinical practice. The medical school administration conducts those surveys.

Perhaps the most gratifying feedback is the reply to the course evaluation item asking for a response to the statement, “The clinical relevance of the basic science material was clear.” Recent responses were “to a very high degree,” 88 students (55%); “to a considerable degree,” 56 students (35%); “to a moderate degree,” 10 students (6%); “to a small degree,” 4 students (3%); “hardly at all,” 1 student ( less than 1%). This shows that the goal of making this material clinically relevant is eminently achievable. You simply have to make it clinically relevant by remaining acutely aware of the audience you are addressing, and by being mindful of the applications of B&E for that audience. Everything needed for a valuable and valued course proceeds from that awareness, such as the nature of the worked examples employed in workshops, and the “journal clippings” approach to the course examinations.

We have discovered that it can actually be fun to construct and deliver a medical school course on biostatistics and epidemiology. We hope that the descriptive material in this paper concerning our course is helpful to others who are about to embark on the task of teaching this subject in the medical school environment, and we would be happy to help others by answering any questions you may have about our experiences.

REFERENCES

1 Holland, B.K. Probability Without Equations: Concepts for Clinicians. Baltimore: Johns Hopkins University Press. 1998
2 Dawson, G.F. Easy Interpretation of Biostatistics. Philadelphia: Saunders. 2008
3 Gordis, L. Epidemiology (Fourth Edition). Philadelphia: Saunders. 2008
4 Holland, B. K. A Classroom Demonstration of Hypothesis Testing. Teaching Statistics.
2007; 29:71-73.

Related