Introducing Critical Appraisal of Biomedical Literature to First-year Medical Students in Histology

INTRODUCTION

As a result of biomedical research and technological advancement, physicians are continuously inundated with new information about preventive measures, diagnostic tests and treatments that will improve the quality of patient care. Consequently, physicians need to be able to critically evaluate the soundness and application of medical research studies in order to distinguish between beneficial preventive, therapeutic, and rehabilitative procedures from those that waste time and money, and may cause more harm then good.¹ A buzzword in medical education that describes the use of contemporary evidence for making patient care decisions is evidence-based medicine (EBM).²

EBM consists of being able to devise questions from clinical problems, finding and evaluating high quality, relevant evidence from the literature, and solving clinical problems with the most current information.³ The life-long practice of EBM, including critical appraisal is a self-directed process that many medical practitioners have either limited or no formal education in research and are not adequately prepared to critically analyze the quality of research they are reading.⁴ More than 50% of medical school graduates report inadequate training in evaluating the methodological quality (i.e., critical appraisal) of biomedical literature.⁵ Specific educational interventions targeting critical appraisal skills are prerequisite for medical students to become effective practitioners of EBM.⁶

The teaching of critical appraisal, a major component of EBM has been increasingly integrated into all levels of medical school curricula.⁷ Several studies have described the teaching of critical appraisal and/or other components of EBM to first-year medical students. Riegelman8 described sixteen-hour, required course at George Washington University School of Medicine, teaching first-year students how to review different types of studies (e.g., prospective studies and controlled clinical trials) and how to apply these data to clinical practice. Results from this study showed that the pre-clinical course in study design and statistics positively affected students’ perceptions of their knowledge of study design and basic statistics. However, the author concluded that the effects of this preclinical course would be ineffective without opportunities for reinforcement in the clinical years.

Stacpoole and coworkers⁹ reported teaching ‘hypothesis-oriented thinking’ to first-year medical students. The authors indicate that clinical diagnosis is a ‘hypothesis-guided process’ in which a patent states a chief complaint, and the physician formulates a set of diagnostic hypotheses that are tested during the history, physical examination and choice of laboratory tests. The entire freshman class served as either study subjects or investigators for hypothesis-driven experimental investigations in a clinical setting. Students’ survey responses of the research program indicated that most considered the hypothesis-driven program to be useful and relevant to their overall education.

Recently, Malathi and colleagues⁷ described an eight-hour EBM course taught to first-year medical students. This program developed from a recommendation by third-year students to introduce EBM material before starting clinical rotations. Student satisfaction surveys and a faculty questionnaire indicated that both the students and the faculty were satisfied with the EBM course, and that the introduction of clinically relevant EBM material to first-year students was practical, feasible, and desirable.

A pilot program in which a ‘motivated’ senior medical student designed and taught EBM and other clinically-focused topics (e.g., physical examination skills) to first-year medical students was recently reported by Josephson and Whelan.¹⁰ The 16-hour, selective course was evaluated favorably by the first-year students, the fourth-year instructor and a faculty advisor. The authors concluded that the course filled an existing need in the first-year curriculum and that the program could be incorporated into diverse academic settings.

Published studies suggest that EBM, including critical appraisal is predominantly taught during the clinical years of medical school.⁷ Dorsch and coworkers¹¹ reported using a multidisciplinary approach for teaching third-year medical students information retrieval and critical appraisal. The ten-week course consisted of medical faculty providing instruction in reading and evaluating research methodology, whereas the library faculty taught online literature searching. Based on formal evaluation and informal feedback, the course showed how the literature search was integral to the critical appraisal of medical literature and the clinical decision-making process.

The effects of two 90-minute interactive seminars on third-year medical students’ knowledge of research design, basic critical appraisal skills and attitudes toward the clinical use of medical literature was examined by Landry and colleagues.¹² Third-year students receiving the intervention were more likely to consider study design important in article selection and use of medical literature critical to patient care decisions compared to the 81 third-year control students. Moreover, the study group did show a significant increase in knowledge of critical appraisal. However, the two seminars did not increase use of medical literature in patient write-ups.

A case study by Fikree and Marsh¹³ reported that the Department of Community Health Sciences of Aga Khan University Medical College taught critical reading of research to third-year students. The first two weeks of the program consisted of three one-hour classroom sessions and four one-hour small group sessions. Thereafter, small groups met monthly to review clinical epidemiological reports. A student questionnaire indicated that all students agreed that critical reading skills were essential, but only 30% strongly agreed that they had mastered the skills. Ninety-seven percent of the students disagreed that year three was too early to start critical reading.

How critical appraisal is taught to medical students during an obstetrics and gynecology rotation was described by Grimes and colleagues.¹⁴ A series of seminars were given to foster critical reading of the literature and evidence-based ward rounds were used to provide practical application of critical appraisal skills. Although, the authors did not have a quantitative assessment of their approach, they indicated student feedback was enthusiastic.

The role of family medicine educators in teaching EBM to third-year students was described by Wadland and coworkers.⁵ However, because clinical faculty had limited experience and training in critical appraisal and research methodology, they attended a 2-day training seminar given by McMaster University faculty before the EBM program was implemented. After clinical faculty training, nine 2- to 3-hour sessions were used to teach third-year students computerized literature searches, medical research paper structure, critical appraisal methods, and oral presentation skills. Survey results from this study showed higher levels of confidence in critical appraisal and research skills. Two cohorts followed into residency reported more appreciation of the instruction and skills obtained from the EBM program than did recent graduates. The authors indicate that the most important finding was that family medicine educators could successfully lead a standardized EBM program across multiple community campuses.

Ghali and colleagues⁶ reported the effects of teaching a four-week EBM course to third-year medical students. Four 90-minute sessions were used to instruct students on clinical question development, MEDLINE searching, critical appraisal, and application of evidence. Third-year students receiving the EBM intervention reported increased self-assessed skills and attitudes, as well as a tendency to use MEDLINE and original research articles to solve clinical problems compared to a control group of third-year students. The authors concluded that the positive impact of their brief EBM intervention was due to active student involvement, clinical relevance and exposure to entire process of evidence-based problem solving.

Teaching critical appraisal to third-year students, using a journal club and letter writing approach during a three-week public health medicine attachment was described by Edwards and coworkers.¹⁵ Small groups of students evaluated a recently published research paper, presented their appraisal to their peers in a journal club, and wrote a letter to the journal editor. Student feedback was ‘overwhelmingly positive’ and 26 letters had been published or accepted for publication. The authors concluded that their approach was an innovative and enjoyable method for teaching critical appraisal and writing skills to medical students.

One study reported teaching EBM throughout the four-year undergraduate medical curriculum.⁴ During the preclinical years, students were introduced to Medical informatics, literature appraisal skills and presentation skills in Library Sciences, Epidemiology and Microbiology, respectively. Students learned how to ask a clinical question based on an individual patient, search and appraise retrieved articles, and apply the information to the patient’s medical problem during their clinical clerkships. The outcome measure developed to assess the program showed an improvement in students’ search skill strategies, and ability to assess accurately the study??bf???bf?s methodology and findings. However, students’ ability to identify relevant studies in order to reach the appropriate conclusion about the question at hand was equivocal. The authors concluded that more data were required before drawing a definitive conclusion about the effectiveness of their integrated approach.

Because undergraduate medical students are expected to learn massive volumes of basic and clinical science information in a relatively brief period, they become proficient at memorizing detailed information found in instructors’ lecture notes, textbooks, and their own hand-written lecture notes. This type of learning lends itself to readily accepting the written and spoken word of authority, rather than the critical appraisal of information. To address this problem, we developed and implemented a resource-efficient method for introducing our first-year medical students to the critical appraisal of biomedical research publications. Our perception of students’ interest toward studying histology was a second reason for conducting the study.

When we implemented our supplemental critical appraisal program into the first-year histology course, the University of Health Sciences College of Osteopathic Medicine (UHS-COM) was utilizing a conventional curriculum in which basic sciences were taught separate from the clinical sciences. Our students seemed to view basic sciences in general and histology in particular as a hurdle to get over in order to enter the clinical component of their medical education. Several studies corroborate this supposition by reporting that medical students’ focus is on acquiring skills to practice clinical medicine¹⁶ and a lack of interest toward basic sciences that are taught in conventional curricular.^17-19 students’ indifference toward histology motivated us toimplement a fresh and practical way of demonstrating histology’s relevance to the field of medicine.

MATERIALS AND METHODS

At the beginning of the first-year histology course, medical students from the graduating class of 2003 (n = 224) at UHS-COM, were randomly divided into 56 groups of four students. A timetable outlining the entire critical appraisal program is listed in Table 1. During the first four weeks of the histology course, students were instructed on the basics of conducting a MEDLINE search and asked to review the biomedical literature for an article that contained histological and clinical elements. Groups were required to submit a copy of the abstract from their selected article for topic approval by the end of week eight. Students’ publication topics are presented in Table 2.

The role of family medicine educators in teaching EBM to third-year students was described by Wadland and coworkers.⁵ However, because clinical faculty had limited experience and training in critical appraisal and research methodology, they attended a 2-day training seminar given by McMaster University faculty before the EBM program was implemented. After clinical faculty training, nine 2- to 3-hour sessions were used to teach third-year students computerized literature searches, medical research paper structure, critical appraisal methods, and oral presentation skills. Survey results from this study showed higher levels of confidence in critical appraisal and research skills. Two cohorts followed into residency reported more appreciation of the instruction and skills obtained from the EBM program than did recent graduates. The authors indicate that the most important finding was that family medicine educators could successfully lead a standardized EBM program across multiple community campuses.

Ghali and colleagues⁶ reported the effects of teaching a four-week EBM course to third-year medical students. Four 90-minute sessions were used to instruct students on clinical question development, MEDLINE searching, critical appraisal, and application of evidence. Third-year students receiving the EBM intervention reported increased self-assessed skills and attitudes, as well as a tendency to use MEDLINE and original research articles to solve clinical problems compared to a control group of third-year students. The authors concluded that the positive impact of their brief EBM intervention was due to active student involvement, clinical relevance and exposure to entire process of evidence-based problem solving.

Teaching critical appraisal to third-year students, using a journal club and letter writing approach during a three-week public health medicine attachment was described by Edwards and coworkers.¹⁵ Small groups of students evaluated a recently published research paper, presented their appraisal to their peers in a journal club, and wrote a letter to the journal editor. Student feedback was ‘overwhelmingly positive’ and 26 letters had been published or accepted for publication. The authors concluded that their approach was an innovative and enjoyable method for teaching critical appraisal and writing skills to medical students.

One study reported teaching EBM throughout the four-year undergraduate medical curriculum.⁴ During the preclinical years, students were introduced to Medical informatics, literature appraisal skills and presentation skills in Library Sciences, Epidemiology and Microbiology, respectively. Students learned how to ask a clinical question based on an individual patient, search and appraise retrieved articles, and apply the information to the patient’s medical problem during their clinical clerkships. The outcome measure developed to assess the program showed an improvement in students’ search skill strategies, and ability to assess accurately the study’s methodology and findings. However, students’ ability to identify relevant studies in order to reach the appropriate conclusion about the question at hand was equivocal. The authors concluded that more data were required before drawing a definitive conclusion about the effectiveness of their integrated approach.

Because undergraduate medical students are expected to learn massive volumes of basic and clinical science information in a relatively brief period, they become proficient at memorizing detailed information found in instructors’ lecture notes, textbooks, and their own hand-written lecture notes. This type of learning lends itself to readily accepting the written and spoken word of authority, rather than the critical appraisal of information. To address this problem, we developed and implemented a resource-efficient method for introducing our first-year medical students to the critical appraisal of biomedical research publications. Our perception of students’ interest toward studying histology was a second reason for conducting the study.

When we implemented our supplemental critical appraisal program into the first-year histology course, the University of Health Sciences College of Osteopathic Medicine (UHS-COM) was utilizing a conventional curriculum in which basic sciences were taught separate from the clinical sciences. Our students seemed to view basic sciences in general and histology in particular as a hurdle to get over in order to enter the clinical component of their medical education. Several studies corroborate this supposition by reporting that medical students’ focus is on acquiring skills to practice clinical medicine¹⁶ and a lack of interest toward basic sciences that are taught in conventional curricula.^17-19 Our students’ indifference toward histology motivated us to implement a fresh and practical way of demonstrating histology’s relevance to the field of medicine.

Materials and Methods

At the beginning of the first-year histology course, medical students from the graduating class of 2003 (n = 224) at UHS-COM, were randomly divided into 56 groups of four students. A timetable outlining the entire critical appraisal program is listed in Table 1. During the first four weeks of the histology course, students were instructed on the basics of conducting a MEDLINE search and asked to review the biomedical literature for an article that contained histological and clinical elements. Groups were required to submit a copy of the abstract from their selected article for topic approval by the end of week eight. Students’ publication topics are presented in Table 2.

Between weeks eight and 12 of the histology course, students were asked to attend two 60-minute seminars on how to critically appraise biomedical publications. The first seminar addressed the typical parts (i.e., abstract, introduction, materials and methods, results, discussion and conclusions) of a biomedical publication and how to critically analyze each component. The second seminar focused on questions to ask when reading diagnostic and treatment studies. A histology instructor gave both seminars during lunch. After the two critical appraisal seminars, groups were asked to meet at a mutually convenient time and answer a set of critical appraisal-type questions (Figure 1). Groups were instructed to type a one-page summary of their approved article by week 12 of the course, using the guidelines provided in Figure 2. By week 15, groups submitted their one-page summary and the answers to the set of critical appraisal questions for instructor analysis.

During weeks 16-19 of the histology course, students revised their one-page summary based on instructor feedback and prepared a four-minute oral presentation for their classmates, faculty and administration. Group oral presentations were given during weeks 20 and 21 of the histology course. Each group member was required to present one of the following four components: introduction, methods, results, and discussion/analysis. After each group’s oral presentation, first-year students, faculty and administration were encouraged to ask questions.

Group presentations were evaluated based on professional appearance and conduct, presentation organization, presentation clarity, analysis and critique, and the quality of the final abstract (Figure 3). These five attributes were worth four points each (i.e., 20 points total) towards the final histology course grade (460 total histology course points). Thus, the entire critical appraisal project was worth 4.3% of their final grade.

To determine whether the students considered our critical appraisal program useful and to address some related questions/issues, we constructed and administered a student survey to the class of 2003 during their senior year of medical school (Figure 4). Because our fourth-year students are completing their clinical clerkships throughout the United States, the survey was distributed via electronic mail. Survey responses were analyzed using Sigma Stat for Windows, version 2.03, SPSS, Inc.

RESULTS

Fifty-one students from the class of 2003 (N = 224) completed and returned our Critical Appraisal Program student survey (Figure 4). Students were asked to rate each of the eight statements using the following scale: ‘indicates disagree strongly’; ‘indicates disagree’; ‘indicates no opinion’; ‘indicates agree’; and ‘indicates strongly agree’. Using frequency analysis, Table 3 summarizes student ratings and the corresponding percent of total respondents for each of the eight statements. Additionally, the mean rating and standard deviation for each statement are also listed in Table 3.

The most populous consensus among nearly all respondents (i.e., 96.1% agreed and strongly agreed) was that the time needed to complete Critical Appraisal program was manageable. Nearly three-quarters of the respondents (74.5%) agreed and strongly agreed that the Critical Appraisal program demonstrated the relevance of histology to medicine, helped in other courses, and fostered collegiality among classmates. Two-thirds of respondents agreed and strongly agreed that the Critical Appraisal program should be expanded throughout the first and second years of medical school. More than one-half of respondents agreed and strongly agreed that the Critical appraisal program had a positive effect on their attitude toward histology. Survey respondents’ rating for whether osteopathic and allopathic counterparts received more critical appraisal training was similarly distributed from disagree to agree strongly. Finally, more than fourth-fifths of respondents (86.3%) agreed and strongly agreed that the Critical appraisal program was worth the effort.

DISCUSSION

Physicians indicate that medical journal reading is the most important activity for keeping updated, ranking above textbooks, colleagues, continued education courses, and pharmaceutical representatives.²⁰ Thus, critical analysis of published information is an essential component to the practice of medicine. The program described herein introduced critical reading of biomedical literature to first-year medical students, while illustrating the applicability of histology to the practice of medicine. Results from student survey data (Table 3) indicate that the course appraisal program was worthwhile and had a positive effect on their attitude toward histology. To motivate students to complete the critical appraisal program, students were graded on their group’s one-page summary and platform presentation.

Each group submitted a drafted version of their one-page summary to the histology instructors for review by week 15 of the course. The drafted summaries were evaluated on the basis of content, organization, distinctiveness from the publication’s abstract, and grammar and spelling. Overall, the quality of the drafted one-page summaries ranged from very well written to poorly written. During weeks 16-19 of the course, each group had the option to revise their critiqued summary before its final submission (weeks 20-21). All groups chose to revise their drafted one-page summary before its final submission. The quality of the summary was worth only four points out of 460 total course points

The quality of group oral presentations (Figure 4) was more impressive than their one-page summaries. Even though only about 3% (i.e., 16 points out of 460 total course points) of total histology course points were associated with each group’s oral presentation, all groups appeared to have put a great amount of time and effort into this part of the critical appraisal program. All groups exhibited a high degree of professionalism, organization, clarity, and critical analysis during their verbal presentations. Students were advised to use either the document camera or the overhead projector for the sake of simplicity, yet over 75% of groups created Microsoft PowerPoint presentations that included scanned images and custom animations. Because no formal training was given on how to use Microsoft PowerPoint, we believe that many students learned to use the program independently during the histology course. An unexpected and unsolicited comment made by many students was that the time and effort required to prepare their four-minute oral presentation gave them a greater understanding and respect for instructors’ 50-minute lectures.

Although student, faculty and administration feedback for this program was enthusiastic, some areas of this program require further consideration for future use. Randomization of our first-year students (N = 224) into groups of four encouraged interactions among students that probably would not have otherwise occurred. Because our first-year students are permanently divided into the same four groups (i.e., A, B, C, and D) during the preclinical years for attending various basic science laboratories, some students expressed difficulty scheduling daytime group meetings when its members were not in the same laboratory group.

Students were encouraged to select a topic of interest, yet it was to contain histological and clinical components. Two issues related to topic selection that were not considered in the creation of this method were study/publication type and date of publication. The two critical appraisal seminars, and the critical appraisal questions and answers assignment were directed toward ‘experimental’ type of biomedical publications. The type of study/publication selected by student groups was variable and included ‘review’ articles, ‘retrospective’ studies, ‘prospective’ studies, ‘clinical trials’, and biomedical ‘experimental’ studies. Allowing students to select any type of publication exposed the first-year class to a variety of biomedical studies that was not addressed during the two critical appraisal seminars. A limitation to our approach was that some students were had difficulty in completing the project within its original format (e.g., data analysis).

Most students selected articles that were published within the last decade. However, several groups selected papers that were decades old. Because physicians should be able to critically evaluate the outcomes of contemporary medical research studies to provide the best possible patient care, we believe one of the greatest limitations of our critical appraisal program was not limiting the publication’s date.

Although we provided our students with only two one-hour seminars on how to critically analyze biomedical publications, there is published data to support its usefulness. Landry and coworkers¹² found that after two 90-minute seminars, third-year medical students were more likely to consider study design important in article selection and the use of medical literature critical to patient care decisions. The format of our two one-hour seminars was kept informal and occurred during lunch. Seminar information was presented in PowerPoint format and placed on our intranet for students unable to attend.

CONCLUSION

Clinicians must be trained to not accept published medical research studies at face value, but to form their own conclusions after careful and critical review. The program described herein is a resource-efficient way to introduce critical reading of biomedical literature to first-year medical students, while illustrating the applicability of microscopic anatomy to the practice of medicine. Our critical appraisal program also gave students a ‘taste’ of scientific writing and an opportunity to learn platform presentation skills.

ACKNOWLEDGEMENTS

The authors would like to thank Ms. Nancy L. Stroud, UHS-COM Interlibrary Loan Specialist, for retrieving and compiling manuscript references.

REFERENCES

1. Shekelle, P.G., Andersson, G., Bombardier, C., Cherkin, D., Deyo, R., Keller, R., Lee, C., Liang, M., Lipscomb, B., Spratt, K., and Weinstein, J. . A brief introduction to the critical reading of the clinical literature. Spine 1994; 19(185): 20285-20315.
2. Elam C., Musick, D., Nora, L., Woods, S., Grigsby, D., and Wilson E. Buzzwords in medical education: what they are, what they mean, and what we are doing about them at the University of Kentucky College of Medicine. Journal of the Kentucky Medical Association 2002; 100(4):145-151.
3. Barnett S.H., Kaiser, S., Morgan, L.K., Sullivan T.J., Siu, A., Rose, D., Rico, M., Smith, L., Schechter, C., Miller, M., and Stagnaro-Green, A. An integrated program for evidence-based medicine in medical school. Mount Sinai Journal of Medicine 2000; 67(2):163-168.
4. Taylor, R., Reeves, B., Ewings, P., Binns, S., Keast, J., and Mears, R. A systematic review of the effectiveness of critical appraisal skills training for clinicians. Medical Education 2000; 34:120-125.
5. Wadland, W.C., Barry, H.C., Farquhar, L., Holzman, C., and White, A. Training medical students in evidence-based medicine: a community campus approach. Family Medicine 1999; 31(10):703-708.
6. Ghali, W.A., Saitz, R., Eskew, A.H., Gupta, M., Quan, H., and Hershman, W.Y. Successful teaching in evidence-based medicine. Medical Education 2000; 34(1):18-22.
7. Srinivasan, M., Weiner, M., Breitfeld, P., Brahmi, F., Dickerson, K., and Weiner G. Early introduction of an evidence-based medicine course to preclinical medical students. Journal of General Internal Medicine 2002; 17(1):58-65.
8. Riegelman, R. K. Effects of teaching first-year medical students skills to read medical literature. Journal of Medical Education 1986; 61: 454-460.
9. Stacpoole, P.W., Fisher, W.R., Flotte, T.R., Geiser, E.A., Theriaque, D.W., and Hutson, A.D. Teaching hypothesis-oriented thinking to medical students: the University of Florida’s clinical investigation program. Academic Medicine 2001; 76(3):287-292.
10. Josephson, S.A. and Whelan A.J. A new first-year course designed and taught by a senior medical student. Academic Medicine 2002; 77(12):1207-1211.
11. Dorsch, J.L., Frasca, M.A., Wilson, M.L., and Tomsic, M.L. A multidisciplinary approach to information and critical appraisal instruction. Bulletin of the Medical Library Association 1990; 78(1): 38-44.
12. Landry, F.J., Pangaro, L., Kroenke, K., Lucey, C., and Herbers, J. A controlled trial of a seminar to improve medical student attitudes towards, knowledge about, and use of the medical literature. Journal of General Internal Medicine 1994; 9:436-439.
13. Fikree, F.F. and Marsh, D.R. Critical appraisal by reading for medical students – a case study from Pakistan. Journal of the Pakistan Medical Association 1996; 46(4): 80-83.
14. Grimes, D.A. Bachicha, J.A., and Learman, L.A. Teaching critical appraisal to medical student in obstetrics and gynecology. Obstetrics and Gynecology 1998; 92(5): 877-882.
15. Edwards, R., White, M., Gray, J. and Fischbacher, C. Use of a journal club and letter-writing exercise to teach critical appraisal to medical undergraduates. Medical Education 2001; 35(7): 691-694.
16. Astin, J., Jenkins, T., and Moore, L. Medical students’ perspective on the teaching of medical statistics in the undergraduate medical curriculum. Statistics in Medicine 2002; 21(7): 1003-1006.
17. West, M., Mennin, S., Kaufman, A. and Galey, W. Medical students’ attitudes toward basic science: influence of a primary care curriculum. Medical Education 1982; 16(4): 188-191.
18. Kaufman, D. and Mann, K. Basic sciences in problem-based learning and conventional curricula: students’ attitudes. Medical Education 1997; 31(3): 177-180.
19. Custers, E.J. and Cate, O.T. Medical students’ attitudes towards and perception of the basic sciences: a comparison between students in the old and the new curriculum at the University of Medical Center Utrecht, the Netherlands. Medical Education 2002; 36(12): 1142-50.
20. Wagner, J.D. and Wagner, S.A. 1992. Keeping abreast of the medical/dental literature: a simplified approach. Journal of Oral and Maxillofacial Surgery 1992; 50: 163-168.