Impact of Podcasting on First-Year Medical Student Note-Writing Skills

INTRODUCTION

As if shrinking financial resources, expanding medical knowledge, and demands for accountability weren’t challenging enough for medical educators, they also face a new cohort of medical students filling their classrooms: “Millennial Generation” students accustomed to multi-tasking and to using technology to choose their own schedules for learning. This new generation of medical students, accustomed to defining their personal space with text messages, videos, music, and web access, will require medical educators to look to new and creative delivery modes—such as podcasting—to offer curricular content.

Almost any audio or video recording that has been saved in a digital format can become a podcast, and thus accessible by computer or available for downloading and transferred to an MP3 player such as an Apple iPod. Podcasts typically use RSS feeds, an acronym for “RDF Site Summary,” but more commonly for “really simple syndication.” RSS feeds allow the publication of frequently updated content such as podcasts, which can then be “pushed” to the user’s own computer.

A podcast is a natural teaching tool, particularly for Millennial Generation students who already own and use MP3 players and multimedia smartphones, and they listen to music and other audio files on their computers and other devices. Podcasting is becoming more common in educational settings, most often simply to re-broadcast a previous lecture. Apple Inc., which clearly recognizes the educational opportunities inherent in podcasting, has partnered with several universities to form iTunes U, a hosted service for colleges and universities that provides student and faculty access to the institution’s audio or video lectures or interviews.

The use of Podcasting in medical education has increased rapidly, with at least ten schools advertising that they use podcasts for teaching content.^1-6 Curricula being taught via podcasts include lecture content as well as demonstration and teaching in surgical procedures.⁷ In addition, medical societies are utilizing podcasting to teach specialty subjects. Saveland and coworkers ⁸ describe the first successful design and implementation of a podcast by a national medical society by the Society of Critical Care Medicine.

The most common use of podcasting in medical schools has been to publish previously given lectures. Boynton et al.⁹ demonstrated that students reviewing a video podcast of a lecture preferred this method and they performed better on written examination of the lecture material.

Some schools have started producing podcast episodes as independent supplemental materials, primarily because they allow students to access information on their own timetable. Some research has assessed podcasting as a curriculum tool outside of medical education, but there is little research confirming that this modality is an effective means of learning for medical students.

The purpose of this project was to research the effectiveness of podcasting in presenting supplemental material to first-year medical students. Previous Objective Structured Clinical Examinations (OSCEs) revealed some struggle in note writing by first-year students. We chose to produce a series of podcasts that taught note-writing skills and to assess the improvement in those skills using OSCEs.

METHODS

The institutional review board approved the following research design. We developed a series of six podcast episodes, each lasting about 15-20 minutes, that address issues related to documentation and writing effective clinical notes. This series, titled “SOAP Note 101,” was designed to supplement the instruction on note writing that first-year students received from faculty mentors in assigned clinics during the Early Clinical Experience (ECE) longitudinal block at Texas Tech University Health Science Center (TTUHSC). One episode was published onto Apple’s iTunes Music Store in each of six weeks from October 30 through December 12, 2006, and they were also posted on TTUHSC website to be available free of charge. These episodes are exclusively audio to ensure their portability, simplify the preparation process, and minimize the file size. Each episode, as listed in Table 1, uses multiple voices and a simulated patient interview followed by commentary on effective documentation of that visit to address the subjective, objective, assessment and plan sections of the patient note.¹¹

For Phase I of the study, conducted during the fall semester of 2006, we recruited 39 students into three self-selected study groups: 13 in Group A, who used TTUHSC-supplied iPods, 14 in Group B, who used their own iPods, and 12 in Group C, who did not commit to accessing the podcasts, but were not prohibited from doing so. Due to the number of TTUHSC-owned iPods, Group A was restricted to the first 14 volunteers, and the size of Group B was held at a consistent number. Although the number of students in Group C was comparable, we did not intend to limit the number of volunteers wishing to be a part of that cohort. No efforts were made to assess student learning styles. Students in both Groups A and B were educated about how to access the podcasts from the iTunes Music Store and transfer files to the iPod. Also during the fall semester, all first-year students were reminded about the availability of the podcasts and told how to find them on iTunes or the TTUHSC website.

In January of 2007 (approximately mid-point of the ECE block), all first-year students were required to participate in a 1-station OSCE experience, which included writing a patient note. The student note writing skills were evaluated from the “Student Note” portion of the OSCE by two physician educators outside of Texas Tech who were blinded to whether the students belonged to one of the three groups. The evaluators used a rubric designed by the course director of the ECE. Significantly, the “Student Note” portion of this OSCE was not used for any graded or summative evaluation of the medical students for the course.

Study participants were asked to fill out a project survey, which included evaluation of their use of podcasts, level of enjoyment of podcasts as educational tools, and retention of concepts taught in the podcasts. Content retention was measured via ten true-false or multiple-choice questions drawn from the published audio content of the podcasts.

For Phase II of the study, conducted in the Spring 2007 semester, we used a dose-response model to assess student outcomes based on the number of podcast episodes to which students listened, recruiting subjects from the same class of first-year students who participated in Phase I of the study. We encouraged all students in the ECE course to enroll in Phase II of the study, and we provided an iPod to any participant needing one; 88 students enrolled in this study phase. Students were encouraged to listen to as many of the podcasts as they wished and accurately report how many they listened to in the project survey. A summative OSCE with a graded “Student Note” was given at the end of the course year in May 2007. The course director graded the clinical note using the same rubric used in Phase 1. All participants were asked to fill out a project survey which included all of the earlier evaluated components (described above).

RESULTS

Of the 39 students who signed consent forms for study Phase I, 21 participants (53.8%) completed the first evaluation instrument, 13 from Group A (TTUHSC iPods), 4 from Group B (personal iPods) and 4 from Group C (control or no commitment). Phase I results indicated that students used the computer as well as iPods to listen to the episodes, especially if they were members of Groups A or B. Students using TTUHSC-supplied iPods especially took advantage of the units, choosing that listening mode 85% of the time. Of the students in Group C, however, none used an iPod to listen to the podcasts. Nine respondents reported listening to all of the sessions, and 14 reported listening to the episodes as a review prior to the OSCE.

Respondents to the survey generally enjoyed the podcasts and appreciated having access to them. A 5-point Likert scale (5=Strongly Agree) designed to assess agreement with several statements describing podcasting in medical education revealed that students agreed most strongly that the podcast episodes were convenient to access (mean=4.0), were helpful in writing an effective OSCE note (mean=4.06), are a valuable teaching tool for medical students (mean=4.33), and are easy to understand (mean=4.44). Students agreed less strongly that they had adequate opportunities to listen to the podcasts (mean=3.67) or that they would seek out podcasts related to their medical education, aside from this project (mean=3.61).

The evaluation of the “Student Note” portion of the OSCE showed that all study groups received essentially the same scores for completeness and quality of their clinical note by the external reviewers. Average scores for each were Group A with 75.6%, group B with 72.3% and group C with 77.3%; these differences were not statistically significant.

All of the 88 students who enrolled in Phase II responded to the second evaluation instrument, including 34 who had been enrolled in Phase I of the study. In this phase only 20 of the participants reported to listening to the podcasts on an iPod or MP3 player, and when asked to rank their preferred listening method, 54% chose a computer over an iPod or other portable listening device. About one-third of the respondents reported listening to the SOAP Note podcasts as a review before their spring OSCE, and all of those students listened to the episodes “all at once.”

On the 5-point Likert scale (5=Strongly Agree) that assessed student opinions about podcasting in medical education, respondents again showed satisfaction with podcasting, but at somewhat lower levels than in the previous survey. Significantly, they still agreed that the podcast episodes are a valuable teaching tool for medical students (mean=4.18), and are easy to understand (mean=4.18). However, a new statement, added to test our hypothesis that use of the podcasts would be higher if the OSCE patient note were graded, garnered the lowest agreement (mean=3.2).

Linear regression analysis of the “Student Note” portion of the OSCE showed that for each podcast listened to, the score of the written note increased by 0.9 percentage points (Figure 1). This result correlated with the participant scores on the 10-item cognitive quiz, which showed an increase of 0.5 percentage point for each heard podcast. The correlations between score and podcast sessions heard were weak (r2 DISCUSSION

The finding that students use their computers rather than the iPods for listening to the podcasts, as well as that students are generally pleased with this form of pedagogy, is consistent with results from educational studies on podcasting in education outside of medicine.¹⁰ Phase II of the project was designed after the results of the initial project were analyzed. We felt that one possible reason that the participants in all study groups had done equally well in the clinical note writing during Phase I of the study was because the “Student Note” portion of the OSCE was formative and ungraded. A summative or graded “Student Note” portion of the OSCE probably did prompt more students to take advantage of the podcasts, although only 32% of participants reported listening to the episodes specifically as a review prior to the OSCE. Though the increase was small, our dose-response model did reveal that as students listened to more podcasts, their OSCE student note grade improved and their cognitive learning scores increased.

In a whitepaper on podcasting in education, Deal suggests podcasting “is only valuable inasmuch as it helps the instructor and students reach their educational goals,” and thus is a function of the educational context in which it is used.¹⁰ We concur with those conclusions, and we further argue that the value of podcasts is very much in the eye of the individual student beholder. Those students who want more control over their time and learning environment like podcasts the most and are more likely to use them. Anecdotal responses and narrative comments suggest that learning styles may be related to satisfaction levels; students who are auditory learners seem to like podcasts better than do those students who are visual or tactile learners.

We further recognize that podcasts represent a passive learning form, where medical students listen to the delivered content. Passive learning, whether delivered digitally or live, remains instructor-driven, rather than learner-driven¹²; as such, podcasts are but one of many means to engage medical students in acquiring and applying knowledge.

Our finding that students are generally pleased with this form of information delivery is likely due to two reasons. First, it allows them control over the time and location to review the materials. The finding that most students listen to podcasts at their own computer shows a conscious decision to listen to the material in the most convenient way. Second, this form of information delivery resonates clearly for a generation of learners who are comfortable with a digital, controllable delivery and with giving continuous partial attention to surrounding stimuli.
Although the six episodes for SOAP Note 101 were prepared specifically for first-year medical students and although both phases of the study have long since been completed, we continue to receive requests for information about these episodes from faculty at our institution and elsewhere and at all levels, including clinical clerkships and residency programs. Statistics from the Texas Tech iTunesU site, which was not in operation during this study period, continues to show more than 10 downloads per week for SOAP Note episodes, suggesting that the content they cover is useful and the format is convenient.

CONCLUSIONS

We caution against the use of podcasting in medical education as a replacement for traditional lectures. Rather, we suggest that podcasts are most valuable to provide teaching content that is supplemental to ongoing curricula and to give students more control over access to their learning environment. Even if students do not actively avail themselves of podcasts, they appreciate their availability and the option of using them. An additional challenge will be finding ways to utilize this medium in a manner that also increases student retention of the material. Encouraging students to produce their own podcasts as an educational exercise, for example, may place them in a more active learning role with potential to increase learning, a project that is currently underway with fourth-year students at our school.¹³

Additional studies on podcasting should assess the added value of student-produced episodes as well as the relationship between learning styles and success or satisfaction with podcasting in medical education.

ACKNOWLEDGEMENTS

We appreciate funding provided by a research grant from the SGEA for this study. We thank Scott Kinkade, MD, MSPH of University of Texas Southwestern Medical Center at Dallas and Mark W. English, MD of Texas A&M Health Science Center College of Medicine who participated as note evaluators.

REFERENCES

1.Ragon, B., and Looney, R.P. Podcasting at the University of Virginia Claude Moore Health Sciences Library. Medical Reference Services Quarterly 2007; 26 (1):17-26.
2.Tripp, J.S., Duvall, S.L., Cowen, D.L., and Kamauu, A.W. Academic podcasting: quality media delivery. AMIA. Annual Symposium Proceedings/AMIA Symposium. 2006;1125.
3.Johnson, L., and Graydon, S. Podcasts – an emerging form of digital publishing. International Journal of Computerized Dentistry. 2006;9(3):205-218.
4.Maag, M. Podcasting: An emerging technology in nursing education. Studies in Health Technology & Informatics. 2006;122:835-836.
5.McCartney, P.R. Podcasting in nursing. American Journal of Maternal Child Nursing. 2006;31(4):270.
6.Rowell, M.R., Corl, F.M., Johnson, P.T., and Fishman, E.K. Internet-based dissemination of educational audiocasts: a primer in podcasting – how to do it. American Journal of Roentgenology. 2006;186(6):1792-1796.
7.Whitehead, D.E., Bray, D., and Harries, M. Not just music but medicine: podcasting surgical procedures in otolaryngology. Clinical Otolaryngology. 2007 Feb.; 32(1):3-6.
8.Savel, R.H., Goldstein, E.B., Perencevich, E.N., and Angood, P.B. The iCritical care podcast: a novel medium for critical care communication and education. Journal of the American Medical Informatics Association. 2007 Jan-Feb.;14 (1):94-99.
9.Boynton, J.R., Johnson, L.A., Nainar, S.M., and Hu, J.C. Portable digital video instruction in predoctoral education of child behavior management. Journal of Dental Education. April;71(4):545-549.
10.Deal A. Podcasting: A teaching with technology whitepaper. Office of Technology Education, Carnegie Mellon University. http://www.cmu.edu/teaching. [Accessed December 1, 2007.]11.McMahon, K.K., Jones, B.G., and Farrell, T.F. SOAP Notes- Podcasts for First Year Medical Students (#774). MedEdPORTAL. www.aamc.org/meded/mededportal, 2008. [Access date: February 17, 2009]12. Bonwell C., Eison, J. Active Learning: Creating Excitement in the Classroom. AEHE-ERIC Higher Education Report No.1. Washington, D.C.: Jossey-Bass. 1991.
13. Jones BG, Dentino A. Texas Tech MedCast Geriatrics Step 2CK Test Prep Series. Available from http://www.ttuhsc.edu/som/fammed/ttmedcast/gerseries/gerstep2ckprep.aspx. [Access date” August 17, 2009]