Student Utilization of Digital Versions of Classroom Lectures

INTRODUCTION

In the last several years there has been an increased use of digitized lectures in medical education, involving audio/screen capture and/or videorecording of lecturer, made available in a web-based format to medical students after delivery of lectures. The positive impact of lecture recordings in individual courses has been reported.^1,2 Student response to recordings in a Molecular Foundations of Medicine course was universally positive and students felt that the availability of lecture recordings aided their studies and reduced stress and anxiety.² However, there have been few studies investigating how the digital recordings are used, subjective effectiveness, or important technical features for lecture recordings across a curriculum. At Michigan State University (MSU), audio recordings of all preclinical medical school lectures have been available to medical students for over 15 years. Beginning in fall 2005, we started recording screen video as well as audio during classroom lectures for some Year 1 basic science courses. The live lectures were recorded digitally using screen and audio recorder software capable of real time content playback, and digital recordings were available through the Internet the day following the lecture. Beginning in spring 2006, asynchronous web-based access to classroom lectures was provided for some Year 2 courses. Beginning in fall 2006, web-based asynchronous access to classroom lectures in both Year 1 and Year 2 courses was routine, because of the positive response by students.

This study was designed to assess student perspectives on the extent and pattern of use of the digital materials, features important to students, and subjective educational benefits. We hypothesized that there would be differences among students in the use of the online lectures in Year 1 courses (discipline-based lecture-based curriculum) and Year 2 courses (modified problem-based learning (PBL) curriculum with more independent study and fewer lectures than in Year 1 courses).

METHODS

Lecture recording methods.

Camtasia Studio (TechSmith, Okemos, MI, http://www.techsmith.com), a low-cost program that records audio and screen content, was used to record classroom lectures. Presentations included one or more of the following digital elements: PowerPoint files, text documents in Windows Journal on a Tablet PC, demonstrations from websites during the lecture, and integration of external video and audio files that enhanced the lecture content. Instructor annotations during the lecture, such as screen arrows, circles, and written text, were recorded in real time. When PowerPoint was used, the title of each PowerPoint slide was automatically inserted into a Table of Contents by the Camtasia Studio software; during playback students could then click on a slide to jump to that point in the presentation. If PowerPoint was not used, the instructor inserted markers/titles after the lecture to generate an online Table of Contents. Technology staff rendered and compressed each file, and prepared it for streaming and for download. The links were usually available within 24 hours after the lecture. Students could adjust the playback speed of the lecture using Windows Media Player.

Student surveys.

Voluntary anonymous surveys (approved by the MSU IRB) were conducted including questions (both Likert scale and open-ended) about the frequency of use of online lectures during the semester, their perceived effectiveness, and the technology used to access such lectures. The surveys involved three cohorts of students (entering classes of 2004, 2005, 2006) surveyed at selected time points to capture feedback after different degrees of experience with online lecture availability as summarized below. The first two surveys were conducted after students had access to online lectures for part of a semester, and not in all courses. We considered that this was not sufficient exposure to be reliable for statistical analysis, but was useful for comparison with the later surveys. The last two surveys were conducted at the end of fall semester 2006, after most lectures in all basic science courses were recorded for a semester.

•Fall 2005 mid-semester – Year 1 students (entering class of 2005; N=106; fall semester lectures recorded for first time in two of the three basic science courses)
•Spring 2006 mid-semester – Year 2 students (entering class of 2004; N=106; pilot use of recordings in Neuromusculoskeletal PBL course; students did not have video recordings for Year 1 courses)
•Fall 2006 end of semester – Year 1 students (entering class of 2006; N=106; most fall semester lectures recorded)
•Fall 2006 end of semester – Year 2 students (entering class of 2005; N=106; most fall semester lectures recorded)

Analysis of comments.

There was analysis (by Dr. Lovell) of written comments, to extract and categorize the comments related to general use of online lectures, type of use in relation to class attendance, type of use in relation to understanding the content, comparison of live and recorded lectures, and technical features of the online lecture system (eg, speed of playback, streaming vs download options).

Statistics.

A chi-square test was used to compare survey results between Year 1 and Year 2 students for the survey data collected at the end of fall semester 2006 (Year 1 students matriculating in 2006; Year 2 students matriculating in 2005). The null hypothesis was that there was no difference between the two groups. The null hypothesis was rejected if p greater than 0.05.

RESULTS

The percentages of student responses for each cohort for most questions are indicated in Table 1. With respect to number of lectures replayed, more students used the lecture recordings in the fall 2006 surveys (over 95%) than in the earlier surveys in semesters when lecture recordings were initiated (80-90%). There was no significant difference (chi-square = 2.85; p = 0.24) in the pattern between the Year 1 and Year 2 students in the fall 2006 surveys.

With respect to the type of use (why/how do you use online lectures?), there was variation among students. Responses in Table 1 show that a greater number of Year 1 students, compared to Year 2 students, used the lecture recordings to selectively review difficult concepts after attending lecture (chi-square = 10.86; p = 0.001 for the fall 2006 survey data). For this survey question, students could check all responses that were appropriate, and the percentages in Table 1 reflect students that selected that option as one of their responses. Analysis of subcategories of responses (different combinations of the three options) provided additional information about lecture attendance. For example, a student who checked ONLY ‘watch after missing lecture’, but not either of the other options (data not listed in Table 1) was assumed to use the online lectures instead of attending class most of the time. For the fall 2006 surveys, the percentage of Year 1 students who selected ONLY ‘watch after missing lecture’ was 18%; for Year 2 students the percentage was 38%. The difference between Year 1 and Year 2 students was significant (chi-square = 8.81; p = 0.003). This suggests that a greater percentage of Year 1 students attended lectures in fall semester; however, the Year 2 students who returned the survey may be biased toward that population who relied more heavily on the digital recordings.

Most students indicated that online lectures were more effective (34% overall) or just as effective (50% overall) than live class lectures, but 14-18% consistently said online lectures were less effective, and many comments indicated that live lectures were extremely important. A significantly greater percentage of Year 2 students perceived that online recordings were more effective (for the fall 2006 surveys, the difference in pattern of the three responses between Year 1 students and Year 2 students was significant, chi-square = 8.84; p = 0.012). Elements that may have contributed to this difference include more variation among Year 2 lectures in the extent of handouts or coursepack material, more lecture style variation among multiple lecturers, and the lower response rate with possible bias in the population responding.

Most students (over 80% overall) used the variable speed playback option, and thought it was very important (Table 1). With respect to the options of streaming video or downloading the lecture files (information obtained through comments), students were variable in their preference, but said it was very useful to have the download option if they were going to be away from internet access or have a connection that was not always reliable for high-speed access.

In the fall 2006 surveys, students were asked about the subjective effects of the “availability of online lectures on exam scores (learning the material better).” For Year 1 students, 88% indicated a subjective positive effect on exam scores, 12% indicated no effect, and 0% indicated a negative effect. For Year 2 students, 75% indicated a subjective positive effect on exam scores, 24% indicated no effect, and 1% indicated a negative effect. There was no significant difference between the two groups (chi-square = 5.64; p = 0.059).

In addition to comments about wanting to have online lecture recordings available the same day as the lecture, representative comments about using online lectures are listed below.

How lectures are used:

•Being able to stop, rewind and playback the parts I want to review to ensure I have the concepts down is very helpful.
•I watch the videos if there is an exam soon and I miss lecture to study.
•I really enjoy the availability of these lectures; I can listen to the lectures at a time when I am ready to learn and at my convenience.
•Recordings are really useful for exam studying at fast playback speeds. I come to lecture intentionally; a recording is ideal for review and clarifications.

Major reasons why online lectures are more effective than live lectures:

•Ability to pause, rewind, replay sections that were not clear in lecture is important, and the feature of finding specific topics in the table of contents is very helpful.
•Recordings provide flexibility and convenience in time.
•Capture of instructor annotations, especially when diagrams or images were used, provides all content information needed.
•Listening to lectures online at fast speed is much more stimulating.

Reasons why online lectures are less effective than live lectures:

•You can’t see the instructor’s gestures.
•Live class lectures are more interactive and visually stimulating.
•I can’t focus as much sitting in front of a computer.

Thus, students indicated that the effectiveness of online lecture recordings was related to key features of the recording and play-back capabilities, as well as having an online video of the content that was delivered.

DISCUSSION

In a summary of research on a variety of Web-based learning options, Cook^3,4 suggested that investigations related to Web-based learning materials should go beyond media-comparative research to determine elements of digital materials that are effective in specific education settings. Increasingly medical schools in the United States are providing students access to recorded classroom lectures, and there have been many discussions about the effectiveness of this approach on student learning and the effects on classroom attendance. A few published studies have looked at some of these issues for medical schools or other educational sites. For example, Pilarski and coworkers.² reported on the impact of lecture recordings in a Molecular Foundations of Medicine first-year course; students felt that the lecture recordings helped them learn course material and reduced stress and anxiety, with no apparent adverse effect on classroom attendance. Billings-Gagliardi & Mazor⁵ concluded that access to electronic materials did not influence students’ choices about lecture attendance. Several studies utilizing medical students in both preclinical and clinical years^6-9 or other educational settings¹⁰ have compared test performance for students attending a live lecture versus students viewing a digital lecture or other digital materials. In general there is no significant difference in outcome (academic performance measure) and the students thought the online materials were as effective as or more effective than the live lectures.

In our experience, preclinical medical students perceived that online lecture recordings available on-demand with variable speed playback were an important component for studying basic science content. Most students in both curricula (Years 1 and 2) rated the online lectures as an important part of their learning strategies, accommodating individual schedules, backgrounds and learning styles, and allowing them to tailor their experience to meet personal learning objectives. This has been also demonstrated by others^2,9. Important features listed by students included variable playback speed, ability to pause and replay sections, effective table of contents, access within 24 hours or less after the lecture, capture of instructor annotations, and availability of multiple delivery formats (eg, streaming, download, iPod and PDA). Variable playback speed was also reported to be important in other studies11. The availability of these features should be considered in selection of technology to record lectures.

In the fall 2006 surveys, there were significant differences between Year 1 and Year 2 students related to specific types of use and effectiveness of lecture recordings. These differences may be related to the year in medical school, the different types of curricula for Years 1 and 2 in the MSU College of Human Medicine, or bias which may have occurred in the lower response rate of Year 2 students. Further studies to clarify the issues contributing to the differences would be helpful in planning for optimal effectiveness of lecture recordings or other digital materials.

Several previous studies, which have investigated the impact of electronic course materials on lecture attendance^{5, 12,13}, indicated that the decisions by students to attend lectures did not appear to be related directly to the availability of online materials. Although our study was not specifically designed to determine effects on lecture attendance, there was a higher percentage of Year 2 students (compared to Year 1 students) who reported watching recordings instead of attending lecture. However, there was no data to support a change in classroom attendance due specifically to the availability of lecture recordings, since the Year 2 students in a PBL curriculum have multiple approaches for learning the basic science content.

A weakness of this study is that we were not able to measure academic outcomes related to student use of online lecture files, or how much of the perceived “learning the material better” was actually realized on exams. However, a number of studies have investigated the effects of digital materials or televised lectures on student performance, and concluded that effectiveness of digital materials was similar to that of traditional methods^3,4,6-8. Investigations of specific features or types of use of lecture recordings or other digital materials that contribute most effectively to student learning are warranted.

CONCLUSIONS

Preclinical medical students perceived that online lecture recordings with variable speed playback and flexible navigation were an important component for studying basic science content. Most students used the online lecture files in addition to attending live lectures, in order to review and master concepts. There were significant differences between Year 1 and Year 2 students for several specific responses, suggesting that issues related to use of electronic materials should be considered for the audience involved. Further analysis of the value of specific features and the patterns of use of the digital recordings will assist with strategic planning for educational technology to improve student learning and retention.

ACKNOWLEDGMENTS

The authors thank many individuals who made the lecture recordings and survey administration possible, including Health Information Technology – Educational Technology unit, Office of Medical Education Research and Development, colleagues in the College of Osteopathic Medicine (the Year 1 basic science courses are taught jointly to M.D. and D.O. students), and teaching faculty who helped initiate the technology applications. Brian Mavis, Ph.D. provided valuable feedback on the manuscript.

REFERENCES

1. Slaby, F. Virtual lectures: a new teaching format for the medical school curriculum, Journal of the International Association of Medical Science Educators. 2006; 14: 23-27.
2. Pilarski, P.P., Alan Johnstone, D., Pettepher, C.C., and Osheroff, N. From music to macromolecules: using rich media/podcast lecture recordings to enhance the preclinical educational experience. Medical Teacher. 2008; 30(6): 630-632.
3. Cook, D.A. Where are we with web-based learning in medical education? Medical Teacher. 2006; 28(7): 594-598.
4. Cook, D.A., Levinson, A.J., Garside, S., Dupras, D.M., Erwin, P.J., and Montori, V.M., Internet-based learning in the health professions: a meta-analysis. Journal of the American Medical Association. 2008; 300(10): 1181-1196.
5. Billings-Gagliardi, S., and Mazor, K.M. Student decisions about lecture attendance: do electronic course materials matter? Academic Medicine. 2007; 82(10 Suppl): S73-6.
6. Davis, J., Crabb, S., Rogers, E., Zamora, J., and Khan, K. Computer-based teaching is as good as face to face lecture-based teaching of evidence based medicine: a randomized controlled trial. Medical Teacher. 2008; 30(3): 302-307.
7. Bertsch, T.F., Callas, P.W., Rubin, A., Caputo, M.P., and Ricci M.A. Effectiveness of lectures attended via interactive video conferencing versus in-person in preparing third-year internal medicine clerkship students for clinical practice examinations (CPX). Teaching and Learning in Medicine. 2006; 19(1): 4-8.
8. Solomon, D.F., Ferenchick, G.S., Laird-Fick, H.S., and Kavanaugh, K. A randomized trial comparing digital and live lecture formats. BMC Medical Education. 2004; 4: 27-32.
9. Ruiz, J.G., Mintzer, M.J., and Leipzig, R.M. The impact of e-learning in medical education. Academic Medicine. 2006; 81(3): 207-212.
10. Hubble, M.W., and Richards, M.E. Paramedic student performance: Comparison of online with on-campus lecture delivery methods. Prehospital and Disaster Medicine. 2006; 24: 261-267.
11. Spencer, S., and Galbraith, J. Asynchronous video-based instruction with variable speed playback: is faster better? In: Barker, P. and Rebelsky, S. (Eds.). Proceedings of World Conference on Educational Multimedia, Hypermedia and Telecommunications. (pages 562-564). Norfolk, VA: Association for the Advancement of Computing in Education. 2002.
12.Bassili, J.N. Promotion and prevention orientations in the choice to attend lectures or watch them online. Journal of Computer Assisted Learning. 2006; 22(6): 444-455.
13. Mattick, K., Crocker, G., and Bligh J. Medical student attendance at non-compulsory lectures. Advances in Health Sciences Education: Theory and Practice. 2007; 12(2): 201-10.