The Pre-Entry Program at UTMSH: Effect on Academic Performance of First-year Medical Students

INTRODUCTION

The University of Texas Medical School at Houston (UTMSH), as an educational institution of The State of Texas, endeavors to attract students who are the best and brightest among the applicant pool while at the same time recruiting a student body of diverse ethnic background that reflects the population of the State of Texas. As a state medical school, a primary goal of UTMSH is to produce physicians that will meet the health care needs of that population. Thus, UTMSH is committed to attracting and retaining students from under-represented populations within the State of Texas including rural, disadvantaged, and minority students (including African-American, Hispanic-American and Native American).

Efforts to increase the percentage of underrepresented minority students in medical schools in the United States started during the period of social activism and the civil rights movement of the late 1960s. In 1968, at the annual meeting of the Association of American Medical Colleges (AAMC), a task force was established to set goals for minority-group enrollment and the strategies by which these goals could be achieved.¹ Nickens, et al.¹ describe the increase in underrepresented minority groups since that meeting as having three phases. In the first phase, there was a rapid increase in minority enrollment from about three percent in 1968 to almost ten percent in 1974. This was followed by a period of stagnation where the absolute number of underrepresented minorities increased due to the increase in the number of accredited medical schools² but their proportion of the total medical school enrollment remained relatively constant at between 8-10 percent. In 1990, the AAMC initiated Project 3000 by 2000, an effort to increase the number of underrepresented minority entering U.S. medical schools each year to 3000 by the year 2000. These renewed efforts to increase minority enrollment resulted in a sharp 27 percent increase over the three-year period from 1990 to 1993.^1,2 In response to these efforts to increase minority enrollment, many medical schools have initiated supplemental programs to recruit and retain minority and disadvantaged students.

Programs for underrepresented minority or educationally disadvantaged students sponsored by medical schools can be generally classified into three types. One type is designed for undergraduate college students to enhance their academic skills and increase their exposure to the practice of medicine. The goals of these undergraduate enrichment programs are to help the students be better candidates for medical school and to improve their chances of admission. They are also frequently recruitment tools for the medical schools. These summer programs usually emphasize academic preparation as well as allow the students to observe physicians in their practice of medicine. A second type of program is a post-baccalaureate program that is designed to help students who have been rejected by medical schools improve their academic credentials so that they can be more competitive in the application process. Most of these programs are usually year-long programs designed to improve the participant’s academic skills, especially in the sciences. The third type of program is designed for students who have already been accepted into medical school but are judged to be at-risk to encounter academic difficulty in medical school, particularly in the basic science years. These pre-entry programs are designed to prepare the students for the rigors of medical school and help ensure that they successfully complete their training. Most pre-entry programs are presented as five-to-six week sessions in the summer prior to the start of the first year.^3-8

The results of studies to determine the effectiveness of pre-entry programs suggest that students who participate in these pre-entry programs encounter less academic difficulty than students who were invited to the programs but did not participate. Richardson and Saffran³ report that students who participated in the pre-entry program at the Medical College of Ohio performed better in all subjects except behavioral science than invitees that did not participate. A report describing the pre-entry program at the Boston University School of Medicine⁴ showed similar results. Despite having lower MCAT scores, participants in the pre-entry program had significantly higher proportions of acceptable grades in two courses than did minority non-participants and slightly higher rates of acceptable grades in six of the nine courses presented in the first year. Kornitzer, et al.⁸ showed that students who participated in the pre-entry program at Mount Sinai School of Medicine had less academic difficulty in their first year of medical school. They propose that these pre-matriculation programs play an important role in the academic success of certain educationally disadvantaged medical students because of the academic preparation they provide for some first-year courses, the opportunity to develop study-skills and confidence, and the relationships that are built with fellow attendees and faculty members.

In 1990, the Pre-Entry Program (PEP) at UTMSH was implemented to assist entering students who were judged likely to encounter academic difficulty in Medical School, particularly in the first year. The PEP is a five-week curriculum presented in the summer prior to matriculation that is designed to prepare the students for the rigors of medical school and help ensure that they successfully complete their training. During these five weeks, the students are presented a review of basic concepts of biomedical sciences (biomolecules, pH, thermodynamics, and kinetics) in addition to instruction in four areas of the basic sciences: human anatomy, biochemistry, neuroscience and physiology. In the biochemistry classes, students are introduced to protein structure, enzymatic catalysis, and intermediary metabolism, focusing on the generation of energy by catabolism of carbohydrates, lipids and proteins. In anatomy, the objectives are to cover basic aspects of musculoskeletal anatomy and cardiopulmonary systems. After introduction to the basic organization of the body, students dissect the muscles of the back and observe the spinal cord, spinal nerves and meningeal coverings. They then study the thorax, focusing on the lungs, heart, and great vessels and sympathetic chain. The physiology sessions introduce the students to the concept of homeostasis and the role of membranes in maintaining the composition of the various body fluid compartments. Students also study the concepts of forces and flows in the body using the cardiovascular system as an example. The neuroscience presentations cover the basic biophysical aspects of neuronal signaling, including the ionic basis of resting potentials, production of action potentials and their propagation along axons, synaptic organization, mechanisms of neurotransmitter release and postsynaptic response, and the integration of responses from multiple presynaptic cells. The students also participate in workshops on ethics and professionalism, improving study skills, managing the stress of medical school, and taking patient histories (from a standardized patient). These introductory courses and workshops are designed to help the student adjust to the rigors of the academic program of medical school.

Entering students are invited to participate in the PEP after they have been accepted into UTMSH. The criteria for selection include a low Medical College Admission Test (MCAT) score or grade point average (GPA) from their undergraduate institutions, a non-science major, a lapse in education since graduation from college, an older age, and attendance at a non-selective undergraduate institution.

The PEP requires a significant commitment on the part of the faculty and staff who present the program as well as on the part of the entering medical students, who sacrifice time that either could have been spent in summer employment or vacation. Demands on the time of medical school faculty are increasing and time that can be devoted to teaching is at a premium. Therefore, it is imperative to determine whether the resources devoted to the PEP are effective in decreasing the incidence of academic difficulty for those students who participate in it.

Following the first five years of its implementation, an initial evaluation of the PEP suggested that those who attended the PEP had better academic performance in their first-year classes and a lower attrition rate than those who declined the invitation to attend.⁵ Since the time of that initial evaluation, the PEP has evolved as faculty members involved in the program have changed; thus a new and updated study was conducted to better understand the impact of the PEP on the academic performance and attrition of first-year medical students at UTMSH.

Purpose of the Study

The purpose of this study was to describe the effect of the PEP on the academic achievement of medical students during their first year of medical school. This study addressed two research questions: 1) How did the performance of first-year medical students who attended the PEP compare with those who were invited to the Program but declined to attend? and 2) How did the rates of attrition from the first-year class compare between those who attended the PEP and those who were invited but declined to attend?

The study investigated the following non-directional research hypotheses: 1) There is a statistically significant difference in the percentage of students with unsatisfactory performance in the first year of medical school between those who attended the PEP and those who were invited but declined to attend; 2) there is a statistically significant difference in the performance on the NBME Subject Examinations in Biochemistry, Gross Anatomy, and Physiology between first-year medical students who attended the PEP and those who were invited but declined to attend; and 3) there is a statistically significant difference in the rates of attrition from the first-year class between students who attended the PEP and those who were invited but declined to attend.

MATERIALS AND METHODS

The participants in this study were those students who were accepted into UTMSH and were invited to participate in the PEP between 1999 and 2005. These students were invited to the program because they were judged likely to encounter academic difficulty in medical school due to their low MCAT score, low undergraduate GPA, a non-science major, a lapse in education since graduation from college, an older age, and/or attendance at a non-selective undergraduate institution. The participants were divided into two groups, (1) those who accepted the invitation and attended the PEP (PEP-A; n = 174) and (2) those that declined the invitation and did not attend (PEP-D; n = 81).

Academic achievement in Biochemistry, Gross Anatomy and Physiology were measured by the National Board of Medical Examiners® Subject Examinations in Biochemistry, Gross Anatomy and Physiology. These examinations have been used by the course directors of the respective courses as an end-of-course examination and as an assessment of student achievement in the course. According to the NMBE Subject Examination Program Information Guide,⁹ the Subject Examinations provide national norms and the items in the examination are selected only after extensive review and pretesting. The examinations are proprietary and confidential and were administered under specific procedures established by the NBME. Prior to December, 2004 the scores were reported on a 70/8 scale and the standard error of measurement was 2.5. From December, 2004 onward the scores are reported on a 500/100 scale and the standard error of measurement is 40. The reliability coefficients on the NBME Subject Examinations range between .85 and .89.¹⁰ The Physiology Examination has only been administered in the Physiology course since 2002, and comparisons for the Physiology Examination were limited to the years 2002 through 2005. For statistical analyses, 70/8 scores were converted to 500/100 scores using tables provided by the NBME.

The incidence of unsatisfactory performance and attrition rates from the first-year class were determined from data obtained from the official student records kept by the Registrar of The University of Texas Health Science Center at Houston. Data collected from the official student records included MCAT scores, undergraduate GPA, final grades in first-year courses, and whether the student withdrew or was dismissed, took a leave-of-absence, had to remediate a course, had to repeat the year or opted for the Alternate Pathway (the option to complete the first-year courses over a two-year span). To preserve confidentiality, each participant was assigned a number by the Office of Educational Programs, which collected and collated the archived student records prior to analysis.

The following statistical analyses were performed to compare the demographics of the two groups (PEP-A and PEP-D): t-test for unpaired samples for incoming age, undergraduate GPA and numeric MCAT score; Mann-Whitney U test for MCAT writing score; and Pearson χ² technique for gender and ethnicity. The rates of attrition and incidence of academic difficulty between those who participated in the PEP and those who declined the invitation were compared using the Pearson χ2 technique. Academic achievement in Biochemistry, Gross Anatomy and Physiology as measured by the NBME Subject Examinations were compared by an analysis of covariance using MCAT score as the covariant. Only those students who took the respective NBME Subject Examination in their incoming year were included in the analysis. All p values Systat 12 software (Systat Software, Inc., Richmond, CA.

RESULTS

Overall, 255 students who matriculated at UTMSH were invited to the PEP between entering years 1999-2005. Of these, 174 students accepted the invitation and participated in the PEP; 81 students declined the invitation and did not participate. As shown in Table 1, a comparison of the two groups indicated that there were no statistically significant differences between the accepters and the decliners with respect to their ages at entry, undergraduate GPA, or MCAT scores. There also was no statistically significant difference in the ethnicities of PEP accepters and decliners (χ² = 7.97, d.f. = 7, p = .335) nor in their MCAT writing scores (U = 6721.50, d.f. = 1, p = .363). There was, however, a statistically significantly greater proportion of males who declined the invitation to the PEP (χ² = 5.83, d.f. = 1, p = .016) than those who accepted and participated in the PEP. Thus, the two groups compare well with regards to their demographics.

Attrition from the first-year class was determined from the number of students who withdrew or were dismissed, took a leave-of-absence during the first year, had to repeat the first year due to academic difficulties, or opted for the Alternate Pathway. The rates of unsatisfactory performance were determined from the number of students who received a non-passing grade (marginal performance or failure) in one of the first-year classes. As shown in Table 2, a Pearson chi-square analysis showed no statistically significant difference in the rates of attrition from the first-year class or in the incidence of unsatisfactory performance between those who participated in the PEP and those who declined an invitation to the PEP. Therefore, the research hypotheses that there were statistically significant differences between PEP accepters and decliners in the proportion of students with unsatisfactory performance and in the proportion of students who left the entering first-year class were not accepted.

An analysis of covariance (ANCOVA) was performed on the student’s performance on the NBME Subject Examinations that were given as final exams in the Biochemistry, Gross Anatomy, and Physiology courses taken by the first-year medical students with the student’s MCAT scores used as the covariate. Only those students who took the examination during their entry year were included in the analysis. Tables 3 – 5 show the results of the analyses of covariance.

The results of the ANCOVA presented in Tables 3 and 4 indicate that there is no statistically significant difference between those who participated in the PEP and those who were invited but declined in their performance on the NBME Biochemistry (F = 0.001, p = .975) and Physiology Subject Examinations (F = 0.14, p = .705). Therefore, the research hypotheses that there were statistically significant differences between the PEP acceptors and decliners in their performance on these two examinations are not accepted. The results presented in Table 5, however, show that the performance on the NBME Gross Anatomy Subject Examination by those who declined the invitation to attend the PEP (adjusted mean = 496.02) was statistically significantly higher than those who participated in the PEP (adjusted mean = 474.71, F = 4.29, p = .040). Therefore, the research hypothesis that there is a statistically significant difference between PEP accepters and decliners in their performance on the NBME Gross Anatomy Subject Examination is accepted. Since the effect size obtained, d = 0.29, was less than a third of a standard deviation, it can be argued that this difference is not educationally significant.

DISCUSSION

The purpose of this study was to determine the effect of the Pre-Entry Program (PEP) on the academic achievement of at-risk students in their first year of medical school. Students accepted into UTMSH are invited to the PEP if they have one or more of the following selection criteria: a low Medical College Admission Test (MCAT) score or grade point average (GPA) from their undergraduate institution, a non-science major, a lapse in education since graduation from college, an older age, or have attended a non-selective undergraduate institution. Students who are invited to the PEP may decline the invitation for several reasons including family commitments during the summer prior to matriculation, summer jobs, military service, vacation or travel plans, remediation of undergraduate requirements or a belief that the program would not benefit them. A comparison of the two groups revealed that male invitees to the PEP are more likely to decline the invitation to participate.

The results of this study do not support the hypotheses that the PEP has an effect on the incidence of unsatisfactory performance for at-risk first-year medical students or on the rates of attrition of these students from the first-year class due to dismissal, leave of absence, having to repeat the first-year, or entering the Alternate Pathway. This is contrary to what was reported in an earlier evaluation5 that was done after the first five years of the PEP. Unfortunately, the data from that study were not available to the authors in order to make a valid comparison of the two cohorts.

In order to obtain a standardized measure of the academic achievement of the at-risk first year students, the performance on the National Board of Medical Examiners Subject Examinations in Biochemistry, Gross Anatomy and Physiology were compared between the students who participated in the PEP and those who declined the invitation. These NBME Subject Examinations are utilized as the final examinations in each of the respective first-year courses. This comparison did not reveal any statistically significant differences in the performance of the two groups on the Biochemistry or Physiology examinations. Students who declined the PEP did perform statistically significantly higher on the Gross Anatomy examination, although the low effect size indicated that the difference was not educationally meaningful.

While this study attempted to determine the efficacy of the PEP on the first-year medical school academic performance and whether participation in the PEP affected the attrition rate from the first-year class, it should be noted that there are many factors that can affect academic achievement and whether a student chooses or is forced to alter their path towards a medical degree. These include personal characteristics such as motivation and self-discipline as well as personal situations such as illness, child rearing responsibilities or relationship difficulties that distract the student from his/her academic endeavors. These factors were not addressed in this study. An additional limitation of this study was that no attempt was made to compare subgroups (such as non-science majors, students with a lapse between graduation and entering medical school, socio-economic status, ethnicity) within the students who participated in the PEP and those who were invited but declined to attend. Additional limitations include: 1) the two groups are self-selected and 2) the reasons why those who declined the invitation to the PEP were not obtained to determine if there are common non-academic characteristics that explain their decision not to participate.

A reasonable conclusion of this study is that the results in academic performance and retention of students do not justify the resources put into the PEP. However, a review of the student evaluations of the PEP following the 2005 and 2006 presentations of the program indicate that the students almost unanimously feel that the program is valuable and would recommend it to an incoming student (data not shown). It is their perception that the PEP has better prepared them for the rigors of the first-year of medical school. The results of this study do suggest that the program should remain a voluntary program for incoming at-risk students; at this stage of their education, they can best judge if they need the program or not. It also suggests that further studies should be performed to determine in what ways other than academic the PEP is beneficial to the students.

ACKNOWLEDGEMENTS

The authors gratefully acknowledge the assistance of Elizabeth Green in the Office of Educational Programs in collecting and collating the student data. Without her help this study could not have been performed. The authors also acknowledge the assistance of Pat Caver in the Office of Student Affairs and the course directors and coordinators of the first-year courses in providing the data for this study. The authors also gratefully acknowledge Dr. Allison Ownby in the Office of Educational Programs for her review and helpful comments on this manuscript. This study was approved the UTHSC-Houston Committee for the Protection of Human Subjects (Protocol #HSC-MS-06-0606).

REFERENCES

1. Nickens, H. W., Ready, T. P., and Petersdorf, R. G. Project 3000 by 2000 – racial and ethnic diversity in U.S. medical schools. New England Journal of Medicine. 1994; 331(7):472-476.

   <liJournal of the American Medical Association. 2003; 289(9):1143-1149.

2. Richardson, B. L., and Saffran, M. Effects of a summer preview program of study skills and basic science topics on the academic performance of minority students. Journal of National Medical Association. 1985; 77(6):465-71.
3. Ugbolue, A., Whitley, P. N., and Stevens, P. J. Evaluation of a pre-entrance enrichment program for minority students admitted to medical school. Journal of Medical Education 1987; 62(1):8-16.
4. Varagoor, G., Butler, P.M., Lund, K., Rosenfeld, G.C., and Schultz, S.G. Pre-Entry Program at UT-Houston Medical School: An Assessment, Proceedings of the Eighth International Ottawa Conference on Medical Education and Assessment, Philadelphia, PA, July 12-15, 1998.
5. Thompson, H. C., III, and Weiser, M. A. Support programs for minority students at Ohio University College of Osteopathic Medicine. Academic Medicine. 1999; 74(4):390-392.
6. Williams, M. T. Pre-matriculation program at the University of South Florida College of Medicine. Academic Medicine 1999; 74(4):397-399.
7. Kornitzer, B., Ronan, E., and Rifkin, M. R. Improving the adjustment of educationally disadvantaged students to medical school: the summer enrichment program. Mt Sinai Journal of Medicine. 2005; 72(5):317-21.
8. National Board of Medical Examiners® Subject Examination Program Information Guide, 10th edition, 2004.
9. National Board of Medical Examiners, personal communication, 2006.
   

   Related