MyReadinessTest educator study investigates ways to identify underprepared students at Johnson County Community College

EDUCATOR STUDY

MyReadinessTest educator study investigates ways to identify underprepared students at Johnson County Community College

Key Findings

  • The average MyReadinessTest diagnostic score for students who ended the courses with a final grade of A/B/C was significantly higher than for students who ended the course with a final grade of D/F/W.
  • Students who scored less than the mean score on the MyReadinessTest diagnostic test had a higher rate of withdrawal (39 percent) than students who scored at or above the mean (26 percent).
  • 69 percent of students who scored at or above the mean diagnostic test score on MyReadinessTest earned an A/B/C while 52 percent of students who scored below the mean earned an A/B/C.

School name
Johnson County Community College, Overland Park, KS

Course name
Human Anatomy & Physiology; Microbiology

Course format
Face to face

Course materials
MyReadinessTest

Timeframe
Summer 2013Spring 2015

Educator
Jennifer Menon-Parker, Chair of Life Sciences and Associate Professor

Results reported by
Betsy Nixon, Pearson Customer Outcomes Analytics Manager

Setting

  • Type: Multi-campus, two-year institution
  • Enrollment: 19,091
  • Attendance: 32 percent full time; 68 percent part time  
  • Ethnicity: 22 percent identify as minorities  
  • Financial Aid: 35 percent Pell Grant recipients; another 25 percent receive institutional or state/local grants

About the Course

At the time the study was conducted, Jennifer Menon-Parker was department chair of Life Sciences and an associate professor of Human Physiology and Pathophysiology. She had been teaching Anatomy, Physiology, Microbiology, Pathophysiology, Biology for the non-science major, and Nursing Chemistry courses for over eighteen years at Johnson County Community College (JCCC) and other institutions.

At JCCC, Anatomy and Physiology is a one-semester, five-credit lecture and lab course. Physiology can be taken as a separate four-credit lecture and lab course. It is the study of the functions of the human organism from the chemical and molecular mechanisms that sustain cellular processes through the control systems responsible for homeostasis and the influence of these systems on the cellular function of organ and systems operation. Anatomy can also be taken as a separate four-credit course which studies the gross and microscopic aspects of cells, tissues, and organ systems of the human body with a concentration on a detailed analysis of the structure of each body system.

Microbiology is a one-semester, three-credit general introductory lecture course with a separate two-credit hour lab that provides a background in many areas of microbiology with an emphasis on medical aspects. The structure, physiology, antimicrobial agents, immunology and host-parasite relationship of microorganisms are studied with an emphasis on bacteria. A separate lab course is offered.

The majority of students taking these courses are pursuing nursing or other health-related programs. At Johnson County Community College, 25 percent of total students are enrolled in the nursing assistant and patient care assistant/aid programs, with an additional four percent enrolled in licensed practical/vocational nurse training.

Challenges and Goals

At JCCC, the Biology department wanted to address the retention issue in these courses. Menon-Parker believed that attrition rates suggested that many students entered the courses underprepared, which she feels is one of the most challenging issues faced by educators in STEM courses.  As a result, she began a research project to try to determine the best way to identify underprepared students, since most placement tests do not focus on science students. Research shows that many STEM entrants leave the STEM program several years after starting by either changing majors or leaving college without completing a degree or certificate. A total of 48 percent of bachelor’s degree students and 69 percent of associate’s degree students who entered STEM fields between 2003 and 2009 had left these fields by Spring 2009. Roughly one half of those leaving switched their major to a non-STEM field, and the rest of them left STEM fields by exiting college before earning a degree or certificate.1  

Menon-Parker decided to pilot MyReadinessTest (MRT) in an attempt to determine if the data collected could help identify how prepared students were coming into these science courses.  The MRT diagnostic test provides detailed information on each student’s mastery and application of essential prerequisite reading, writing, and math skills, and on core skills in anatomy and physiology, chemistry, and physics coursework. It then generates a personalized study plan for each student based on his or her performance on the diagnostic test

Implementation

As part of the study, instructors were required to administer the MRT diagnostic test at the start of the semester to determine if the data collected could be used as a leading indicator to inform instructors on student preparedness. In the analysis conducted at Johnson, 439 students were given the MRT diagnostic test with an emphasis on the biological sciences. Prior to testing, these same students had already completed PSAT, SAT, ACT, or placement examinations, but those scores were not available for this analysis.

Beginning in Summer 2013 and continuing through Spring 2015, the MRT diagnostic test was implemented in the following courses:  combined Anatomy and Physiology, stand-alone Human Physiology, and Microbiology. The MRT diagnostic test was required the first week of class, however, no further conditions were placed on faculty, so it was implemented differently across the courses. Some faculty gave extra-credit, while others simply informed students that the outcomes were being used for research purposes to enhance teaching effectiveness and would have no negative impact on student grades. The test data were then collected from the MRT gradebook and compared to course performance.

MRT generates a personalized study plan for each student based on the diagnostic performance to help students remediate in areas where they are weak. Instructors were not required to assign or recommend the study plan that was generated, but students had access to it during the semester if they chose to do it. There was no data or information provided to identify if any students worked on the study plan. Use of the remediation tools in the study plan would be a variable which could impact course outcomes but was not measured in the study.

Results and Data

Since the goal of the analysis was to evaluate if performance on the MRT diagnostic test was a leading indicator of course performance, Menon-Parker aggregated the data across multiple courses. Table 1 shows final grade distributions across the participating students for Anatomy & Physiology, Human Physiology, and Microbiology for Summer 2013 through Spring 2015. A low number of students earned a D or F, but there was a high number of students who withdrew, which was the primary issue causing concern among faculty. During the period of this study, 30 percent of students withdrew from courses they were enrolled in which are necessary to continue in the health and nursing programs. Students can withdraw for many reasons beyond academic, and there is no information available to explain why students left the course.

Final Grade Distributions for Anatomy & Physiology, Human Physiology, and Microbiology

Course Grade Number of Students Percent of Students
A 101 23
B 126 29
C 52 12
D 21 5
F 7 2
W 132 30

Table 1. Summer 2013 through Spring 2015

Figure 1 shows the average MRT diagnostic score for the period of the study based on final course grades. The average MRT diagnostic score for students who finished the course with an A/B/C was 62 percent, just four percentage points higher than those who ended the course with a D/F/W who averaged 58 percent, but it was significant (p<.05).

MyReadinessTest Pre-diagnostic Test Score Based on Course Grades

Figure 1. A/B/C (n=243); D/F/W (n=160), *significant at p<.05

The average MRT diagnostic test score for all students was 61 percent. Grouping students based on the MRT average shows the following results (figure 2):

  • Students who scored below the mean on the MRT diagnostic test withdrew from the course at a higher rate than students who scored at or above the mean (difference of 13 percentage points).
  • Students who scored at or above the mean on the MRT diagnostic test had a higher rate of As in the course  than those who scored below the mean (difference of 33 percentage points).
  • Students who scored at or above the mean on the MRT diagnostic test had a higher rate of earning an A/B/C than those who scored below the mean (difference of 17 percentage points).

Percent of Students earning a course grade grouped by MRT scores

Figure 2. MRT<61% (n=209); MRT ≥61% (n=194)

The data indicate that students who did better on the MRT diagnostic test tended to stay in the course and earn higher grades than students who scored lower. This provided evidence for Menon-Parker that this information could give additional insight to instructors at the beginning of a semester about those students who may be at risk for not completing or succeeding in the course and who may need additional remediation.

Conclusion

Remediation can be accomplished through numerous approaches, however, the first step is to be able to identify students who are coming into the course underprepared. Menon-Parker feels that if students can be identified before starting a course, by using data that show that those students may be at risk of not succeeding in the course, then interventions can be set up, such as a remediation supplement while students are engaged in other coursework. Many colleges are looking to collect and analyze data to understand student preparedness and then determine what changes they can make to assist at-risk students. Menon-Parker feels that it’s important to better understand student preparedness early and to address the specific academic challenges of STEM students to increase success in this area.  

The outcomes of the study conducted at JCCC show that performance on the MRT diagnostic test tended to indicate that if a student was prepared with the necessary skills for the course, they had a better chance of success than less prepared students who may be at risk of dropping the course. The analysis looked at the combined final course grades for Anatomy, Physiology, and Microbiology.  Menon-Parker believes the findings underscore a need for a placement test focused on science and subsequent remediation for students who are at risk before they enroll in these required courses. JCCC is undergoing a redesign to incorporate student self-assessment across the disciplines utilizing a pre/post-test format and will continue to analyze and evaluate course results as they relate to student preparedness and success.

_______________________

1 STEM Attrition: College Students’ Paths Into and Out of STEM Fields, U.S. Department of Education