Mastering Microbiology educator study explores using Mastering’s Student Learning Outcomes feature

EDUCATOR STUDY

Mastering Microbiology educator study explores Student Learning Outcomes feature at Manchester Community College

Key Findings

  • Correlations between Mastering Microbiology homework and exam averages were strong with the strongest correlation between Dynamic Study Modules and the exam averages (r=.71).
  • The Student Learning Outcome feature was used to track student learning in Mastering, and results show a high completion rate and average score on tracked SLOs.

School name
Manchester Community College, Manchester, CT

Course name
Microbiology

Course format
Face to face

Course materials
Modified Mastering Microbiology, and Introduction to Microbiology by Tortora, Funke, and Case

Timeframe
Fall 2015

Educator
Jonathon Morris, Instructor

Results reported by
Betsy Nixon, Pearson Customer Outcomes Analytics Manager

Setting

  • Enrollment: More than 15,000 annually
  • Founded: 1963
  • Gender: 54 percent male/46 percent female
  • Average age: 25
  • Ethnicity: ~45 percent of the credit students from under-represented racial and ethnic groups.
  • Transfer: Guaranteed admission to a Connecticut State University

About the Course

Jonathon Morris is a full-time instructor at Manchester Community College (MCC), teaching Microbiology, Genetics, General Biology, Biotechnology, and Biochemistry. He has been at MCC since 2004, but has been teaching Microbiology since 1994.

Microbiology is a four-credit combined lecture and lab course. The majority of students taking the course are enrolled in Nursing or Health Science programs. Prerequisites are completion of Introduction to Biology and Concepts of Chemistry, both with a C or higher.

This course is designed to provide students with an introduction to microbiology. Students learn the fundamentals of microbiology, survey the world of microbial organisms, and study the interactions between microbes, their hosts, and their effects on the environment. There are also laboratory exercises each week that teach the basics of handling, culturing, and identifying microbes. By the conclusion of this course, students should be able to do the following:

  • Summarize the harmful, as well as the beneficial, effects of microorganisms on their host and the environment;
  • Describe the ecology, genetics, life cycle and biological processes found in microorganisms;
  • Discuss the classifications of microorganisms and the tools used to study these organisms;
  • Explain how microbes cause human disease and the defenses, which are used to suppress microbial diseases, including both those that are a part of the human body as well as those that are medical; and
  • Evaluate and critique the reliability or accuracy of microbiology-related information and references.

Challenges and Goals

Morris’ main goal for students taking this course is that they are continually learning and developing a clear understanding of the concepts and how to apply them, not merely getting the homework answers to earn a high score or to memorize facts for an exam. Because students were expected to be doing homework outside of class to further their learning, Morris wanted to collect and review data during the semester to help evaluate student progress and understanding. His goals included using the data collected to:

  • Gain a better understanding of student progress before the exam and beyond just the recorded homework score;
  • Determine if any students were likely copying or cheating on the homework; and
  • Understand what specific student learning outcomes were problematic to help focus his lecture content.

Morris was using Modified Mastering Microbiology for homework, and beginning Fall 2015 he decided to implement the Mastering Student Learning Outcome (SLO) feature to track student performance throughout the semester. This study reports on the SLO implementation and results from his initial pilot for one small section of the course, and details he learned that he can use in future semesters.

Implementation

Student learning outcomes help an instructor assess skills and understanding related to course objectives. Some of the reasons why they are important to identify and track were presented by the Teaching and Learning Center, Cornell University, and included:

  • They guide decisions about content and coverage.
  • They guide development of evaluation methods (test and assignments).
  • They allow you to undertake broader assessments.
  • They result in convergence of content, assignments, and grading.1

Mastering contains publisher-written student learning outcomes, and the assignable Mastering items are tagged to those learning outcomes. In addition, instructors can add their own course-specific, department-wide, or institution-wide learning outcomes and tag those to the Mastering assignment items as well. Student homework performance can then be tracked against specific learning outcomes at both the individual student and class level.  Three Mastering SLO reports are available for export and include:

  • Summary: high level data for each SLO
  • Item details: data for each Mastering assignment item that is associated with each SLO
  • Student Item details: every student’s score for each Mastering item associated with each SLO

In the pilot, Morris wanted to learn more about using the SLO feature and the diagnostic data that is generated to understand how he could incorporate that information into his planning, both at the individual class-level and overall for the course. He planned to use the SLO data to assess understanding and learning and to compare what students seemed to know on the homework to their exam performance.

For homework, Morris required two Mastering homework assignments per chapter. The Mastering chapter (parent) homework assignments were due as the lecture on that topic was completed. Adaptive Follow-Up (AFU) assignments were generated based on performance on the parent homework. They were designed to address individual gaps for each student. Students who earned a 90 percent or higher on the chapter (parent) homework tested out of the AFU assignment receiving full credit. AFU assignments were required, so students either had to test out of or complete the assignment. In addition, Dynamic Study Modules (DSM) were required assignments due prior to each exam. Points were deducted for any assignments turned in late.

For the lecture portion of the course, Morris administered three exams and a comprehensive final. A separate final was administered for the lab portion.

Students were encouraged to keep a course portfolio during the semester. Points were earned based on a portfolio rubric given to the students. These points were then used to add points to a student’s final course grade.

Learning outcomes were detailed on the student syllabus by unit to help students understand what they should be able to do upon completion of that unit, and that assessments were going to be based on those learning outcomes.

Assessments

  • 40% Exams (4)
  • 20% Safety quiz and lab exam
  • 20% Lab reports
  • 10% Mastering Microbiology homework, written answers to class questions, three                                                articles, class participation and attendance
  • 10% Class portfolio

Results and Data

To understand the relationship between homework and exam performance, a correlation analysis was first completed. A correlation measures the strength of a relationship between two variables, where r is the correlation coefficient. The closer a positive r value is to 1.0, the stronger the correlation. Table 1 shows correlations (r) between different homework categories and the comprehensive final exam and the exam average. For the individual assignment types, correlations were the strongest for Dynamic Study Modules to exams. The overall Mastering (MMB) average to final exam correlation was strong at r=.77; the overall Mastering average to the exam average was also strong at r=.70.

Mastering to exam correlation values

Table 1. Correlation Values (n=17)

Because the data showed strong correlations between the Mastering homework scores and the exams, reviewing the performance on the learning outcomes tagged in Mastering should provide insight into student performance on exams if exam questions represent the same SLOs and are a comparable level of difficulty. However, tagging the exam questions with the same SLOs would yield a more accurate analysis of student performance based on these outcomes and would allow for a stronger comparison between homework and exam performance.

The following analysis looks at the student learning outcomes for Morris’ pilot course. Morris tagged Mastering homework items to five learning outcomes (table 2). Figure 1 shows the Mastering SLO results at the course level. The bars represent the percent of SLO items completed and the line shows the average score for each SLO. This graph can be used to determine if there are any gaps between the amount of work being completed on an SLO and the average score.  For Morris’ pilot, a minimum of an average of 80% of the items for each SLO was completed and scores averaged 86% percent or higher. No big gaps were noted when reviewing the data at the course level. Instructors would want to review these data in more depth if they found that a low percent of the items tagged to an SLO were completed or if a high percent of items tagged to an SLO were completed but students had a low average score.

Student learning outcomes

SLO

Learning Outcomes

1 Students will be knowledgeable of the classifications of microorganisms and the tools used to study these organisms.
2 Be able to evaluate and critique the reliability or accuracy of microbiology related information and references.
3 Be able to understand how microbes cause human disease and about defenses which are used to suppress microbial diseases, including both those that are a part of the human body as well as those that are medical.
4 Become acquainted with the harmful as well as the beneficial effects of microorganisms on their host and the environment.
5 Be familiar with the ecology, genetics, life cycle and biological processes found in microorganisms.

Table 2.

Student learning outcomes – percent completed and average score

Figure 1. (n=17)

Table 3 presents additional data exported from Mastering with performance by chapter for the tagged SLOs. The lowest percent of items completed for SLOs were in Chapters 7 and 8. The lowest scores were for Chapters 4, 6, and 7. Chapter 3 had the lowest number of items tagged to an SLO compared to other chapters. The data seem to indicate that students had the most issues with Chapter 7, with the lowest average score and percent of SLO items completed. That chapter did have the highest number of assigned tagged items (along with Chapter 1). In addition, while students had a high average score for Chapter 8, the completion rate was the lowest of all chapters.

This report provides Morris with more detailed information by chapter and more in-depth analysis of the items to evaluate student performance would need to be completed to better understand the results. However, by reviewing these data, it provides a starting point to further analyze performance and plan future course content. These data can be used to determine if changes are warranted in the number and type of items assigned and tagged to SLOs in upcoming semesters The instructor can also run this report after each chapter during the semester to evaluate how students are progressing and can make changes as needed to address any issues.

Finally, not shown here, instructors can export a report by student, allowing for additional analysis to understand individual student performance.

Student learning outcomes by chapter

Table 3.

Conclusion

Morris decided to implement the Student Learning Outcome feature in Mastering with the goal of utilizing the data generated to better understand student behavior, performance, and overall achievement of the required course outcomes. The results of the pilot semester show that correlations between all of the individual types of Mastering assignments and the overall Mastering score to the comprehensive final and the exam average were strong. This presents evidence that students are likely working through the homework rather than merely copying or cheating to complete it.

The SLO reports showed a high completion rate of tagged SLOs and a high average score on those same SLOs. Based on that report and strong correlations, it provided further evidence that what students learned and understood from the homework translated to their performance on exams. In addition, by monitoring the results, Morris was able to determine if there were gaps in learning or completion for any of the SLOs. Because this was a pilot, it was a small group of students. Morris planned to do additional analysis on his own moving forward and felt that the pilot provided insight about using the diagnostic information in the future to evaluate learning.

___________________________

1https://www.cte.cornell.edu/documents/presentations/Student%20Learning%20Outcomes.pdf