MasteringChemistry educator study examines progression from Prep to Gen Chem at University of Hawaii at Manoa
- Data indicate there was a strong positive correlation between the MasteringChemistry average scores and exam averages in Preparation for Chemistry.
- Data indicate there was a strong positive correlation between performance in Preparation for Chemistry and General Chemistry I.
- Students who averaged 70 percent or higher on General Chemistry I exams and had taken Prep Chem during the period of this study had earned higher Prep Chem Mastering and exam averages than students scoring below 70 percent on General Chemistry I exams.
- The instructor recommended using Mastering diagnostics to track student performance and identify areas of misconception to review in class or with additional homework.
University of Hawaii at Manoa, Honolulu, HI
Preparation for General Chemistry
MasteringChemistry and Learning Catalytics, Introductory Chemistry, by Russo and Silver
Fall 2014-Fall 2015
Amanda Fenner, Instructor
- Type: Four-year, public university founded in 1907 and part of the University of Hawaii system
- Enrollment: 18,865 (13,689 undergraduate and 5,176 graduate)
- Student population: 36% Asian, 23% Caucasian, 16.7% Native Hawaiian or other Pacific Islander, 14% Multiracial, 7% International, 2% Hispanic, 2% Black or African American, and .3% American Indian or Alaska Native
- Graduation rate: 43.4% for the five-year rate for the cohort starting in 2005
About the Course
Instructor Amanda Fenner taught the Preparation for General Chemistry (Prep Chem) course from Summer 2014 through Spring 2016. During the period of this study, she generally taught one section in the fall, one section in the spring, and a small section in the summer. The course taught was a face-to-face lecture with no required lab and intended for students with no past chemistry experience or who needed review before enrolling in General Chemistry I (Gen Chem). The course provided a background in algebra and elementary concepts of chemistry in preparation for Gen Chem.
The student learning outcomes for Prep Chem were:
- Understand atomic structure and use this information to predict compound formation
- Understand trends in physical and chemical properties of elements based on the periodic table
- Use conversion factors and equation rearrangement to solve algebra problems and perform calculations
- Balance chemical equations, classify reactions, predict products of precipitation reactions
- Understand intermolecular forces in solids, liquids, and gases
- Use the mole concept in solving stoichiometry problems involving solids, liquids, gases and solutions
- Solve acid/base neutralization problems
Challenges and Goals
Students were required to pass a placement test to enroll directly into Gen Chem I at University of Hawaii. However, the placement test was optional, and less than half of the students took it. Instead, many students signed up for the Prep Chem course as the first step. During the period of this study, students who elected to take the placement test but did not pass had to enroll in Prep Chem before they could enroll in Gen Chem. If a student earned a C or higher in Prep Chem, they could then enroll in Gen Chem without taking the placement test. Students who plan to enroll in Gen Chem usually are required to take it as part of their program, so it is important that they are able to complete Gen Chem to continue in their desired area of study.
Fenner found that students came into the Prep Chem course with different backgrounds and levels of knowledge and preparedness so providing resources to help students review and remediate, along with understanding student progress during the semester, was important. Fenner used MasteringChemistry and Learning Catalytics to help facilitate those goals so that students could successfully complete Prep Chem and then enroll in Gen Chem.
When Fenner began teaching at the University of Hawaii in 2013, MasteringChemistry was in use, so she always used it as part of this course. In 2015-16, she added Learning Catalytics to the course. The remainder of course credit came from three exams and a final. The course components for Prep Chem were as follows:
Exams: All exams were administered in-class via paper and pencil. Students could use a scientific calculator and a periodic table. Students were permitted to bring one page of handwritten notes to use during the final exam. There were three mid-term exams and a comprehensive final exam. No exams could be made up. The final exam could be used to replace a student’s lowest normalized score from the mid-term exams, or serve to replace a zero if an exam was missed for a valid and documented reason. For this study, the analysis used an exam average of the three mid-term exams and final.
MasteringChemistry: All Mastering homework became available at least three days before the due date. There were two untimed assignments per chapter with one to two assignments in Mastering per week. The assignments contained approximately 12-17 questions with a mix of open-ended, multiple choice, and tutorial items. Assignments were given post-lecture but were available before lecture so students could work ahead if desired. All homework was due by 11:59 p.m. on its due date. Multiple attempts were allowed, but students lost credit for each incorrect attempt.
There was a 25 percent deduction for every day that an assignment was late, prorated according to when it was turned in. Late penalties were only applied to the questions that were completed late, not the whole assignment. There was no penalty for opening a hint and no penalty for getting a hint wrong, and students were encouraged to use the hints to learn how to do the problem. Students were also encouraged to rework Mastering homework questions after the assignment was submitted as review for the exams. Homework was expected to take about one to two hours outside of class for every hour spent attending class. Fenner told students, “[T]he best way to get better at chemistry is to practice solving problems.”
Learning Catalytics: Fenner added Learning Catalytics (LC) to the course in Spring 2015 and it was accessed through MasteringChemistry. Learning Catalytics was used periodically throughout the semester for individual and small group questions. When activities were planned using LC, Fenner reminded students to bring a wifi-enabled device to class which included smartphones, tablets, or laptops. After class, the LC activities could be accessed so students could review the questions and solutions on their own. LC was used in three ways:
- Answer pre-demo questions;
- Predict results; and
- Follow-up on an in-class demo.
Students had to participate or they did not receive any points. She used LC approximately five times per semester in class. Additionally, if she had time, she used LC for exam review.
Fenner wanted students to understand the importance of practicing problem-solving by doing and reviewing the Mastering homework. She believed the assigned homework would help students develop the type of study habits and skills needed to do better in future chemistry courses. In addition, Fenner used the MasteringChemistry diagnostics to track student performance, determine problem areas, and to develop exams. She would often look to see which questions students struggled with the most and would try to review the concepts in class when possible. Students were encouraged to rework the MasteringChemistry problems and review the Learning Catalytics activities, and she used some of those questions on exams.
- 60% Exams (3)
- 20% Final Exam
- 20% MasteringChemistry and Learning Catalytics
Results and Data
An analysis of results for Prep Chem using data from Fall 2014 through Fall 2015 was conducted. Several factors may have impacted the results. Since the placement test was optional and students self-selected into the Prep Chem course, the skill level of students taking the course was unknown. Some students may have taken a chemistry course in the past, but elected to take Prep Chem feeling they wanted the review, while others may never have had chemistry in the past.
Students who did not take the final exam for the Prep Chem course were considered to have not completed the course and were not included in the following analysis. The number of students who did not take the Prep Chem final exam were as follows: Fall 2014 – 26 (8 percent); Spring 2015 – 12 (7 percent); Fall 2015 – 15 (4 percent).
To understand the relationship between the MasteringChemistry homework and exam performance, a correlation was calculated each semester. Table 1 shows the correlation values for the Prep Chem Mastering homework averages to the Prep Chem exam averages with a strong positive correlation reported each semester. The closer the r value is to 1.0, the stronger the correlation. The corresponding p-value measures the statistical significance/strength of the correlation, where a p-value <.05 indicates a correlation between these two variables.
For both Prep and Gen Chem, the exam average of all exams and the final was used for analysis due to variability in how exams were administered and graded. Data for Gen Chem I from Spring through Fall 2015 was used in the following analysis grouping students who took both the Prep and Gen Chem courses, and students who took Prep Chem but not Gen Chem during the period of study. Scores for Gen Chem for Spring 2016 were not available at the time of analysis.
Students were not required to move directly from Prep Chem to Gen Chem the following semester, and reasons for when students would take the subsequent course may vary. Some students who took Prep Chem during the period of this study may plan to take Gen Chem after Fall 2015; however, only students who took both Prep and Gen Chem during the semesters included in this study are included in the analysis presented in figures 1 and 2. The analysis showed a strong, positive correlation between the Prep and Gen Chem exam averages with r=.66 (p<.001) for students who took Prep Chem in Fall 2014, and r=.7 (p<.001) for students who took Prep Chem in Spring 2015, indicating performance in Prep Chem may be related to subsequent performance in Gen Chem.
Additional analysis was done grouping students who took Prep Chem during the first semester of the study (Fall 2014) and completed Gen Chem by Fall 2015. Students were grouped based on their Gen Chem exam average score, with one group having a Gen Chem exam average of less than 70 percent and the second group having an exam average of 70 percent or higher. The data shows that students who had the higher exam averages in Gen Chem had significantly higher Prep Chem Mastering and exam averages (figure 3).
The goal of Prep Chem was for students to develop the skills and knowledge needed to progress to Gen Chem and to do well in that course, a required curriculum course for many students. By understanding how students were doing in Prep Chem, providing the needed resources to practice problem-solving, and helping students develop required skills, Fenner believed students would be better prepared in Gen Chem, and the findings of this study support that. Further analysis would need to be done to better understand the results.
Correlation of Mastering and exam averages for Prep Chem, Fall 2014-2015
|MasteringChemistry Average||r=.62 (p<.05)||r=.58 (p<.05)||r=.55 (p<.05)|
Table 1, Correlation of Mastering and exam averages for Prep Chem, Fall 2014-2015
Correlation of exam averages for Prep Chem to Gen Chem in Fall 2014
Figure 1, Correlation of exam averages for Prep Chem to Gen Chem in Fall 2014, (n=220), p<.001
Correlation of exam averages for Prep Chem to Gen Chem in Spring 2015
Figure 2, Correlation of exam averages for Prep Chem to Gen Chem in Spring 2015, (n=79), p<.001
Prep Chem performance grouped by Gen Chem exam average in Fall 2014
Figure 3, Prep Chem performance grouped by Gen Chem exam average in Fall 2014, Gen Chem<70% (n=171); ≥70% (n=52)
The Student Experience
Students posted comments online related to the resources provided by Fenner, which included:
- “She [Fenner] provides every possible tool to help you be successful. You just have to try.”
- “You definitely need to study for the tests and do the homework.”
General Chemistry is a curriculum course many students must successfully complete for their program of study. During the period of this study, either a successful placement test score or completion of Prep Chem with a C or higher was required for students to enroll in Gen Chem at University of Hawaii. Since students were not required to take the placement test, many students decided to enroll directly into Prep Chem first, either wanting the review or feeling that they needed the remediation. Fenner’s goal for Prep Chem was to provide students with resources to help develop their problem-solving and study skills, and to monitor and understand their progress throughout the semester. She believed this would help them succeed in Prep Chem and would help them as they moved on to Gen Chem, and she feels that using MasteringChemistry helped her address those goals in the course.