MasteringAstronomy® educator study evaluates the use of tutorials in course performance at University of Toledo
- Students who were assigned required MasteringAstronomy tutorial homework scored an average of three to five percentage points higher on exams than students who were given the assignments as optional study resources.
- Among students who were assigned required MasteringAstronomy tutorial homework, 53 percent earned an A/B/C compared to 33 percent of students in the section in which the same assignments were optional.
- In the future, Shan plans to include the Mastering tutorial problems as part of required homework rather than offering them as an optional resource.
University of Toledo, Toledo, OH
Survey of Astronomy
MasteringAstronomy®; The Essential Cosmic Perspective, by Bennett, Donahue, Schneider, and Voit; Lecture Tutorials for Introductory Astronomy, by Prather, Slater, Adams, and Brissenden
Kathy Shan, Lecturer
- Established in 1872, the University of Toledo is a public metropolitan research university with approximately 23,000 undergraduate and graduate students.
- The university became a member of the state university system in 1967.
- The University of Toledo and the Medical University of Ohio merged in July 2006 to form the third-largest public university operating budget in the state.
- The university has more than 300 undergraduate, graduate, and professional programs.
- In 2014, 65 percent of students received some type of financial aid.
- The average retention rate for freshmen in 2014 was 66 percent.
About the Course
Lecturer Kathy Shan has taught at the university for over ten years. She currently teaches the Survey of Astronomy course, a three-credit, one semester face-to-face course designed for non-science majors. Students typically come from humanities, education, or business programs. The course is not open to science majors, and they will not receive program credit if they take the course.
Course topics include appearance of the sky, and nature and evolution of the earth, moon, solar system, stars, galaxies, and the universe. It begins with astronomy on the most local scale, with observations made from earth, motions of the stars and planets, seasons, etc., and then moves on to the solar system, stars, and galaxies. Some history of the study of science, the nature of science, the possible origins of the universe, and of life on earth is also discussed. Current events and recent discoveries are included, such as the Hubble Telescope, the recent Mars missions, and the Cassini mission to Saturn. The main goals of this course are for students to have an understanding of the nature of science through the eyes of astronomy and to understand the big ideas in astronomy.
There are no prerequisites for this course, however, some material requires general knowledge of high school level algebra and geometry. Students are expected to understand concepts such as scientific notation, basic algebraic equations and proportions, basic geometrical concepts such as the general characteristics of circles and ellipses, and some mathematical reasoning. Class time is not used to review any math concepts. Students who feel they need extra help with math concepts are instructed to talk with Shan about getting outside tutoring.
Challenges and Goals
Shan believes that learning is not simply the acquisition of correct information. Learning any subject well requires that students integrate new information with their own knowledge and experiences, and delivery of information by itself does not help students to develop their own understanding of the material. She emphasizes to students that she cannot simply transmit knowledge, but that they must take an active role in the learning process, and that rote memorization and recall are not real learning.
Because her goal is that students gain a conceptual understanding of the topics in the course, she feels it is essential that students are prepared for class time, which is primarily spent doing active learning rather than passive participation with students listening and taking notes. Each class meeting is organized in the same way, starting with a mini-lecture or introduction to a concept, followed by a series of questions or activities and tutorials given to the students, and then peer discussion.
To facilitate the goal of students coming to class prepared, Shan began requiring MasteringAstronomy for her course in 2011. Prior to that, Mastering had been optional, with students encouraged to utilize the study area on their own.
Graded components for the Survey of Astronomy course in Spring 2013 included MasteringAstronomy assignments, planetarium activities, in-class activities, three unit exams, and a comprehensive final exam.
MasteringAstronomy homework and quizzes: Each week one or two homework assignments were posted in MasteringAstronomy. Homework assignments included tutorials and activity questions designed to help students learn on their own and develop a better understanding of the course content. In addition, there were weekly Mastering quizzes on the reading and lecture material which were primarily multiple choice questions. The Mastering homework and quizzes were due at the end of the week that the material was covered in lecture. Late assignments were accepted, with the grade dropping by 10 percent each day late, but never below 50 percent. Students could complete the work for half credit up until the end of the semester.
In-class activities: The class was composed of a series of mini-lectures which augmented in-class activities taken from Lecture Tutorials for Introductory Astronomy. The tutorials helped students apply the concepts and develop a deeper understanding of the material. There was at least one tutorial due per week. Students did not submit every lecture tutorial or other activity that was done in class for grading, but they were occasionally required for submission. Tutorials were meant to be completed collaboratively, with students working together in small groups of two to three to discuss concepts and answer the questions.
Clicker quizzes: Daily clicker quizzes based on the reading for that day or on previous material were administered and graded for correctness to help Shan identify any areas of weakness. In addition, there were almost daily clicker questions that were graded for participation only. These in-class activities could not be made up if missed.
Planetarium activities: Planetarium visits and observations were a required component of this course.
Exams: There were three unit midterm exams based on the notes, weekly quizzes, lecture tutorials, in-class activities, and Mastering homework assignments. The exams were multiple choice and were expected to take approximately one hour. Make-up exams were given only in accordance with the university’s missed class policy.
Final exam: The final exam was cumulative and approximately twice as long as the unit exams. It was based on the previous exams, notes, weekly quizzes, and homework assignments. The final exam could not be made up if missed.
In the spring 2013 semester, Shan taught two sections of Survey of Astronomy, both with required MasteringAstronomy. Due to accessibility circumstances in one section, that section was only assigned the Mastering quizzes as required, and the Mastering tutorial homework was optional.
Section 1 had 127 students enrolled. Required MasteringAstronomy assignments consisted of both interactive, tutorial homework, and separate multiple choice Mastering quizzes.
Section 2 had 134 students enrolled at the start of the semester. This class was not assigned required Mastering tutorial homework, but they were required to complete the Mastering multiple choice quizzes. The second section did have access to the tutorial assignments for individual self-study, but the scores for the optional exercises were not part of the course grade.
Both classes met twice each week for one hour 15 minutes. Both classes were given the same lecture notes and lecture activities, and participated in the same peer instruction. Since students met on the same day, the same schedule was in effect, and all students were given the same exams and final exam.
Since one of the sections did not have the required Mastering tutorial homework, the weight given to the individual Mastering quizzes and in-class activities increased because the assessment of course components remained the same.
- 35% Mastering homework/quizzes and in-class activities
- 30% Exams (3)
- 25% Final exam
- 10% Planetarium activities
Results and Data
An evaluation of the grade distributions comparing the two sections was conducted (figure 1). This analysis includes all students, even if they did not take the final exam. Section 1 was assigned the required Mastering tutorial homework in addition to the Mastering quizzes. In that section, 53 percent of students earned an A/B/C in the course compared to 33 percent of students in section 2 with optional Mastering tutorials who earned an A/B/C.
To further evaluate student performance, an analysis was done of individual exam scores by section, with section 1 having the required tutorial homework. Each section completed the same exams. There was not a pretest given in the course and no GPAs were available to compare student ability between sections. The scores for exam 1 show a difference of three percentage points which was not statistically significant at p<.05. The difference in average scores increased for exams 2 and 3, to five percentage points, and was statistically significant. The data shows that while section 1 did have a higher final exam score than section 2, it was not statistically significant.
Students in section 1 were then divided into two groups based on the required MasteringAstronomy (MA) tutorial homework participation. There were 10 required MA tutorial homework assignments. The mean number of these homework assignments skipped in section 1 was one (10 percent of the total number of tutorial assignments). Figure 3 presents the following information:
- 62 percent of students in section 1 attempted all MA tutorial homework assignments, while another 14 percent skipped only one assignment. These students averaged 65 percent on the final exam.
- 23 percent of students in section 1 skipped more than one required tutorial MA assignment, and earned an average of 35 percent on the final exam.
- An additional analysis of final exam scores for only the students who took the final exam (removing students who had a 0 on the final exam), but skipped more than one required tutorial homework assignment, shows those students scored an average of 56 percent on the final exam, still a statistically significant difference compared to the group who attempted all or skipped only one Mastering tutorial homework assignment.
While other factors can impact these results, such as motivation and study skills, Shan observed that students in the section with the required MasteringAstronomy tutorial homework performed better on average on exams and did better overall in the course.
Final course grades with and without required MasteringAstronomy tutorials
Figure 1. Final Course Grades for two sections—Section 1 with Required MasteringAstronomy tutorial homework and Section 2 with Optional MasteringAstronomy tutorial homework, Spring 2013 (section 1, n=125, section 2, n=134)
Comparison of exam scores with and without required MasteringAstronomy tutorials
Figure 2, Comparison of Exam Scores for two sections—section 1 with Required MasteringAstronomy tutorial homework and section 2 with Optional MasteringAstronomy tutorial homework, Spring 2013 (section 1, n=125, section 2, n=134)
Final exam average based on skipped MasteringAstronomy tutorial homework
Figure 3, Final Exam Averages Based on Skipped MasteringAstronomy Tutorial Homework, Section 1 (skipped 0/1 required tutorial assignments, n=96; skipped 2 or more required tutorial assignments, n=29)
The Student Experience
Shan has received positive feedback from students about Mastering. One student in particular commented on the course evaluation, “I liked the MasteringAstronomy material because it helped me study and also helped to bring my grade up.”
Shan believes that students should have a basic understanding of concepts when coming to class so they can participate in activities that help to develop higher level thinking and conceptual understanding. She feels that by providing two types of Mastering homework, students had different resources to help with learning outside the classroom. Mastering quizzes with multiple choice questions tested their understanding of the reading and the concepts. Mastering tutorial homework helped students develop a better understanding of the concepts and how to apply them.
Shan found that students who were required to do the tutorial assignments as part of their course grade performed better in the course and on the exams than students who were given the assignments as optional homework. Shan intends to require both types of Mastering homework in class moving forward.