MasteringA&P® educator study reports on implementation results at University of Texas at Tyler
- A higher percent of students earned an A, B, C in A&P I and II after MasteringA&P was implemented.
- The exam average for students was higher after MasteringA&P was implemented.
- Students who worked more of the multiple types of MasteringA&P assignments in A&P I and II had a higher exam average than students who worked fewer assignments.
- A recommendation from the instructor is to utilize the different resources in MasteringA&P to create multiple types of assignments to engage students and to provide different and more opportunities for learning.
University of Texas at Tyler, Tyler, TX
Anatomy and Physiology I and II
MasteringA&P; Human Anatomy & Physiology by Amerman
Suzanne Pundt, Senior Lecturer/Anatomy and Physiology Coordinator
- The University of Texas (UT) at Tyler is part of the University of Texas System that includes 14 institutions located throughout the state.
- Founded in 1971, UT Tyler enrolls more than 8,500 students across multiple campuses.
- UT Tyler’s students represent 144 Texas counties, 50 states, and 52 countries.
- Nursing students have a pass rate of 95.6 percent on state licensure exams.
- The six year graduation rate in 2013 was 44 percent.
About the Course
Senior Lecturer Suzanne Pundt has taught at UT Tyler since November 1982. She started as the assistant director of the Clinical Laboratory Program, but when that program closed, she began teaching biology courses. She currently teaches the Anatomy and Physiology courses.
Anatomy and Physiology (A&P) is a two-semester sequence providing an overview of the structure and function of the human body. The course is designed for nursing students and related health areas, and students take a concurrent lab. Both A&P I and II are prerequisites for the nursing program. Because of the number of students seeking admission to the nursing program, it is recommended that students earn an A or B in the A&P courses in order to be competitive in the admission process.
Table 1 shows the course objectives for each of the A&P courses.
Challenges and Goals
Pundt finds that students procrastinate working on their homework and then run out of time as it becomes due. She feels that many students either do not have effective study skills or are unwilling to spend the time necessary to learn the course material on their own. To try to address this, Pundt adopted MasteringA&P in fall 2015 to provide more interactive learning opportunities in an attempt to reach both types of students. She felt that for students with poor study skills, Mastering walks them through the material in an organized and engaging way. In addition, they can repeat the activities if needed, and they have access to additional study material. Otherwise, many of these students would only read the PowerPoint files and think that they were adequately prepared for the exams.
For students who aren’t motivated to work on their own, assigning required homework forces them to spend time engaging with the course materials because they tend to do the assignments to earn the points. She finds these students are more likely to attempt the homework if it has course credit attached to it. Without the ability to assign graded homework, she feels there are students who would not use Mastering at all. This is supported by a study published in the International Journal for the Scholarship of Teaching and Learning which looked at student attitudes towards online homework. The study found that, “…although most students felt that [online] homework was useful, most admit that they would not do it unless required.”
Given the challenges and goals identified, Pundt engaged in this study to begin to evaluate the relationship between participation and performance on MasteringA&P assignments and the course.
The lecture portion of class focuses on physiology, and the lab primarily discusses anatomy. The course includes the following components:
Exams: Four exams are administered with two in-class exams taken via clicker (exams 2 and 4) and two exams administered online in Blackboard (exams 1 and 3). The online exams are timed with one and a half minutes allotted per multiple choice question and four minutes per essay question. Exams 2 and 3 contain approximately 10 percent of content from the previous unit. The final exam includes 30 percent of content from the previous three units, and the remainder from material covered after Exam 3. To enhance test security for the online exams (exams 1 and 3), students are required to use Tegrity proctoring software that is available from the University. Students record themselves taking their exam, and the video also captures their computer screen to verify student identification and to ensure they are not going to other sites. This video is automatically made available to the instructor for review.
Exam questions are multiple-choice and short essay and limited to material from the textbook, with the majority of that content having been covered in lecture. Two essay questions per exam are based on discussion board questions. Following each exam, the questions answered correctly by 30 percent or less of the class are posted on Blackboard as retake questions, and students have one opportunity to answer them again using any information source other than another student and with no time limit. A point is added back to the raw exam score for each retake question answered correctly. This provides an opportunity for every student to earn extra credit, but it is only available if the class’s raw exam average is less than 70 percent.
Weekly Quizzes: A quiz is assigned every Wednesday, available from 9 a.m. to 9 p.m. on Blackboard. It covers material from the prior Tuesday and Thursday classes. The number of quizzes per semester varies, but only the top 13–15 scores are calculated as part of the total course grade. Quizzes are limited to a maximum of two minutes per question (points are deducted if the limit is exceeded).
MasteringA&P Assignments: For each chapter, there are three types of MasteringA&P assignments: chapter, Knewton Adaptive Follow-Up (AFU), and Dynamic Study Module (DSM) homework. The assignments must be completed by the posted due date for full credit. Chapter homework and AFU assignments may be completed after the due date with penalties. DSMs cannot be completed after the due date, so no credit is given if not completed on time. All MasteringA&P assignments are due the day before the corresponding exam, with all assignments having the same due date.
Pundt has considered having different due dates for the different types of assignments, but decided against it in favor of simple, consistent deadlines. She says, “All assignments are due the day before the exam at 9 p.m., so it is easy for them to remember. Then it is up to them to pace their work.” Homework is expected to take students approximately one to two hours outside of class each week. Pundt uses the MasteringA&P gradebook to review student performance prior to class and determine the most problematic questions from the homework to review those concepts with students.
For MasteringA&P chapter homework, students have two attempts but have a 25 percent deduction in course credit if the first answer is incorrect. For Adaptive Follow-Up assignments, which are generated for each student based on their performance on the chapter homework, students earning 95 percent or higher on the chapter homework test out of and automatically receive full credit for the corresponding AFU assignment. There is a 20 percent deduction if the AFU assignment is late.
The following applies to both the weekly quizzes and MasteringA&P assignments:
- Students may use lecture notes or the textbook; however, the quizzes (but not the MasteringA&P assignments) are timed so students must know the material to do well.
- Students may not receive assistance from other students or any other reference material, including Internet sources.
- Students may not share the contents of a weekly quiz with other students.
Class Participation (Clicker): Students can earn points during lecture by answering clicker questions. One point is given for each correct answer and half a point for each incorrect answer. If students forget to bring the clicker or have technical problems during class, they may answer the questions on paper. Only a maximum of 75 percent of the available points can be earned if turned in by paper.
- 45% Exams (3)
- 20% Final exam
- 15% Weekly quizzes
- 10% MasteringA&P homework
- 10% Clicker questions
Assessment is the same for both A&P I and II.
Results and Data
MasteringA&P was implemented in fall 2015 in A&P I, and the student success rates for fall 2013, 2014, and 2015 were compared. For fall 2013 and 2014, when MasteringA&P was not in use, 63 and 57 percent of students respectively received an A, B, or C in the course. In fall 2015 when MasteringA&P was adopted, 70 percent of students earned an A, B, or C (figure 1). For A&P II, 56 percent of students in the spring 2014 semester earned an A, B, or C. Data for the spring 2015 semester was not included in the study because the instructor did some experimental activities and felt the course was not taught in a similar way. For spring 2016, after MasteringA&P was adopted, the success rate was 83 percent (figure 1).
In addition, for A&P I, figure 2 shows that students in the fall 2015 semester had an exam average of 64 percent compared to 55 and 53 percent for the prior two semesters. The exam average included three exams and the final and used the raw exam scores before extra credit retake questions were added. The fall 2015 exam average was significantly higher, and the breakdown of scores was as follows:
- Students in fall 2014 scored 10.7 percentage points lower on exams (M=53%, SD=19%, N=115) than students in fall 2015 (M=64%, SD=18%, N=111) (p< .01).
- Students in fall 2013 scored 8.7 percentage points lower on exams (M=55%, SD=21%, N=142) than students in fall 2015 (M=64%, SD=18%, N=111) (p< .01).
Exam results were similar for A&P II with a higher average for all exams in spring 2016 when MasteringA&P was in use compared to spring 2014 without Mastering. The spring 2016 average exam score was 12 percentage points higher (M=63%, SD=17%, N=118) than the exam average for spring 2014 (M=51%, SD=18%, N=166).
An analysis was conducted for fall 2015 and spring 2016 on the number of MasteringA&P assignments attempted by students compared to the exam average. For A&P I in fall 2015, Pundt assigned 12 of each type of MasteringA&P homework (chapter, AFU, DSM). The average number of assignments skipped was 4.96 out of 36 (14 percent). For purposes of this study, a skipped assignment was considered to be one with a score of 0. For students who attempted all or skipped fewer than five Mastering assignments, the average exam score was significantly higher than for students who skipped five or more (p<.05) (figure 3).
For A&P II in spring 2016, Pundt assigned 11 of each type of MasteringA&P homework (chapter, AFU, DSM). The mean number of assignments skipped was 3.4 out of 33 (9 percent). Figure 3 shows the exam average for students who skipped fewer than four MasteringA&P assignments compared to students who skipped four or more assignments. The difference in exam averages was seven percentage points, and it was significant (p<.05).
The same analysis was done for each specific type of assignment with the following results:
A&P I, Fall 2015:
- The mean number of MasteringA&P chapter assignments skipped was one out of 12. Students who skipped no chapter assignments had an average exam score of 71 percent compared to 48 percent for students who skipped one or more chapter homework. The difference was significant (p<.05).
- The mean number of Adaptive Follow-Up assignments skipped was two out of 12. Students who skipped fewer than two AFU assignments had an average exam score of 71 percent compared to 52 percent for students who skipped two or more AFU assignments. The difference was significant (p<.05).
- The mean number of Dynamic Study Modules skipped was two out of 12. Students who skipped fewer than two DSM assignments had an average exam score of 71 percent compared to 48 percent for students who skipped two or more DSM assignments. The difference was significant (p<.05).
A&P II, Spring 2016:
- The mean number of MasteringA&P chapter assignments skipped was one out of 11. Students who skipped no chapter assignments had an average exam score of 66 percent compared to 59 percent for students who skipped one or more chapter assignment. The difference was significant (p<.05).
- The mean number of Adaptive Follow-Up assignments skipped was one out of 11. Students who skipped no AFU assignments had an average exam score of 66 percent compared to 62 percent for students who skipped one or more AFU assignments. The difference was not significant (p=.08).
- The mean number of Dynamic Study Modules skipped was two out of 11. Students who skipped fewer than two DSM assignments had an average exam score of 67 percent compared to 61 percent for students who skipped two or more DSM assignments. The difference was significant (p<.05).
While other factors, such as motivation, can impact these results, Pundt observed that the outcome for students who did more Mastering assignments was consistent in both A&P I and II for the first two semesters analyzed. Pundt plans to share this information with students at the start of the semester to help them understand the importance of doing the homework as a way to encourage them to develop good study skills.
Success rates (A/B/C) with and without MasteringA&P in A&P I and II
Figure 1. A&P I: Fall 2013 ( n=142); Fall 2014 (n=115); Fall 2015 (n=111); A&P II: Spring 2014 (n=166); Spring 2016 (n=118)
Exam average fall 2013–fall 2015, A&P I
Figure 2. Fall 2013 (n=142); Fall 2014 (n=115); Fall 2015 (n=111)
Exam average based on number of MasteringA&P assignments attempted
Figure 3. Fall 2015 A&P I: Attempted All/Skipped <5 (n=79); Skipped ≥5 (n=32); Spring 2016 A&P II: Attempted All/Skipped <4 (n=86); Skipped ≥4 (n=28)
The Student Experience
Pundt has found that students seem to uniformly find using MasteringA&P beneficial. She does hear students that say the work they do in Mastering is time-consuming. However, this is intentional on her part because she understands the amount of work needed to be successful, and often students don’t understand how much work is really required to do well in the course. She also has heard from some students that the AFU questions tend to go back into previous material that is not on the upcoming test, so students feel it’s not worth their time. The AFU content may pull review or content from prior chapters based on what students miss in the chapter homework. Overall though, Pundt feels that students believe Mastering is helping them in the course.
Pundt began using MasteringA&P in fall 2015 to encourage students to engage more often with course content and to help them improve their study skills. Results for the first semester of A&P I and II show improved success rates in addition to higher exam averages. The results for the first semester of A&P I and II showed that students who consistently did more of the interactive Mastering assignments tended to do better on exams. Pundt is not planning any changes to her course at this time, other than to incorporate new activities in Mastering as they become available. She assigns multiple types of Mastering homework (chapter, Knewton Adaptive Follow-Up, and Dynamic Study Modules) to provide different learning activities, and believes this is a best practice when implementing Mastering. She finds it encourages students to engage with and review the chapter content more frequently and beyond completion of the homework.
To learn more, go to our blog and read the interview of Suzanne Pundt, Homework: How an online program got my students to do it.