Mastering Engineering educator study investigates homework copying at Texas Tech

EDUCATOR STUDY

Mastering Engineering educator study investigates homework copying at Texas Tech University

Key Findings

  • After Modified Mastering Engineering replaced paper-and-pencil homework, graders were no longer needed, freeing up time for teaching activities and reducing departmental costs.
  • After implementing changes to address student copying on homework, time-spent patterns differed, and exam averages increased.
  • Student survey results show that a majority of respondents (92 percent) said that they try to use the available resources, such as the textbook and online resources, when they are doing homework and have questions.
  • The instructor shared several best practices to discourage cheating and to encourage students to view the homework as a learning tool to help them do better on exams.

School name
Texas Tech University, Lubbock, TX

Course name
Mechanics of Solids

Course format
Face to face

Course materials
Modified Mastering Engineering for Mechanics of Materials by Hibbeler

Timeframe
Spring 2015 and Fall 2016

Educator
Ameri Gurley, Instructor

Results reported by
Betsy Nixon, Pearson Customer Outcomes Analytics Manager

Setting

  • Enrollment: 36,551 graduate and undergraduate
  • Established: 1923
  • Graduation: six-year rate of 74 percent
  • Retention: 82 percent average (freshmen)
  • Race/ethnicity: 63 percent White; 19 percent Hispanic; 11 percent Other; 6 percent Black

About the Course

Ameri Gurley has been an instructor at Texas Tech University for 12 years in the Civil, Environmental, and Construction Engineering Department. In addition to Mechanics of Solids, she teaches Statics, Structural Analysis, Mechanics of Fluids, Engineering Analysis, Construction Materials Lab, and Mechanics of Solids Lab.

Mechanics of Solids is generally taken in the junior year and has a prerequisite of a C or higher in Statics. The three-credit course covers the theory of stress and strain in elastic and inelastic bodies subject to various conditions of loading. A separate one-credit Mechanics of Solids lab is taken after successfully completing the course. Approximately 200300 students take the course each semester, and students come from a mix of majors including Civil and Mechanical Engineering.

Course outcomes include:

  • Understand member internal force equilibrium based on external static loads.
  • Draw and relate load and internal force diagrams.
  • Understand the relationships between concepts of force to stress, force to strain, stress to strain.
  • Be able to solve for total combined stress and strain in deformable bodies in the elastic range and in some cases the inelastic range of material behavior.
  • Construct Mohr’s Circle for stress on a material.
  • Formulate equations for slope and deflection based on the elastic curve.
  • Develop equations for critical buckling of structural components within specified constraints.
  • Demonstrate advanced application of basic principles using the method of superposition for indeterminate systems.
  • Develop an appreciation of mechanical behavior of bodies and materials that will be useful in subsequent courses.

Engineering involves applying scientific and mathematical principles and knowledge to solve technical problems. To enable students to accomplish the program educational objectives, the curriculum is designed to assure that at the time of graduation, students are able to do the following:

  • Apply knowledge of mathematics, science, and engineering.
  • Design and conduct experiments and analyze and interpret data.
  • Design a system, component, or process to meet the constituency needs within realistic constraints.
  • Function on multidisciplinary teams.
  • Identify, formulate, and solve engineering problems.
  • Understand professional and ethical responsibility.
  • Communicate effectively.
  • Understand the impact of engineering solutions in global, economic, environmental, and societal context.
  • Engage in life-long learning.
  • Possess knowledge of contemporary issues.
  • Use techniques, skills, and modern engineering tools.

Challenges and Goals

Problem-solving is an important skill for success in the course and in the field of Engineering. Students need to practice in order to improve their critical thinking skills to successfully solve problems. Getting timely feedback on homework is important in that process since it helps students know immediately if they understand the concepts or have any misconceptions. Gurley generally teaches multiple sections of Mechanics of Solids, and with homework assignments due weekly, considerable time was needed to grade and return the paper-and-pencil assignments. Prior to adopting Modified Mastering Engineering®, she hired multiple graders every semester, but student feedback was still not delivered at the teachable moment when students were doing homework. In addition, Gurley needed to develop her own problems and solutions to provide the amount of practice she believed students needed.

She adopted Mastering to provide a pool of publisher-developed online problems for homework, and to ensure students received instant feedback and consistent grading. Using Mastering freed up additional time she could use for other teaching activities since she did not need to hire and train graders or write out her own problems and solutions. In addition, there was a cost savings to the department since graders were no longer hired, but that was not an initial priority when the decision to adopt Mastering was made. Gurley also believed that assigning digital homework would help engage the current plugged-in generation of students who are always online and motivate them to practice more.

When Gurley was asked what she felt were the biggest issues with reaching her course goals, she said, “The modern student is the biggest roadblock I see. They don’t know how to use textbooks (if they even buy one), and they tend to Google an answer before they even try to solve it. They do that because they lack critical thinking skills, and they think because they did the work searching for an answer online that it’s not cheating. They are not dumb! They just don’t know how to think, so I try to teach that in class.”

Implementation

Prior to adopting Mastering, paper-and-pencil homework consisted of four to five problems per lecture with two to three assignments per week. A grader was assigned to each section to facilitate grading homework for completion and accuracy. After adopting Mastering Engineering in Spring 2015, course components included the following:

Exams: Four paper-and-pencil exams were administered emphasizing problem-solving similar to homework. Exams were closed book and closed notes, but students were provided with an equation sheet.  All work and solutions (with correct units and direction) were required to be shown on the exam. The first exam tends to be primarily a review of engineering and Statics concepts. While the exams in the two semesters in this study were different problems, Gurley believes they were of the same difficulty.

Final exam: The final exam was a common final and combined aspects of the homework, lecture examples, and earlier exams. It was closed book and closed notes, but students were provided an equation sheet.  All work and solutions (with correct units and direction) had to be shown on the final exam. The final could replace a student’s lowest exam grade if it was to their benefit. If a student had a zero on a prior exam because of an unexcused reason, the final exam score could not be used to replace that grade. All work was to be submitted in pencil, and exams filled out in pen were not graded.

Mastering Engineering homework and participation: At least two Mastering Engineering homework assignments were required each week. No late homework was accepted. Students were given five attempts per assignment, and the Mastering homework was not timed. Students received three percent extra credit for not using hints, but Gurley deducted 10 percent for each incorrect submission. Homework included a mix of tutorial and end-of-section chapter problems, and sometimes Mastering videos were assigned for extra credit. Gurley advised students that even though homework was submitted online, they should continue to work in the handwritten format taught with the Given, Find, and Solution statements for solving problems before submitting their answer in Mastering for scoring. Students could be asked to submit a paper assignment, do an in-class assignment, solve a problem on the board, or have attendance taken as part of their course credit for their final homework grade.

After analyzing course results for Spring 2015, Gurley made a point of talking with students early in the following semesters about the importance of doing the homework rather than finding and copying the answers. She explained to them that while they would be able to find the answers online, if they didn’t do the work and understand how to solve the problems, they would struggle taking exams, which were worth a larger percent of their grade. She also removed the random number generator on the Mastering homework so students could work together more easily to encourage peer learning rather than finding an answer online if a student couldn’t figure out a problem. In the Fall 2016 student survey, “called a friend” was the second highest response when asked what they did if they encountered a homework problem they couldn’t answer.

Gurley has considered reducing or eliminating homework credit entirely and keeping the assignments open for the full semester to further emphasize using the Mastering homework to practice and learn.

Assessments

  • 60% Exams (four)
  • 25% Final exam
  • 15% Mastering Engineering homework

Results and Data

Because Gurley found that students have a tendency to try to find answers online before working the homework problems, she undertook a study to examine student behavior around doing the Mastering homework.  By trying to better understand how and why students copy answers and cheat on homework, she felt she could take proactive measures to minimize it since she believed it can have an impact on learning. An online Pearson resource available to help instructors address this issue is “Mastering & Modified Mastering, Methods to Prevent Cheating.” It references a Massachusetts Institute of Technology (MIT) study in which a group of educators investigated the effect of homework copying on student performance and found that homework copying is associated with decreased learning. MIT identified some changes they implemented to their Mastering homework to address this issue, and the Pearson article also provides additional best practices for Mastering users.

After analyzing results from the first semester Gurley used Mastering, she implemented the following changes in her course to address the issue of copying and cheating on homework:

  • Early in the semester, talk to your students about cheating on homework and emphasize how it can hurt them on exams.
  • Include Mastering tutorial homework problems which are designed to facilitate learning and provide help solving the problems.
  • Use the recommended Mastering settings designed to minimize cheating.
  • Focus on Mastering as a tool for learning and preparing for exams rather than as a primary way to earn course credit.
  • Analyze course results to understand any trends or issues that may need to be addressed in upcoming semesters.

An analysis was then conducted after Fall 2016, and compared the results to the first semester Mastering was in use, Spring 2015. The data from these two semesters show the following (figure 1):

  • Students in the Spring 2015 and Fall 2016 semesters had the same exam 1 average, and it was the highest exam average for both semesters.
  • The average for exam 2 dropped for both semesters. The Spring 2015 average was significantly higher than Fall 2016 (p<.05).
  • At exam 3, the average for Fall 2016 was higher, but it was not statistically significant (p=.09)
  • The averages for exams 4 and the final for Fall 2016 were significantly higher than the averages for Spring 2015 (p<05).

Comparison of exam averages, Spring 2015 and Fall 2016

Figure 1. Spring 2015 (n=141); Fall 2016 (n=43)

While there are no data available to compare the abilities of the two semesters of students, such as GPAs or ACT scores, the average performance on exam 1 was the same. Since exam 1 tends to primarily cover prerequisite engineering and Statics concepts, this indicated that the students had comparable knowledge coming into the course.

Mastering records time spent, but it’s important to note that the data may not be reflective of actual work in Mastering since students could log on and do something other than homework, or could print the Mastering problems, work offline, log back on, and input their answers. Therefore, it’s recommended that instructors track this over multiple semesters to watch for trends in student behavior that may impact results. While understanding how effectively the time spent was used working in Mastering is difficult, the data can provide insight when evaluated in conjunction with other results, such as homework and exam scores. Figures 2 and 3 present the data for each semester which includes the average time spent and the Mastering and exam averages by unit, showing the following:

  • For Spring 2015 (figure 2), unit homework average scores remained steady between 81 and 87 percent. Exam scores steadily declined after exam 1, with a slight increase on the final.
  • For Fall 2016 (figure 3), unit homework average scores ranged from 58 to 77 percent, a wider range than Spring 2015. Exam scores initially dropped at exam 2, but steadily increased thereafter.
  • For Spring 2015, unit average time spent steadily decreased with a slight increase for the last unit; it was on the same trajectory as average exam scores.
  • For Fall 2016, average time spent increased during unit 2, the first unit that new concepts were being introduced, and decreased after that. The second exam was the lowest this semester, but scores steadily increased thereafter.

Comparing behavior and performance between Spring 2015 and Fall 2016 shows the following (figures 2 and 3):

  • Students in Fall 2016 spent overall more time logged into Mastering than students in Spring 2015.
  • Students in Spring 2015 had higher average time spent than students in Fall 2016 for the first and last units; Fall 2016 students had higher average time spent in units 2, 3, and 4.
  • Fall 2016 students spent an average of 62 minutes per assignment, while Spring 2015 students spent an average of 48 minutes per assignment.

Comparison of Mastering and exam averages with time spent, Spring 2015

Figure 2. Spring 2015 (n=141)

Comparison of Mastering and exam averages with time spent, Fall 2016

Figure 3. Fall 2016 (n=43)

For Spring 2015, after doing an in-depth review of individual time spent diagnostics, it appeared that at unit 2 more students started cheating or copying homework. Homework scores remained high, but exam scores and time spent continued to drop. Time spent seems to indicate that a number of students were spending so little time on homework that they would not have had time to read and work the questions.

While students in Fall 2016 had decreasing average time spent after unit 2, and spent less time on average in the first and last units than Spring 2015 students, they spent more overall time and more time on average per assignment. The Fall 2016 exam averages started rising mid-semester and were significantly higher on exam 4 and the final than for Spring 2015. A Journal of Chemical Education article may provide insight into the findings. The articles states that “high-achieving students engaged in certain behaviors earlier in the semester, such as seeking instructor assistance and engaging in practice problems. In contrast, low achieving students waited until later in the semester but studied more intensively, actually surpassing high-achieving students in frequency and duration of practice problems during the third and fourth months of the semester. These results match patterns observed by previous studies, adding to a growing body of evidence that for certain study behaviors, procrastination has detrimental consequences that cannot be compensated for by belated efforts.”1

Because students in both semesters started out with the same average on exam 1, it appears that they had comparable knowledge coming into the course. Students in Fall 2016 had a lower exam 2 average but had more recorded time spent in Mastering which may suggest they were practicing more and may have been more motivated to learn. Their remaining exam scores continued to increase. For Spring 2015, one possible explanation may be that more students in this class were less motivated and were not using the homework as a tool to learn but rather finding answers online to complete the homework and get the credit. With the increase in time spent in the last unit for Spring 2015, students may have tried to increase efforts to improve their final scores. The pattern observed during the two semesters reported in the study demonstrates that the group that showed a higher overall time spent in Mastering ended up with significantly higher final exam scores. More research would need to be done to understand how students used the time to better understand the results.

Beginning with the Spring 2017 semester, Gurley plans to reduce course credit for Mastering to 10 percent to further emphasize using the practice for learning and to increase the credit for exams to 65 percent. She plans to evaluate her results on an ongoing basis.

The Student Experience

An online survey was conducted with the Fall 2016 students and had a response rate of 54 percent. One question asked was, “With 1 being not very likely and 10 being extremely likely, indicate how likely you would be to recommend Mastering to another student?” The average answer was 7.24. In addition, students were asked what they did when they ran into a question they couldn’t answer while doing the homework. Students could check as many answers as applied to them. While 64 percent of respondents said they tried to find the homework question on the internet, 92 percent of respondents said they tried to use the resources which included the textbook and online resources, the answer most selected. More respondents said they called a friend than said they would try to find the homework question on the internet (table 1). A survey was not administered during the Spring 2015 semester, so there were no results to compare.

Student survey response to, “What did you do when you ran into a question you couldn’t answer?”

Response

Percent of Respondents

Tried to use the resources to find an answer including the textbook, online resources, etc. 92%
Called a friend 72%
Tried to find the homework question on the internet 64%
Went to the learning/tutoring center 28%
Talked with the professor before I turned in homework 24%
Quit and waited for the instructor to explain it in class 12%

Table 1

Finally, to better understand the student experience, students were asked the open-ended question, “What did you like most about Mastering?” Student responses included:

  • “Multiple attempts are allowed, problems are similar to those in book, which makes it easier to learn while doing the homework, by using similar strategies.”
  • “I like that you were able to get hints on the homework and that you got immediate results.”
  • “How neat the interface is and how easy it is to read the questions.”
  • “I like when Mastering would give you hints and try to guide you to the process of figuring out and answering a problem.”
  • “It is online & provides different resources for students to learn the material.”

Conclusion

Problem-solving and critical thinking are important skills for engineering students. Homework is one way to help students develop those skills. Paper-and-pencil homework needs to be graded and does not provide instant feedback, so Gurley adopted Mastering Engineering for homework. This provided her with an increased pool of problems to assign, along with immediate feedback and hints to students as they worked. It also saved her time because she did not need to continually hire and train graders each semester, and it provided a cost savings to the department by eliminating the cost of graders. In addition, she also feels that as online homework becomes more common, students are responding to it differently. They are engaging with it and using it for learning, rather than just as a way to increase their course grade. Comments made by students in the survey reinforce this.

Many instructors using digital homework find that students attempt to get answers online rather than do the work themselves. This is a common concern among professors using online homework systems. Copying homework, as well as trying to cheat on exams, has been a part of education since before computers or online programs existed. Gurley herself said, “Students are going to copy. They copy with paper assignments; they copy with online assignments. Where that affects them is the exams.” To address this concern, Gurley made changes to her course which included:

  • Emphasizing the benefits of doing the Mastering homework to practice problem-solving and develop needed skills;
  • Explaining the impact that cheating on homework can have on course and long-term success; and
  • Changing Mastering homework implementation to minimize the ease of cheating.

While all homework carries the risk that students will copy from another source, homework is still shown to be beneficial in student learning. In the MIT study referenced above, researchers found that homework copying is associated with decreased learning. Rather than eliminate homework, they recommended using strategies to combat cheating as a better way to address the issue.

With the importance of problem-solving in Engineering, and the considerable research showing that doing homework leads to greater learning,finding ways to educate students about the benefits of doing their own homework to learn and the consequences of cheating seem to be an effective approach to address the problem. In addition, following Mastering best practices can minimize the student’s ability to quickly and easily find the answers online without doing the work. With early communication to students and proper implementation, Mastering Engineering homework can be an effective learning tool to help students develop needed problem-solving skills and become more successful learners.

 

______________________________

1Szu, Evan Szu, Nandagopal, Kiruthiga, Shavelson, Richard J., Lopez, Enrique J., Penn, John H., Scharberg, Maureen, and Hill, Geannine W. (2011). Understanding Academic Performance in Organic Chemistry. Journal of Chemical Education, 88 (9): 1238–1242.

2Bloom, B. S. (1984) The search for method of group interaction as effective as one-to-one tutoring. Educ. Res. 13: 4; Tuna, D. T. and Reif, F., Problem Solving and Education, Lawrence Erlbaum, Hillsdale, NJ, 1980.