MasteringEngineering Mechanics of Materials educator study investigates student performance in a hybrid format at Tennessee Technological University
- During the period of study, the enrollment increases were primarily from students entering the program with no available ACT scores, thus placing them in an unknown risk category as described in the study.
- Instructors found that MasteringEngineering resources allowed them to replicate the recitation experience outside the classroom when it was convenient for students, which also meant additional faculty resources and time were not required.
- Students taking the hybrid course format provided positive feedback on their experience with MasteringEngineering.
- The instructor recommended communicating to students early in the semester about the value and importance of doing the MasteringEngineering homework, taking advantage of its different resources, and using good problem-solving techniques.
Tennessee Technological University, Cookeville, TN
Mechanics of Materials
Traditional and hybrid
MasteringEngineering; Mechanics of Materials by Hibbeler
David Huddleston, Professor
Results reported by
Betsy Nixon, Pearson Customer Outcomes Analytics Manager
- Enrollment: 10,314 (10 percent graduate students)
- Type: public institution founded in 1915
- Demographics: 44 percent female; 56 percent male
- Area: rural with 74 percent of students living off campus
- Graduation rate: 53 percent (six-year rate; 2009 cohort)
- Financial aid: almost two-thirds of all students receive some form of financial aid
- History: from 1916 to 1924, the university offered courses only at the high school and junior college levels. By 1929, the State Board of Education had authorized a complete college program.
About the Course
David Huddleston is a Professor at Tennessee Technological University (TTU) having joined the faculty in 2004. He started teaching in 1991, and teaches the Mechanics of Materials course, along with upper-level engineering courses. In Spring and Fall 2014, Huddleston taught the Mechanics of Materials (MOM) course. The course covers stress; strain; Hooke’s law, extension, torsion and bending; beam deflections, column buckling and combined stresses. Students must have successfully completed Statics with a C or higher prior to enrolling in MOM.
The goal of the course is to introduce students to the behavior of deformable bodies subjected to forces, moments, and thermal change. The course learning objectives include:
- To analyze and solve problems in static equilibrium;
- To develop fundamental principles of mathematics and engineering mechanics to understand and solve problems related to internal stresses, strains, and deformations of objects; and
- To be introduced to statically indeterminate structures and fundamental column behavior.
Measurable outcomes for the course include students being expected to:
- Calculate normal and shearing stresses on specified planes of members subjected to axial loads;
- Calculate strains and deformations of members subject to axial loads as well as to relate stresses to strains;
- Calculate torsional shearing stresses and deformations of circular shafts subjected to torques;
- Calculate bending stresses for a variety of beam cross sections and loading cases;
- Understand and calculate beam transverse shearing stresses for any location in the beam cross section;
- Combine diverse load cases and corresponding stresses to calculate maximum stress conditions;
- Solve the basic differential equation governing beam behavior and compute elementary statically indeterminate beam reactions; and
- Apply Euler’s Equation to calculate column load capacity.
Challenges and Goals
Background: The Tennessee Public Agenda for Higher Education focuses on increasing statewide educational attainment by implementing Tennessee’s Complete College Tennessee Act (CCTA) (2010), which emphasizes the importance of the state’s educational system for leveraging economic development. In addition, the state of Tennessee’s Drive to 55 Alliance engages private sector partners, leaders, and nonprofit organizations to support the state’s initiative of 55 percent of Tennesseans equipped with a college/associate degree or certificate by the year 2025. To work towards these goals, the current emphasis is on integration of higher education and cost-effective instructional delivery systems.
In 2013, The Tennessee Board of Regents (TBR) funded an initiative that solicited course revitalization proposals that demonstrate a cost-effective approach to student success in high-enrollment gateway courses. Engineering faculty at TTU participated in that initiative and designed and conducted a study in 2014 implementing a hybrid model for two gateway engineering mechanics courses—Statics and Mechanics of Materials—both required for the Civil and Mechanical Engineering majors.
Issue: TTU faculty who teach these courses generally believe that the most effective way to learn the material is by practicing and understanding how to solve problems. At TTU, the course has historically been taught with a conventional lecture format, along with extensive individual instruction to supplement the lecture. This traditional format tends to be labor-intensive for faculty and therefore difficult to scale to larger sections. As part of the 2014 TBR study, TTU developed a hybrid model comprised of a traditional, in-class lecture with online homework and digital resources for learning outside of class. The hypothesis was that a successful implementation of the hybrid model would mean that faculty would be able to administer a more efficient and effective course as follows:
- A more efficient course was defined as the same level of required faculty resources (instruction and grading time) for an increased number of students in the class.
- A more effective course was defined as demonstrating improved student understanding and retention of the material.
For this preliminary study, TTU considered the minimum acceptable course results to be overall performance at the same level as prior years when only the traditional format was offered, so there would be no negative impacts from the change.
Process: TTU had been using the Hibbeler textbooks and adopted MasteringEngineering in Spring 2014 for the online resources for the following reasons:
- The learning aids in MasteringEngineering would facilitate the hybrid delivery format.
- MasteringEngineering textbook problems with randomly-assigned parameters could help control collaboration.
- Selected Mastering tutorial problems included context-sensitive hints for students who have common errors in their solutions. The hints were developed by experienced subject-matter experts. Because a student could access an annotated solution with more detail than the examples in a typical textbook, this was viewed as a tool to facilitate student learning outside of class and office hours.
- Homework would be automatically graded and provided immediate feedback to the student, which meant that a course could have higher enrollments, but the workload to faculty grading homework was not increased.
- Faculty believed that MasteringEngineering’s asynchronous student-learning environment could be used with instructor-guided assignments or as an independent study aid, thereby facilitating both types of instruction and resources needed for the hybrid portion of the course.
- Faculty believed that, in many respects, MasteringEngineering could replicate a recitation period without the scheduling constraints and would provide resources whenever it was convenient for students.
The results of this research study were published in 2015 in the paper, An Academic Program Assessment Methodology to Leverage Drive to 55 in the CCTA Integrated Higher Education Environment, by David Huddleston, David Elizandro, Jane Liu, Guillermo Ramirez, and Elizabeth Hutchins, and the results were presented at the American Society for Engineering Education (ASEE) conference in July 2016. This educator case study will summarize and report on the Mechanics of Materials’ course findings, and a separate educator case study will feature the Statics’ results.
The ACT’s annual STEM report for 2016 stated that nearly half (48 percent) of the 2.1 million 2016 American high school graduates who took the ACT test expressed an interest in STEM majors or careers, but only 26 percent of those graduates met or surpassed the ACT College Readiness Benchmark in STEM. In order to better understand student readiness and course performance, the 2014 TTU preliminary study was designed as an initial review of the course data. The long-term goal was to develop an experimental design to collect data to better understand the impact of the hybrid model on student learning. TTU utilized ACT scores to categorize students in groups to facilitate evaluating student performance. The ACT requirements to enroll in the engineering program at TTU were as follows:
- Incoming freshmen from high school must have a high school GPA of 3.0 or a 20 ACT composite with a 22 ACT Math sub-score.
- For freshmen 21 years of age or older, COMPASS exam scores are used in lieu of the ACT requirements.
- International students are required to have a high school diploma, demonstrated language proficiency, and an ACT Math sub-score (usually via the ACT COMPASS) of 19.
- Transfer student requirements are a 2.0 composite GPA, a 2.0 GPA in the last full-time semester, and a C or higher in a pre-calculus mathematics course.
For Spring and Fall 2014, both hybrid and traditional formats were offered in Mechanics of Materials, and data were collected. Students self-selected into either course format. Two MOM hybrid sections were taught in Spring and Fall 2014 with enrollments of 60 and 61 students respectively, with a third section of MOM enrolling 18 students (for a total of approximately 50 percent of students). Spring 2015 data were not available at the time of analysis and are not included.
While each instructor developed their own course content, faculty collaborated on content covered, pace, and selection of homework for both courses. Implementation for each format was as follows:
Traditional: Students in the traditional sections received the same in-class lecture as the hybrid sections, but they did not have access to the MasteringEngineering resources, including automatic scoring, hints and feedback, and additional study resources. All homework was done via paper and pencil, completed on engineering paper, and was to follow a GIVEN, REQUIRED, and SOLUTION type format. Diagrams were to be drawn neatly using a straight edge. The homework was handed in for instructor grading.
Hybrid: The hybrid classes were a blend of in-class lecture, electronic instruction, and MasteringEngineering homework and resources that included tutorial aids. Students in the hybrid sections received the same in-class lecture as the traditional sections, which included example problems similar to the MasteringEngineering assigned homework problems. While homework was assigned using MasteringEngineering, solutions for homework had to be completed on engineering paper and follow a GIVEN, REQUIRED, and SOLUTION type format. This provided students the same problem-solving practice as traditional students, but then allowed them to input their answer in Mastering and know immediately if the calculations were correct.
Two semester exams and one comprehensive final were given each semester, and class attendance and participation was highly recommended. If the final exam grade was higher than the lowest semester exam grade, the average of the two replaced that semester exam grade.
The MasteringEngineering homework included the end-of-section and tutorial problems. The assignments were not timed, and students were allowed multiple attempts. Some assignments were due before lecture, but most were due after in-class lecture. The instructors used macroscopic performance data to determine specific areas that may need extra review based on student performance and would cover that during in-class time.
- 60% Exams (two)
- 30% Comprehensive final exam
- 10% MasteringEngineering homework and conventional assignments
Results and Data
To evaluate the results of the study and better understand student performance, TTU analyzed course scores based on ACT scores. For the analysis, the student population was categorized in four mutually-exclusive groups based on ACT scores as follows:
- Category 1: Students with ACT scores ≥25. Most engineering schools view these as core students who are adequately prepared to begin engineering degree coursework.
- Category 2: Students with ACT scores between 22–24. These are regional mission specific students who, with mentoring, should be able to complete engineering degree requirements.
- Category 3: Students with ACT scores <22. These are believed to be at-risk students who may have problems mastering a college of engineering curriculum.
- Category 4: Unknown-risk students. These tend to be transfer students who are not required to submit ACT test scores and international students without an ACT score.
Because a portion of the student population repeats a course, data represent the number of students enrolled, including duplicates, for the period of study. Figure 1 presents enrollment numbers for 2009–2014. During 2014 and continuing into 2015 (2015 data not available at time of analysis), increases in course enrollment were primarily the result of increases in unknown-risk students (category 4). For the calendar years 2009–2014, the number of students in the unknown-risk category increased from approximately 10 to 30 percent of the total enrollment. The increase in the number of students enrolled in 2014 is indicative of an improvement in course efficiency, one of the parameters of the study, so the remainder of the focus of TTU’s analysis was on course effectiveness.
Figure 2 shows the partial results presented in the TTU study. Data in figure 2 represent course grades for the academic years 2013 (all traditional sections) and 2014 (mix of traditional and hybrid sections) by ACT category. Data are available for 2009–2014, but due to space limitations, only two years are presented in this study. All results can be found online in the full version of the paper at the ASEE site.
The results show the following:
Category 1 (ACT ≥ 25): Data indicate no overall performance changes for student success rates for the Category 1 group with the total A/B/C students remaining essentially the same over the last three years.
Category 2 (ACT 22–24): In 2014, the A/B/C rate was less than 60 percent in MOM for this category of students. This outcome affirms the 2013 reduction of C or better grades in Statics. These results will continue to be monitored and evaluated in conjunction with performance in Statics.
Category 3 (ACT <22): Overall MOM grade distributions for category 3 are visibly different from the first two categories and reflect a bias to lower course grades. It is important to note that in both calendar years 2013 (traditional format) and 2014 (traditional and hybrid formats), there were increases in the percent of Cs and D/F/Ws in Statics, so the trend towards lower course scores was increasing for this group before the hybrid model was implemented.
Category 4 (unknown-risk): Grade distributions for this category also tend to be visibly different from students in categories 1 and 2. The combination of the last two categories (3 and 4) appear to have a bimodal distribution for course effectiveness. In 2014, the A/B/C rate decreased compared to 2013, but the success rate in 2014 was higher than 2010 and 2012 despite an increasing number of students in this category. Additional analysis would need to be done to better understand the performance of this group of students.
The analysis conducted by TTU showed that with increased enrollments and after implementing the hybrid course format with MasteringEngineering, students in the first category continued to perform at about the same level, while category 2 students had a drop in success rates which tended to reflect performance from the Statics course from the prior year. Since grades for categories 3 and 4 students continued to be problematic in the revised courses, effectiveness seems to be independent of the course format. The study reports that class effectiveness seems to be dependent on academic profile, and that “…it is important to note that variation in student performance data is also the result of systemic issues in higher education that introduce a major source of noise in course effectiveness measurements and complicates a meaningful experimental design.”1
The study was designed as an initial review, so further data collection and analysis will continue in an attempt to better understand the impact of moving from a traditional lecture format with paper and pencil homework to a hybrid model with lecture and MasteringEngineering interactive homework and learning resources. TTU faculty have identified some best practices based on their experience that they recommend to others implementing Mastering into their course:
- Build homework around MasteringEngineering videos to provide students with a visual resource.
- Early in the semester during lecture, emphasize the coaching tools in MasteringEngineering, and encourage students to use the different resources available.
- Continue to encourage students to develop their study skills, note-taking skills, and organizational skills.
- Communicate to students that MasteringEngineering provides resources to help all learning styles.
Enrollment by ACT score
Figure 1, Enrollments 2009–2014, 2009 (n=~195); 2010 (n=~165); 2011 (n=~205); 2012 (n=~210); 2013 (n=~225); 2014 (n=~245)
Course grade by ACT score category
Figure 2. Mechanics of Materials Course Grade by ACT Score Category
The Student Experience
As part of the TBR-funded study, the faculty at TTU collected feedback from students in various formats including student surveys and individual student reviews of MasteringEngineering. TTU faculty concluded, “Based on student feedback, the electronic study feature is one of the most effective blended features of PME [MasteringEngineering].”2 One student stated in his written evaluation, “MasteringEngineering is the most technologically advanced tutorial and homework system. It tutors engineering students individually while providing instructors with rich teaching diagnostics.”
The analysis of the effectiveness of a hybrid course delivery using MasteringEngineering at TTU indicates that more students were enrolled in the course with no additional faculty resources required, and there were no apparent adverse effects on student performance with the hybrid format for core and regional mission students (categories 1 and 2). However, the analysis identified critical issues that exceeded the scope of the original project by confirming that almost half of the at-risk and unknown-risk students had difficulty in these courses. As a result of the analysis of the data, it became apparent to faculty that, “improving class effectiveness (student success) and efficiency (leveraging faculty effort) is dependent on the academic profile of students, which is dependent on the higher education environment created by the CCTA, as well as the institutional mission and resources to sustain the mission.”3
Instructors also solicited student feedback from students using MasteringEngineering as part of the study. Student feedback was positive and provided insight into the educational benefit of the resources available for students to use and learn outside of class that is not available with paper and pencil homework. TTU plans to continue to offer both a hybrid and traditional delivery option. Most of the changes to the hybrid course following the study have been minor refinements. TTU will continue using Mastering and has stated that, “MasteringEngineering offers learning tools that serve many students quite well based on their learning style. Other students respond better to more traditional methods. Having both available within a hybrid format seems to provide learning opportunities that can reach a diverse audience effectively.”
1 An Academic Program Assessment Methodology to Leverage Drive to 55 in the CCTA Integrated Higher Education Environment, David Huddleston, David Elizandro, Jane Liu, Guillermo Ramirez, and Elizabeth Hutchins