MyLab Operations Management educator study analyzes exam scores and use of Simulations at DePaul University

EDUCATOR STUDY

 

MyLab Operations Management educator study analyzes exam scores and use of Simulations at DePaul University

 

Key Findings

  • Students earning MyLab Operations Management scores above the median also earned higher exam and final course grades.
  • Data indicate a strong correlation between MyLab homework scores and average exam scores.
  • 94 percent of students responding to an end-of-semester survey agreed that the MyLab Simulations allowed them to gain hands-on experience with decision making in Operations Management.

School name
DePaul University, Chicago, IL

Course name
Principles of Operations Management

Course format
Face to face

Course materials
MyLab Operations Management with Principles of Operations Management: Sustainability and Supply Chain Management by Heizer, Render, and Munson

Timeframe
Fall 2016

Educator
Lori Cook, Associate Professor

Results reported by
Candace Cooney, Pearson Customer Outcomes Analytics Manager

 

Setting

  • Locale: large, urban, private, Roman Catholic four-year institution in Chicago
  • Enrollment: more than 23,000 students
  • Undergraduates: 67 percent
  • First generation: 33 percent
  • Class size, 20–50 students: 63 percent
  • Student-faculty ratio: 16:1
  • Four-year graduation rate: 56 percent
  • Gender: 53 percent female
  • Diversity: 45 percent

About the Course

Lori Cook has been teaching for 24 years, including 19 years at DePaul, where she has been offering Operations Management to approximately 50-150 students each quarter. This one-quarter, four-credit course is a required core course for business majors. Operations Management focuses on the effective application of managerial techniques and concepts related to the delivery of services, manufacturing, and supply chain processes. Topics include operations strategy, forecasting, project management, quality management, supply chain management, facility location and layout, productivity, inventory management, and scheduling.

Upon satisfactory completion of this course, students should be able to:

  • Describe the strategic importance of operations and how operations can provide a competitive advantage in the marketplace;
  • Compare and contrast the relationship between operations and other business functions like marketing, finance, accounting, and human resources;
  • Evaluate the issues related to designing and managing operations; and
  • Apply the appropriate techniques to design and manage operations.

Challenges and Goals

Cook was looking for a digital course companion that would require students to work through homework problems individually. At the time, students were working together on problems in small teams which she hand-graded, but the workload was uneven, with stronger students taking the lead on most assignments. Weaker students were then unable to replicate the work on exams. With an average of 48 students per section, hand-grading individual assignments in a timely manner was not possible, so she sought a system that allowed for automated grading and also offered student feedback during assignment completion. By Fall 2012, she had adopted MyLab Operations Management to give her students the personal practice they needed to prepare for quizzes and exams. Additionally, the MyLab tools paired well with her recorded lectures, making it possible to offer the course online in 2016.

Implementation

MyLab Operations Management is required; the program is used primarily by students working at home on a personal computer. Cook’s goals for assigning work in MyLab are to introduce new concepts, provide homework and practice opportunities, and to help students assess their own understanding of the course material. As the course instructor, her role is to assign content and homework in MyLab and provide remote support to students using the program at home.  Cook anticipates that students will spend at least two hours per week working in MyLab, which may include reading the eText and completing assignments. Based on results of a voluntary, end-of-quarter survey, 48 percent of students said they spent 2–3 hours using MyLab while an additional 26 percent of students indicated they spent more than three hours working in the program.

Learning Management System Integration

Cook opted to integrate her MyLab course with Desire2Learn (D2L) for the following reasons:

  • Grade transfer – grades are easily transferable from MyLab to D2L and there is one single gradebook for the course;
  • Single sign-in process – students are ready to work in MyLab on the first day of class; and
  • Content linking – ability to link to MyLab directly from D2L.

Cook’s students now have just one access code and a single sign-in process instead of the need to log in to D2L and additionally sign in to MyLab. This results in a clear-cut way for students to start their MyLab assignments, ensuring that they are ready to work from the first day of class, as indicated on the student survey:

  • 89 percent of students strongly agreed or agreed that they were able to access MyLab through Canvas and appreciated not having a separate, second log-in and password for MyLab.

MyLab Assignments

Following a Pearson best practice, Cook assigns Chapter 0: Getting Started with MyOMLab the first week of class. This quick quiz familiarizes students with how to enter answers in the system, the type of questions they will encounter, and the various learning aids. This assignment also ensures all students are registered in the MyLab and puts them on the path to success immediately. Cook reports that an unintended benefit of assigning chapter 0 is that she receives very few questions about the functionality of MyLab now, freeing up her time early in the quarter for more important tasks. Additionally, she identifies the 14-day temporary access as providing students with a cushion, in the event they are waiting on financial aid or just unsure if they will remain in the class, allowing for prompt registration into her course.

Students have weekly assignments in MyLab, consisting of approximately 10 quantitative, multi-part problems. They have unlimited attempts at completion because Cook’s goal is for students to use MyLab as a learning tool, identifying and remediating their mistakes. All learning aids are turned on and student feedback has been very favorable: 87 percent of respondents on the student survey indicated they “always” or “usually” accessed the learning aids when unable to start or finish an exercise in MyLab. Student comments from the survey include:

  • “I liked the ‘Help Me Solve This’ option in MyLab because it clarified common mistakes I was overlooking.”
  • “If I had a question on a homework problem, I could easily send the problem to my professor [using the ‘Ask My Instructor’ learning aid].”
  • “’Help Me Solve This’ really allowed me to answer all of the questions I wasn’t sure how to solve at first.”

Cook also assigns four of the five MyLab Operations Management simulations. These interactive simulations give Cook’s students hands-on experience in real-world roles, helping them link course concepts to on-the-job application. Using real-life situations, students evaluate information and engage in decision-making and critical analysis, which results in real-time feedback so they can see the impact of their choices and gauge their performance against individual, peer, and system metrics. Topics integrate with learning objectives and include inventory, quality control, forecasting, and project management. Students have unlimited attempts at completion, allowing them to see the wide-ranging effects of their various decisions.

On the end-of-quarter survey, students reported the following:

  • 94 percent of respondents agreed that the simulations in MyLab were a real-world application of the chapter material that allowed them to gain hands-on experience with decision-making in OM.
  • 96 percent of respondents agreed that the simulations were a different and engaging way of working with the chapter concepts.

Comments included:

  • “I really enjoyed how intuitive the simulations were, and I think they helped me understand what was being taught in class.”
  • “The simulations allowed me to apply theory to actual situations and have a better understanding of how these decisions and calculations affect various management situations.”
  • “I liked most of the simulations. They were a very fun and different approach to learning the material. They did make me feel more hands-on and visual, which makes it a lot easier to learn the information.”
  • “Everything I learned from each one of the MyLab simulations were linked with in-class concepts. Interesting to actually see what the teacher explains in a real-world simulation.”

Two paper-and-pencil, multiple-choice exams are administered, comprised 60 percent of quantitative questions and 40 percent concept-based questions. Students complete at least 30 multiple choice questions and three multi-part problems, and are given 90 minutes to complete the mid-term exam and 135 minutes to complete the final exam. The problems on exams are similar in format and difficulty to MyLab homework exercises.

Cook begins each lecture with 4–5 personal response, polling-type questions, using an application called REEF, which review previously covered material. When beginning a new chapter, students work in small groups on an experiential activity that introduces the new material. Cook covers the new concepts in a lecture that is followed up by problem-solving both as a class and in small groups.

Assessments

  • 50% Exams (2)
  • 30% MyLab homework assignments
  • 10% MyLab simulations
  • 10% Class participation

Results and Data

Students were divided into two groups based on the median MyLab homework score. Students who earned a MyLab score above the median also earned higher exam scores and final course scores than students who scored below the median on MyLab assignments (figure 1).

  • Students scoring above the median on MyLab assignments earned average exam scores nine percentage points higher than students who scored below the median.
  • Students scoring above the median on MyLab assignments earned average final course scores 11 percentage points higher than students who scored below the median.
  • 48 percent of students scored above the MyLab median assignment grade.

Relationship between MyLab scores and exam and final course grades

Figure 1. Relationship between Median MyLab Scores and Average Exam and Final Course Grades, Fall 2016 (n=140)

Figure 2 is a correlation graph; correlations do not imply causation but instead measure the strength of a relationship between two variables, where r is the correlation coefficient. The closer the positive r value is to 1.0, the stronger the correlation. The corresponding p-value measures the statistical significance/strength of this evidence (the correlation), where a p-value <.05 shows the existence of a positive correlation between these two variables.

  • A strong positive correlation exists between average MyLab homework scores and average quiz scores where r=.55 and p<.05.

For students, the formative MyLab homework is intended to help them identify where they are in terms of successfully completing the summative assessments; it appears that performance on these assignments could be a leading indicator of course success (additional research is needed to develop and test this concept further). Cook states that the MyLab assignments encourage students to engage with the course material, and builds their confidence prior to exams.

Correlation between MyLab homework and quiz scores

Figure 2. Correlation between Average MyLab Homework Scores and Average Exam Scores, Fall 2016 (n=140)

The Student Experience

Responses from the Fall 2016 end-of-quarter, voluntary survey of Cook’s students indicate that the majority of responding students recognize the value of MyLab Operations Management.

  • 98 percent of students strongly agree or agree that their understanding of the course material increased as a result of using MyLab.
  • 96 percent of students strongly agree or agree that they would recommend MyLab to another student.
  • 96 percent of students strongly agree or agree that the use of MyLab positively impacted their quiz and exam scores.
  • 95 percent of students strongly agree or agree that MyLab provided additional resources that helped them learn more than they would have from traditional paper-and-pencil homework.

Student survey responses to the question, “What did you like most about MyLab?” include:

  • I liked how MyLab provided plenty of practice problems to demonstrate the concepts covered in the course. It was a great resource for studying for the exams.”
  • “I enjoyed the simulations as they related to the real world. The balance between the quantitative portion of homework vs simulations was great.”
  • “[I really liked] the fact that it provided instant feedback. I’d get a question wrong and it would remind me of which formulas to use. It helped me understand which concepts were relevant at different points, and it helped me practice for the tests.”
  • “I liked the simulations the most because they were something different than most online assignments. They helped me connect the concepts I learned in class to real-world situations.”

Conclusion

Recognizing that student success is augmented through practice, Cook adopted MyLab to give her students individual assignments with personal feedback. “If I didn’t use MyLab, many of my students would struggle,” she states, “and their confidence would be lower heading into the exams.” Assigning MyLab as a significant portion of the grade compels students to complete the assignments, and given the unlimited attempts at completion, most realize the ultimate goal is learning, not simply completion. “I liked that I could work on assignments until I got the question correct. Since it was set up this way, I was able to learn through my mistakes,” said a student on the end-of-quarter survey. “It really allowed us to practice the things we learned in class. There was a lot of repetition in the type of questions and I liked that because that way I was able to really learn the concept,” said another. With grading done in MyLab, Cook knows her students are receiving accurate and timely feedback, enabling them to practice more where they are weak, leading to increased confidence on exams.