MyLab Economics U Delaware

EDUCATOR STUDY

MyLab Economics educator study analyzes the impact of Learning Catalytics at University of Delaware

Key Findings

  • Students who scored higher on Learning Catalytics in-class assignments, on average, earned higher exam grades and higher final course grades.
  • 90% of student survey respondents agreed that the in-class Learning Catalytics questions helped their understanding of the lecture content or helped them identify misconceptions they had about the lecture material.
  • Regression analysis suggests a positive relationship between MyLab homework scores and average exam scores.

School name
University of Delaware, Newark, DE

Course name
Principles of Macroeconomics

Course format
Face to face with hybrid modules

Course materials
MyLab Economics and Learning Catalytics with Macroeconomics by Hubbard and O’Brien

Timeframe
Spring 2017

Educator
Jens Schubert, Assistant Professor

Results reported by
Candace Cooney, Pearson Customer Outcomes Analytics Manager

Setting

  • Locale: large, public, four-year college located in the small town of Newark in northern Delaware.
  • Enrollment: 23,000 students
  • Four-year graduation rate: 72%
  • Diversity: 27% ethnic minority

About the Course

Introduction to Macroeconomics is a three-credit, one-semester course required of all business majors. The enrollment is 275 students per section; the majority of students are in their freshman year. Introduction to Macroeconomics studies the aggregate behavior of households, firms, and the government using basic economic graphs. It covers essential macroeconomics topics such as gross domestic product, unemployment, inflation, the business cycle, fiscal policy, and monetary policy. The course emphasizes economic thinking, applying macroeconomic concepts and techniques to a variety of contexts. The course employs collaborative and reflective learning methods which require active student participation.

Challenges and Goals

For the past few years, Professor Jens Schubert has been assigned two large sections of either Introduction to Macroeconomics or Introduction to Microeconomics each semester. He strongly believes that problem solving through homework will help students be successful and sought an online assessment solution that provides a variety of ways for students to engage with course material outside of the classroom. Also important was that the system provide his students with frequent feedback about their learning achievements. Based on his previous experience with MyLab™ in smaller sections, he adopted MyLab Economics for his large sections in Fall 2016.

Additionally, the built-in Learning Catalytics™ platform allows Professor Schubert to ask questions during lecture, increase student engagement, and encourage in-class critical thinking. Using Learning Catalytics helps him identify topics that students find challenging, clarify difficult concepts before moving on to new topics, and assess students’ understanding of new material in real time during lecture.

Implementation

First day of class experience
Professor Schubert devotes the first day of class to introducing and demonstrating the features of the MyLab learning platform: how to register, how to access the eText, and how to navigate the homework assignments, videos, and other resources. He also assigns a non-credit exercise, Get Started with MyLab Economics, to introduce students to the instructional navigation window and the learning aids. This is a Pearson MyLab best practice which provides his students with a positive start to their digital homework experience and reduces confusion during the critical start-up period.

MyLab assignments
Professor Schubert’s goal in assigning MyLab homework is to provide students with sufficient exposure to quantitative material necessary to practice and prepare for exams. He believes that students develop a deeper understanding during this engagement with applied practice problems. Since the concepts are new to them, students must work with the ideas and apply what they understand from lecture and the textbook. MyLab helps bridge the gap between class and the time in which homework is completed, which could be a few days later.

Professor Schubert uses the MyLab question metrics when he creates a homework assignment, and his expectation is that students will spend approximately two hours per week completing these assignments in MyLab. On a voluntary, end-of-semester survey of his students (47% response rate), 91% of respondents indicated they spent 13 hours in MyLab completing homework each week.

MyLab homework problems: Students complete weekly assignments comprised of approximately 20 questions, which are a mix of multi-step quantitative problems, graphing exercises, and some multiple-choice questions. Professor Schubert chooses problems based on the learning objectives covered in class. The problems are algorithmic so students can practice as much as needed. Students are given two attempts to complete a problem, and the final attempt is recorded as the score. Learning aids, which provide hints and assistance on a just-in-time basis as students work through a problem, are turned on for homework assignments. On the end-of-semester survey, 58% of respondents agreed that they used available learning aids when unable to start or complete a homework problem. One student commented, “For the complicated questions, Help Me Solve This was very beneficial because I would remember how to do the question after practicing it once and then retrying it again.”

Learning Catalytics
Learning Catalytics (LC), a bring-your-own-device personal response system, complements lecture instruction by using real-time questions and answers to capture student feedback to content questions Professor Schubert poses in class. LC allows him to assess student understanding instantly. Generally, 46 LC questions are asked per lecture, using numerical response, multiple-choice, and graphing question types. Some questions are simple, while others are more challenging, requiring more thought and time. For the most part, however, students are encouraged to think quickly as they would on an exam, and a counter is activated to monitor the question response time. When the answer rate is low for a particular question, Professor Schubert explains the underlying concept again. Then, the same question is repeated to assess their learning. “This immediate feedback during lecture is critical,” says Schubert, “I am able to reassess my explanation and give students a different description of the concept.” Scoring of the LC questions is 50% participation and 50% accuracy of response.

The end-of-semester survey identified:

  • 90% of respondents agreed that in-class Learning Catalytics questions helped their understanding of the lecture content or helped them identify misconceptions they had about lecture content.
  • 96% of respondents strongly agreed that when the class scored low on an LC question, they found it helpful that Schubert explained the concept again and then asked a similar question to confirm understanding of the material before moving on.
  • 91% of respondents agreed that Learning Catalytics made lecture more interactive and helped them focus on the material Schubert was covering.
  • 82% of respondents agreed that Learning Catalytics was an important part of the learning process that helped them succeed in this course.
  • 91% of respondents agreed that they would recommend Schubert continue to use Learning Catalytics in this course.

Student survey comments included:

  • “The use of Learning Catalytics pushed me to pay attention in class and learn the material, because when prompted with a question, I always wanted to get it right to improve my grade.”
  • “With the class being 75-minutes long and in the morning, it was easy for my mind to wander. Learning Catalytics was important in bringing my focus back and allowing me to stay engaged and practice the material.”
  • “I feel that being able to use answer questions in class helped me identify the concepts that I thought I understood but needed to work on more.”
  • “The use of Learning Catalytics in class kept class interactive and engaging and forced me to apply the information we were covering in class. It helped me retain the concepts so I could later apply it to homework and exams.”
  • “Looking at the history of all the questions asked on Learning Catalytics helped me study for exams. It helped to see what I previously got wrong and why I got it wrong as well as seeing all the correct answers.”

With the class being 75-minutes long and in the morning, it was easy for my mind to wander. Learning Catalytics was important in bringing my focus back and allowing me to stay engaged and practice the material.

—Student, University of Delaware

Additional course assessments
Students complete two paper-and-pencil midterm exams covering four chapters each during regular class periods, and a comprehensive final exam. Each midterm is comprised of 40 multiple-choice questions and students have one hour for completion. The comprehensive final exam is a 60-question, multiple-choice exam that students must complete in one hour and 45 minutes. All exams are closed book and closed notes.

Assessments

  • 40% Midterm exams (2)
  • 30% Final exam
  • 20% MyLab homework assignments
  • 10% Learning Catalytics in-class exercises

Results and Data

Students were divided into two groups based on their average Learning Catalytics score. Students who earned LC scores above average earned higher average exam scores and higher final course scores than students who earned LC scores below average (figure 1).

  • Average Learning Catalytics score: 80%
  • Students who scored above average on the LC exercises earned average exam scores seven percentage points higher than students who scored below average.
  • Students who scored above average on the LC exercises earned average final course scores 11 percentage points higher than students who scored below average.

A t-test, which measures whether the means of two groups are statistically different (Learning Catalytics scores above and below average in this study), was used to compare average exam grades and final course grades. Results of the t-test indicate that the mean differences in figure 1 are both statistically significant at the five percent significance level.

  • Average exam score: t(366) = -5.92 and p<.05
  • Final course score: t(295) = -11.05 and p<.05

Average Learning Catalytics scores and average exam and final course scores

Figure 1. Average Learning Catalytics Score and Average Exam and Final Course Scores, Spring 2017 (n=545)

To better understand the impact of MyLab and Learning Catalytics on exam scores, a simple linear regression was conducted with the average exam score as the dependent variable and the Learning Catalytics score and the MyLab homework score as independent variables. Although other influences need to be explored1, the regression results suggest a positive relationship between both Learning Catalytics scores and MyLab homework scores and the average exam scores:

  • An increase of one percent in a Learning Catalytics score, on average, results in a 0.067% higher average exam score.
  • An increase of one percent on the MyLab homework score, on average, results in a 0.247% higher average exam score.

Effect of Learning Catalytics and MyLab homework score on exam score: simple regression analysis

Variable Coefficient Estimate p-value Standard Error
Learning Catalytics .067 p<0.01 0.021
MyLab 0.247 p<0.001 0.046
Constant 48.724 p<0.001 3.723
R-squared 0.109

Table 1. Simple Regression Analysis: Dependent Variable is Exam Score in Percentage Points, Standard Errors in Parentheses, Spring 2017 (n=545)

The Student Experience

Responses from the Spring 2017 end-of-semester survey of Professor Schubert’s students indicate that the majority of responding students recognize the value of MyLab Economics.

  • 96% of respondents agreed that their understanding of the course material increased as a result of using MyLab.
  • 85% of respondents agreed that use of MyLab positively impacted their exam scores.
  • 90% of respondents agreed that MyLab provided additional resources that helped them learn more than they would have from traditional paper-and-pencil homework.
  • 94% of respondents agreed that they would recommend MyLab to another student.

Students responses to the question, “What are the benefits of MyLab?” include:

  • “MyLab was a helpful recap of what we discussed in class that day, and I still would have used it to study even if it was not mandatory.”
  • MyLab made me go over the material that I would otherwise not open. I actually had to open the book and read it and use it and it helped me understand the overall material. I think the real-world questions in MyLab helped, too.”
  • MyLab was a source of practice. Reading the textbook and going to class will without a doubt help me learn the material, but it is essential to practice it and apply the knowledge in order to do well.”
  • “It forced me to practice the material learned in class. It gave me the ability to test myself and figure out which areas I needed to focus on.”
  • “The MyLab homework provided a check for me to see if I really knew the material and as practice for exam questions.”
  • “I liked MyLab much better than other programs we used for homework. It’s easy to understand and comes with many resources that help students.”

MyLab was a source of practice. Reading the textbook and going to class will without a doubt help me learn the material, but it is essential to practice it and apply the knowledge in order to do well.

—Student, University of Delaware

Conclusion

Teaching large sections creates a challenge for instructors who believe regular homework and practice will lead to successful course completion, but do not have the bandwidth to manually grade assignments and provide students with frequent, timely feedback. Adopting a digital homework solution was an easy decision for Professor Schubert who was teaching more than 250 students per section. The immediate, individualized feedback and helpful learning aids in MyLab proved to be a combination that he trusted would support his students once they left his lecture and set out to apply the concepts on homework assignments. On the student survey, respondents agreed overwhelmingly, with 90% of students indicating that the MyLab resources helped them learn more than they would have from paper-and pencil homework.

In addition, the use of Learning Catalytics allowed Professor Schubert to keep his students engaged and on topic by periodically checking their comprehension of new concepts. He was able to identify misunderstandings of content in real time and take corrective action in lecture immediately. This was also a common theme on the student survey, as articulated by this student: “Learning Catalytics helped me understand the concepts that I didn’t understand, because it would be explained afterwards.” MyLab paired with Learning Catalytics gave Professor Schubert tools for inside and outside the classroom, keeping his students involved and prepared to master their homework assignments.

__________________________________________________________________________________________________

1 An alternative specification controlling for student demographics yields similar results.