MyMarketingLab® educator study analyzes Dynamic Study Module and quiz scores at College of DuPage
- Data indicate a strong, positive correlation between Dynamic Study Module scores and MyLab quiz scores.
- Students who completed more than the average number of Dynamic Study Modules earned higher average quiz and exam scores.
- Students collectively agreed that the test-review-retest pattern of the Dynamic Study Modules helped them retain information about important course concepts.
College of DuPage, Glen Ellyn, IL
Principles of Marketing
Flipped, online, and face to face
MyLab Marketing with Principles of Marketing by Kotler and Armstrong
Mary Rojas-Carlson, Assistant Professor
Results reported by
Candace Cooney, Customer Outcomes Analytics Manager, Pearson
- Locale: large, suburban, public, two-year university
- Enrollment: approximately 28,000
- Student-faculty ratio: 24:1
- Full-time students: 35 percent
- Full-time retention rate: 68 percent
- First-time, full-time students: 17 percent
- Gender: 53 percent female
- Age: 63 percent of students 24 years or younger
- Diversity: 41 percent minority
About the Course
Mary Rojas-Carlson has been teaching for approximately 11 years, spending the past ten years at College of DuPage, where she has been teaching both Principles of Marketing and Economics. Principles of Marketing is a one-semester, three-credit course which enrolls about 450 students annually. The majority of students in this course are business majors, including transfer and returning education students. The course is the study of satisfying customer needs for goods and services, covering marketing environments, marketing planning, and marketing research. Target market identification, competitor analysis, and marketing strategy are also modeled.
Challenges and Goals
To encourage her students to read the textbook and come to class with some understanding of the chapter content, Rojas-Carlson created an untimed, five-question, multiple-choice quiz in Blackboard for each chapter. However, students did not receive automatic feedback regarding the accuracy of their quiz answers, and Rojas-Carlson was not noticing an increase in student comprehension of lecture content, either. Her goal was to offer her students an algorithmic method for completing homework that would require students to read the text prior to lecture. Because of this, the capabilities of the Dynamic Study Modules (DSM) in MyLab™ Marketing were appealing to Rojas-Carlson. Students would be required to read the textbook in order to complete the DSM prior to lecture, ensuring that they were ready to thoughtfully contribute and participate in class discussion and activities. Believing that students can be pushed to try harder, she hoped that by doing more work before class, students would be more involved in the lecture. Rojas-Carlson adopted both MyLab Marketing and the Dynamic Study Modules in Spring 2016.
MyLab Marketing is required; the program is used primarily by students working at home on a personal computer. Students use MyLab for understanding content, homework assignments, and some quizzing. Rojas-Carlson’s goals for assigning work in MyLab are to teach new concepts, provide homework and practice opportunities, help students assess their own understanding of the course material and track their progress, and identify at-risk students. As the course instructor, her role is to assign content, homework, and assessments in MyLab and provide support and remote monitoring to students using the program at home.
Rojas-Carlson anticipates that students will spend approximately one hour per week working in MyLab. Her students confirmed this on a voluntary, end-of-semester Spring 2016 survey (87 percent response rate)—48 percent of students said they spent 1–2 hours per week working in MyLab, while an additional 27 percent of students said they spent more than two hours each week working in the program.
Dynamic Study Modules (DSM) are the only homework assigned by Rojas-Carlson in MyLab, with a goal of encouraging her students to read the textbook. For students, Dynamic Study Modules continuously assess student performance and activity, then use data and analytics to provide personalized content in real-time to reinforce concepts that target each student’s particular strengths and weaknesses. For Rojas-Carlson, Dynamic Study Modules offer a robust reporting dashboard that delivers key analytics about learner performance, making it easy for her to adjust her lecture as needed. She can dedicate more class time to higher-order ideas, expand on complex concepts, and help students reinforce what they’ve learned through collaborative and real-world activities.
On the end-of-semester survey, students reported success using the Dynamic Study Modules:
- 94 percent of students strongly agreed or agreed that the test-review-retest pattern of the Dynamic Study Modules helped them to learn and remember chapter content.
- 76 percent of students strongly agreed or agreed that the use of confidence levels in the DSM (‘I am sure’, ‘I am partially sure’, ‘I don’t know yet’) helped them identify chapter content they needed to focus on.
Student responses when asked what they liked best about the Dynamic Study Modules:
- “I liked how it would shuffle the questions you answered incorrectly into the remaining questions to repeat them.”
- “The ability to build knowledge before the topic was discussed in class.”
- “They helped me a lot before tests and for taking notes for tests.”
- “If you get the Dynamic Study Modules wrong, a paragraph filled with more information is provided for a better understanding of the topic.”
Initially, students were expected to complete weekly quizzes in MyLab. The first six quizzes were given in MyLab and each was a ten-question, pre-built, multiple-choice quiz based on the Dynamic Study Module content. However, to encourage lecture attendance, Rojas-Carlson switched to paper-and-pencil, multiple-choice, in-class quizzes at week seven. Students worked in small groups and Rojas-Carlson was able incorporate application questions to generate collaboration and debate, with students sharing ideas and learning from one another.
Four tests are administered, each comprised of approximately 35 multiple-choice questions and several short-answer, application-oriented questions. Most questions are from the Pearson test bank, but Rojas-Carlson also writes her own questions, often using company names that she uses in lecture as real-world examples. Students have one hour and 45 minutes to complete these in-class tests. The final exam is cumulative, but optional, as students can drop their lowest test score and the four highest scores are averaged in the gradebook. Students earn a zero for any missed exam and there is no opportunity for a make-up test.
Individual research assignments are required, with students choosing 10 topics out of 25 options. These research assignments are related to chapter content and material covered in lecture. For example, a topic like new product development might find a student searching social media to find out what consumers are saying about Burger King’s new hot dog entrée. Most research takes students about 30 minutes, and Rojas-Carlson then uses their research assignments as relevant, real-world applications of the chapter topics in lecture. For these assignments, students work in pairs and create both a ten-minute, in-class presentation or activity and a four-page research paper.
Learning Management System (LMS) Integration
Rojas-Carlson was interested in the ability to integrate MyLab with Blackboard, the LMS system used at DuPage, particularly the seamless integration of the gradebooks. Ultimately, she opted to integrate her course for the following reasons:
- Grade transfer: grades are easily transferable from MyLab to Blackboard and there is one single gradebook for the course;
- Single sign-in process: students are ready to work in MyLab on the first day of class; and
- Content linking: ability to link to MyLab directly from Blackboard.
The availability of grade syncing made the decision to integrate MyLab and Blackboard quite simple. Students now have just one access code and a single sign-in process instead of the need to sign in to Blackboard and additionally sign in to MyLab. Rojas-Carlson believes that easy access to MyLab is important, particularly for remote online students; anything that minimizes potential start-up issues early in the semester is appreciated. As one student on the end-of-semester survey said, “I liked that it was easy to get to my assignments.” The single sign-in process has resulted in a simple way for students to begin their work in MyLab.
- 100 percent of students strongly agreed (70 percent) or agreed that they were able to access MyLab through Blackboard and appreciated not needing a second, separate login for MyLab.
- 94 percent of students strongly agreed (60 percent) or agreed that the sign-in and registration process for MyLab was simple and fast through Blackboard.
- 40% Tests
- 20% Dynamic Study Modules
- 20% Homework quizzes
- 10% Individual research assignments
- 5% In-class project
- 5% Participation
Results and Data
Figure 1 is a correlation graph; correlations do not imply causation but instead measure the strength of a relationship between two variables, where r is the correlation coefficient. The closer the r value is to 1.0, the stronger the correlation. The corresponding p-value measures the statistical significance/strength of this evidence (the correlation), where a p-value <.05 shows the existence of a positive correlation between these two variables.
- A strong positive correlation exists between average MyLab quiz scores (Chapters 1–6 only) and average Dynamic Study Module scores (Chapters 1–6 only) where r=.63 and p<.05.
- After chapter six, quizzes were modified to be in-class and paper-and-pencil with a somewhat different format. Analysis showed a moderate correlation where r=.40 and p<.05 exists between all Dynamic Study Module scores (Chapters 1–20) and all quizzes (Chapters 1–20).
For students, the formative Dynamic Study Modules are intended to help them identify where they are in terms of successfully completing the summative quizzes and tests; it appears that performance on these assignments could be a leading indicator of course success (additional research is needed to develop and test this concept further).
Students were divided into two groups based on the average number of Dynamic Study Modules they completed. Students who completed at least 18 out of 20 Dynamic Study Module assignments earned higher average quiz and exam scores than students who completed fewer than 18 out of 20 Dynamic Study Module assignments (figure 2).
- Average number of Dynamic Study Modules skipped: <3
- Students who completed at least 18 out of 20 Dynamic Study Module assignments had average quiz grades 10 percentage points higher than students who completed fewer than 18 out of 20 Dynamic Study Module assignments.
- Students who completed all Dynamic Study Module assignments had average exam grades six percentage points higher than students who completed fewer than 18 out of 20 Dynamic Study Module assignments.
- 42 percent of students completed all Dynamic Study Module assignments.
Correlation between average MyLab quiz score and average Dynamic Study Module score (chapters 1-6)
Figure 1. Correlation between Average MyLab Quiz Score (Chapters 1–6) and Average Dynamic Study Module Score (Chapters 1–6), Spring 2016 (n=38)
Relationship between Dynamic Study Module completion and average quiz and exam scores
Figure 2. Relationship between MyLab Dynamic Study Module Completion and Average Quiz and Exam Scores, Spring 2016 (n=38)
The Student Experience
Responses from the Spring 2016 end-of-semester, voluntary survey of Rojas-Carlson’s students (87 percent response rate) indicate that the majority of responding students recognize the value of MyLab.
- 94 percent of students strongly agree or agree that their understanding of the course material increased as a result of using MyLab.
- 85 percent of students strongly agree or agree that the use of MyLab positively impacted their quiz and exam scores.
- 91 percent of students strongly agree or agree that they would recommend MyLab to another student.
Student survey responses to the question, “What did you like most about MyLab?” include:
- “I liked that it was interactive, instead of just reading out of a textbook.”
- “If you didn’t answer a question correctly, there was an opportunity to learn from it.”
- “It was easy to use and I saw results from using the system on my homework and tests!”
- “I liked how if I got an answer wrong, the [DSM] would make me redo it a couple times to make sure I understood the right answer.”
Intrigued by the adaptive learning opportunity presented by the Dynamic Study Modules, Rojas-Carlson hoped that requiring students to complete DSM before class would encourage them to read the textbook prior to class and attend lecture with a clearer idea of what chapter content they understood and the topics they did not. The modules deliver just 25–30 questions to maintain motivation while also maximizing learning. Analysis of Spring 2016 data suggest that Rojas-Carlson and her students made progress related to these goals. Most students in the end-of-semester survey found the DSM to be quite successful in helping them remember and understand chapter content. One student said, “I could take notes during the DSM and they helped me remember key concepts,” while another similarly said, “[The DSM] told me what concepts I still needed to learn.” Additionally, with her students coming to lecture having read the textbook, Rojas-Carlson can more often incorporate case studies, new items, videos, group work, and other experiential activities during class, focusing more on real-world applications that are relevant and significant to her students.