MyITLab educator study explores relationship between average Dynamic Study Module score and final exam score at Kennesaw State University
- Data indicate that students earning higher Dynamic Study Module scores also earned substantially higher final exam scores.
- Students showing mastery of Excel by earning a letter grade of A, B, or C on the Excel Grader Project had Excel Simulation scores 29 percentage points higher than students who earned a D or F letter grade on the Excel Grader Project.
- In an end-of-semester student survey, most respondents agreed that the test-review-retest pattern of the Dynamic Study Modules helped them learn and remember chapter content.
Kennesaw State University, Kennesaw, GA
Computers and Your World
MyITLab; Technology in Action (eText) by Evans, Martin and Poatsy
Meg Murray, Professor
Results reported by
Traci Simons and Candace Cooney, Pearson Customer Outcomes Analytics Managers
- Locale: large, urban, four-year university located north of Atlanta
- Enrollment: approximately 33,000
- Student-faculty ratio: 21:1
- Graduation rate (6-year, Fall 2009 cohort): 41 percent
- Retention rate (Fall 2014 cohort): 78 percent
- Average age: 23
- Gender: 51 percent male
- Diversity: 42 percent minority
About the Course
Professor Meg Murray has been teaching for approximately 35 years, including 15 years at Kennesaw, where she and a colleague originated the Computers in Your World course ten years ago. Computers in Your World is a one-semester, three-credit course enrolling approximately 400 students each semester. It is open to all students and required of communication majors. Murray offers Computers in Your World in a fully online format with no scheduled on-campus class meetings. Students explore how computers and the Internet have revolutionized society. The course aims to help students become well-rounded and informed users of computing technologies. The course consists of three modules: 1) hardware and operating systems; 2) spreadsheet, presentation, and database software; and 3) networks, the Internet, and information literacy. Other topics include security, privacy, globalization, Web 2.0, diversity, and ethics.
Upon successful completion of this course, students should be able to:
- use computer applications to solve common problems encountered at school, work, or home;
- protect a networked computer from various security and privacy threats;
- organize, locate, copy and manipulate files in common operating system file managers;
- conduct research on the Internet, assessing the reputability of information sources;
- articulate a position on ethical issues regarding computers and the Internet; and
- assess how the Internet has impacted globalization and diversity.
Challenges and Goals
In Spring 2016, although Murray was currently using MyITLab, she was looking to make her course more interactive, ensuring that the course content and technology used was progressive and up-to-date. Having completed her own research on adaptive learning, the Dynamic Study Modules (DSM) in MyITLab presented an interesting opportunity for Murray to explore the hypothesis that adapting instruction to a student’s learning style might result in improved learning outcomes. For her Spring 2016 classes, Murray added the DSM to her MyITLab assignments as a way to engage her students in the course content and encourage them to complete more of the assigned course work. To further persuade students to complete homework in the course, Murray designated 70 percent of the course grade to come from assignments other than exams.
MyITLab is required; the program is used primarily by students working at home on a personal computer. Students use MyITLab for understanding content, working on office programs in the application, and testing. Murray’s goals for assigning work in MyITLab are to provide homework and practice opportunities, help students assess their own understanding of the course material and track their progress, and identify at-risk students. As the course instructor, Murray’s role is to assign content, homework, and assessments in MyITLab and provide support and remote monitoring to students using the program at home. All course content is included or referenced in the school’s learning management system, Desire2Learn (D2L). Course requirements, including exams, are conducted online, and all assignments are submitted either via D2L or MyITLab.
Murray anticipates that students will spend 2–4 hours per week working in MyITLab. Her students confirmed this on a voluntary, end-of-semester Spring 2016 survey (83 percent response rate) in which 45 percent of students said they spent 2–3 hours per week working in MyITLab, 48 percent of students said they spent 3–4 hours per week working in MyITLab, and an additional seven percent of students said they spent four or more hours working in the program.
Module Guides: The course is organized into modules. Generally, students complete one module per week during the Fall/Spring semesters and two modules per week during the Summer semester. A guide is provided for each module listing the objectives, activities, assignments, and assessments required for that module. Students are responsible for reading and completing all work outlined in the module guides. An average of 3–4 activities are assigned per module. Murray encourages students to read the textbook, review the MyITLab Sound Bytes or annotated PowerPoints, which are similar to voice-over lectures, and then begin the Dynamic Study Module assignment.
Dynamic Study Modules: Most modules include an adaptive learning exercise in MyITLab called Dynamic Study Modules. Students complete this exercise in conjunction with the textbook reading assigned for the module. These assignments are required and graded—the grade is based on the percentage of work completed by the due date. DSM questions continuously assess student performance and activity, using data and analytics to provide personalized feedback in real-time to reinforce concepts that target the individual student’s strengths and weaknesses. Students complete a set of questions with a unique answer format that also requires them to indicate their confidence level, and the questions repeat until the student can answer them all correctly and confidently. On the end-of-semester survey, students noted the following:
- 79 percent of students agreed or strongly agreed that the Dynamic Study Modules pattern of test-learn-retest helped them retain information about important course concepts.
- 62 percent of students agreed or strongly agreed that when completing the Dynamic Study Modules, the use of confidence levels when answering questions (‘I am sure’, ‘I am partially sure’, ‘I don’t know yet’) helped them identify chapter content they needed to focus on.
In particular, students on the survey often commented about the valuable answer feedback in the Dynamic Study Modules. “I liked how when you got an answer wrong in the Dynamic Study Modules it would explain why the right answer was correct, and that it made you re-answer the question two more times before counting it correct,” said a student. For additional information on how to assign DSM, visit Dynamic Study Module Implementation.
Excel/Access Assignments: To learn application software, students use MyITLab’s Simulation Trainings to practice application activities and are assigned Grader Projects in Microsoft Office as the application assessment. Simulation Trainings provide realistic, skill-based training so students can practice Microsoft Office application skills using multiple methods of completion. Students have unlimited attempts at completion, and the highest score is recorded in the gradebook. Learning aids called Read, Watch, and Practice are turned on. Grader Projects are outcomes-focused and offer Murray the ability to assign project-based assignments that assess skills needed for future jobs. Students submit projects for immediate grading and receive a Grader Project report with just-in-time feedback. Students have three attempts at completion for each Grader Project. Student feedback from the end-of-semester survey has been quite positive regarding the MyITLab exercises:
- 72 percent of respondents in the end-of-semester survey said they always used available learning aids if they were unable to start or complete an activity in the Simulation Training.
- 79 percent of respondents strongly agreed or agreed that the Simulation Trainings in MyITLab helped them get familiar with and practice skills in Microsoft Office before completing the Grader Project assignments.
- 65 percent of responders always or usually completed a second or third attempt on the Grader Project assignment if they were unhappy with their score on the first attempt.
Examinations: Students take two exams online in D2L, each consisting of 60 multiple-choice, true/false, and matching questions. Questions are self-created by Murray and are also taken from the Pearson test bank. The questions are mapped to objectives covered by the Dynamic Study Modules and other course assignments. Exams are open book and open notes, and students have 90 minutes for completion once the exam has been accessed.
There are no make-up assignments, and no late assignments are accepted. A make-up exam is only given when a student provides documented evidence of an excused emergency absence; however, Murray does provide extra credit opportunities which can be used to make up for missed assignments.
Finally, in addition to the Simulation Trainings, Grader Projects, and Dynamic Study Modules that students use in MyITLab, Murray also takes advantage of the program’s integrity violations report. Murray believes that because students know the program can detect violations, she doesn’t see many; however, she appreciates that they can easily be seen by both her and her students.
- 21% MyITLab Dynamic Study Modules
- 19% D2L assignments
- 15% Exam 1
- 15% Exam 2
- 14% Discussion posts
- 8% MyITLab Simulation Trainings
- 8% MyITLab Grader Projects
Results and Data
Grade distribution data in terms of course success (students earning an A, B, or C) indicate that students who earned higher Dynamic Study Module scores also earned higher final exam letter grades (figure 1).
- Students showing mastery of course content by earning an A, B, or C on the final exam earned Dynamic Study Module scores 18 percentage points higher than students who earned a D or F on the final exam.
- Students earning a final exam letter grade of A recorded an average Dynamic Study Module score of 89 percent.
Grade distribution data in terms of Excel success—students earning an A, B, or C on the summative Grader Project—indicate that students who earned higher average Simulation Training scores also earned higher Grader Project letter grades (figure 2).
- Students showing mastery of Excel by earning an A, B, or C on the Grader Project earned Simulation Training scores 29 percentage points higher than students who earned a D or F on the Grader Project.
Figures 3 and 4 are correlation graphs; correlations do not imply causation but instead measure the strength of a relationship between two variables, where r is the correlation coefficient. The closer the r value is to 1.0, the stronger the correlation. The corresponding p-value measures the statistical significance/strength of this evidence (the correlation), where a p-value <.05 shows the existence of a positive correlation between these two variables.
- A very strong positive correlation exists between average MyITLab Excel Simulation Training scores and MyITLab Excel Grader Project score, where r=.79 and p<.05.
- A very strong positive correlation exists between average Dynamic Study Module scores and final course scores, where r=.80 and p<.05. (Note: Dynamic Study Modules make up 21 percent of the final grade impacting this analysis).
For students, the formative MyITLab homework assignments are intended to help them identify where they are in terms of successfully completing the summative exams; it appears that performance on these assignments could be a leading indicator of course success (additional research is needed to develop and test this concept further).
Relationship between average Dynamic Study Module score and final exam letter grade
Figure 1. Relationship between Average Dynamic Study Module Score and Final Exam Letter Grade, Spring 2016 (n=85)
Relationship between average MyITLab Excel Simulation Training score and MyITLab Excel Grader Project letter grade
Figure 2. Relationship between Average MyITLab Excel Simulation Training Score and MyITLab Excel Grader Project Letter Grade, Spring 2016 (n=85)
Correlation between average MyITLab Excel Simulation score and MyITLab Excel Grader Project score
Figure 3. Correlation between Average MyITLab Excel Simulation Score and MyITLab Excel Grader Project Score, Spring 2016 (n=85)
Correlation between average Dynamic Study Module score and final course score
Figure 4. Correlation between Average Dynamic Study Module Score and Final Course Score, Spring 2016 (n=85)
The Student Experience
Responses from the Spring 2016 end-of-semester, voluntary survey of Murray’s students (83 percent response rate) indicate that the majority of responding students recognize the value of MyITLab.
- 90 percent of students agree or strongly agree that their understanding of the course material increased as a result of using MyITLab.
- 83 percent of students agree or strongly agree that the use of MyITLab positively impacted their exam scores.
- 79 percent of students agree or strongly agree that they would recommend MyITLab to another student taking this course.
Student survey responses to the question, “What did you like most about MyITLab?” include:
- “I like that it gives so many resources to help you understand the material better and pass the assignments. I also like that it gives you so many chances to get the questions right.”
- “I really like how the questions tell you when you are wrong, explain the correct answer, and then re-ask the question later on.”
- “The interactive sections in the e-book that showed pictures, as well as the interactive help desk training.”
- “I liked the test-retest of the DSM and I liked the tools to help you complete training assignments.”
Murray believes the adaptive nature of the Dynamic Study Modules did make her course more interactive and encouraged her students to complete all required assignments. Although the DSM added an additional layer to existing assignments, she noted that students were engaged and motivated to complete their work. Similarly, data indicated that students who showed mastery of course content by earning an A, B, or C on the final exam also had substantially higher average Dynamic Study Module scores, further demonstrating that the DSM may be an indicator of course success.