MyLab IT educator study examines impact on test scores at Durham College

EDUCATOR STUDY

MyLab IT educator study examines impact on test scores at Durham College

Key Findings

  • Students with an overall MyLab IT score of 70 percent or higher had average test scores 29 percentage points higher than students with an overall MyLab IT score of less than 70 percent.
  • 88 percent of student survey respondents said they were able to use the Word skills learned from MyLab IT in real-life situations (another course, a job, internship, etc.) during the course of the semester.
  • Assigning MyLab IT Simulation Trainings in advance of class time facilitates the flipped classroom environment preferred by Professor Umrah.
  • Umrah sees the Project Creation Tool in MyLab IT as a valuable feature of the program, as it allows him to customize assignments and bring better consistency to project grading.

School name
Durham College, Oshawa, Ontario, Canada

Course name
Business Computer Applications

Course format
Face to face and flipped classroom

Course materials
MyLab IT for Exploring Series

Timeframe
Fall 2016

Educator
Howard Umrah, Professor and Program Coordinator of Business Fundamentals Program

Results reported by
Sara Owen, Pearson Customer Outcomes Analytics Manager

Setting

Founded in 1967, Durham College of Applied Arts and Technology is located on the east end of the greater metropolitan Toronto area, with campuses in Oshawa and Whitby, a learning site in Pickering, and community employment services in Uxbridge, Port Hope, Port Perry, Beaverton, Oshawa, and Bowmanville. More than 12,000 full-time post-secondary and apprenticeship students are enrolled, in addition to thousands of part-time, continuing education, and/or online studies students. Currently more than 140 full-time, market-driven programs are offered with the goal of preparing students to find rewarding employment. A variety of diploma-to-degree options through agreements with universities in Ontario and around the world are also available. Finally, the School of Continuing Education (part-time and OntarioLearn study) offers more than 1,500 courses, 50 certificate programs, 14 diploma programs, three post-graduate programs, and one apprenticeship certificate in a wide variety of career and personal-interest subject areas.

About the Course

Business Computer Applications I (two credit hours) is the first half of a course introducting students to computer skills required in business today. The 15-week course provides over 400 first-year business students with Microsoft Word and PowerPoint skills necessary to be successful in the business community. In total, about seven educators teach the Business Computer Applications I course.

Students receiving credit for this course will have demonstrated their ability to:

  • Utilize the College’s intranet and learning management system features efficiently.
  • Store and retrieve files to and from the network drives or other sources (hard drives, USB, etc.) in accordance with basic file management principles.
  • Use Microsoft Word features relating to formatting, layout, and reference tools to create professional documents and templates.
  • Create PowerPoint presentations suitable to various types of audiences using a variety of advanced PowerPoint features.

Durham College has a laptop program that equips all students with a personal computer. Classrooms are equipped with wireless Internet, so students are expected to bring their laptops and come to class ready to work.

Challenges and Goals

Professor Howard Umrah started using MyLab IT three years ago after being asked to teach an Introduction to Excel course at the college. Umrah teaches a flipped-style course and liked being able to assign MyLab IT Simulation Trainings for students to complete before coming to class. He notes that in assigning this pre-class work, “Students come to class with some basic understanding of the content about to be covered, and they also come with questions.” Umrah also found MyLab IT useful because of its automatic grading.

Umrah is coordinator for the Business Fundamentals Program in the School of Business, IT & Management and is also the course lead for Business Computer Applications I. To bring consistency to the course, he created a course shell in MyLab IT with identical exercises and assessments for all sections to use. Umrah teaches two sections of Business Computer Applications I in the Fall semester and one section of Business Computer Applications II in the Spring.

Implementation

When students purchase MyLab IT, they also gain access to the corresponding Exploring eText. Throughout the course of the semester, students are assigned eight Simulation Trainings in MyLab IT. Students must receive a minimum score of 50 percent to receive credit and have unlimited attempts on the exercises, as they are intended to be low-stakes practice/learning opportunities. Each Simulation Training is due immediately before class begins, when the content is covered in more detail by Umrah.

Students also complete eight Grader Projects in MyLab IT during class time. Students have five attempts on each Grader Project. Umrah recently increased this from three to five attempts in order to give students more opportunities to improve their grades. Umrah wants this to be a low-stakes learning tool, so if students do not do the Grader exercise correctly the first time, they have multiple chances to make up that mark. Students must achieve a minimum score of 70 percent, and the highest score is counted. Umrah notes that some eager students who finish their Grader Projects during class ask to have the next class session’s Simulation Trainings made available and will sometimes complete the Simulations before they leave class. A majority of students complete the Simulation Trainings in the middle of the time they have available, and only a few wait until the very last minute.

Two in-class tests are given during the semester, completed as Grader Projects in MyLab IT. The tests mirror the skills learned through the Simulation Trainings and Grader Projects.

Students also complete two Projects over the course of the semester. One is a Microsoft Word project where students conduct research and present their findings in a professional research document. The second project is a PowerPoint project based on the Word project.

Building on MyLab IT knowledge
Umrah received basic training in MyLab IT from his local Pearson rep. Training with the Project Creator Tool was conducted by a faculty member from another college, and Umrah received additional one-on-one training from Pearson’s IT implementation specialist over the summer. New faculty who will be teaching with MyLab IT at Durham College attend a training session with the local Pearson rep, plus they have the advantage of using a ready-to-use course shell in MyLab IT with all the weekly exercises selected and loaded in advance by Umrah.

Over the past three years, Umrah has built on his knowledge of the MyLab IT program. At first, he found it challenging to use, but has now learned where to find what he is looking for. He has discovered there is a plethora of exercises that can be assigned. Umrah had never used the Project Creation Tool (PCT) until recently, but now sees it as a valuable feature of the program. Umrah decided to make students’ PowerPoint project (worth 10 percent of the final grade) into a PCT Grader Project so that MyLab IT would automatically score it, bringing consistency to the assignment grading. He plans on using PCT-created Grader Projects for the Excel-based Business Computer Applications II course next semester.

Best practices
Umrah recommends that new users be aware of the Browser Tune Up that students need to do before using MyLab IT and also ensure that students remember that pop-up blockers must be turned off. He notes, “This simple thing causes major headaches and frustrations, and students will take that out on the instructor.”

Assessments

  • 40% Tests (2)
  • 25% MyLab IT Grader Projects (8 in-class assignments)
  • 20% Projects (2)
  • 10% MyLab IT Simulation Trainings (8 self-study assignments)
  • 5% Assorted assignments

Results and Data

Figures 1 and 2 are correlation graphs; correlations do not imply causation but instead measure the strength of a relationship between two variables, where r is the correlation coefficient. The closer the value is to 1.0, the stronger the correlation. The corresponding p-value measures the statistical significance/strength of this evidence (the correlation), where a p-value <.01 shows the existence of a positive correlation between these two variables.

A strong positive correlation exists between the overall MyLab IT assignment score and the average of the two course tests, where r=.78 and p<.001 (figure 1). The two course tests are designed to mirror the skills learned in the Simulation Trainings and Graders. It appears that performance on these assignments could potentially be a leading indicator of exam success (additional research is needed to develop and test this concept further).

Correlation between MyLab IT assignments and tests

Figure 1. Correlation between MyLab IT Assignment Average and Course Test Average, Fall 2016 (n=40)

A very strong positive correlation exists between the MyLab IT self-study Simulation Training assignments and the MyLab IT in-class Grader Projects, where r=.80 and p<.001 (figure 2). By completing Simulation Trainings in advance of class time, students get a head start on learning the content to be covered that day in class. It appears that performance on the self-study Simulation Trainings could potentially be a leading indicator of success on in-class Grader Projects (additional research is needed to develop and test this concept further).

Correlation between self-study Simulation Trainings and in-class Grader Projects

Figure 2. Correlation between MyLab IT Self-study Simulation Trainings Score and In-class Grader Projects Score, Fall 2016 (n=40)

Students with an overall MyLab IT score (Simulation Trainings + Grader Projects which count as 35 percent of the final grade) of 70 percent of higher had average test scores 29 percentage points higher than students with overall MyLab IT scores less than 70 percent (figure 3).

Average test scores for students with MyLab IT score 70 percent and higher and less than 70 percent

Figure 3. Fall 2016 Average Test Score for Students with an Overall MyLab IT Score 70 Percent or Higher (n=23) and for Students with an Overall MyLab IT Score Less than 70 Percent (n=17)

The Student Experience

An end-of-semester online survey was sent to approximately 70 students in two sections to gather information on MyLab IT usage and student impressions on the MyLab IT experience. A total of 27 students took the survey, a response rate of 39 percent. The following bullet points summarize student responses:

MyLab IT access:

  • 91 percent of students (20 out of 22) reported they were able to successfully log into MyLab IT during the first week of class (5 students did not remember).

MyLab IT use:

  • 63 percent of students reported spending 1–2 hours working in MyLab IT outside of class time; 30 percent reported spending 2–3 hours; 7 percent reported spending 3–4 hours.
  • 60 percent of students agreed, “The Simulation Trainings in MyLab IT helped me get familiar with and practice skills in Microsoft Word and PowerPoint before completing the Grader Projects.” (18 percent neither agreed nor disagreed, 22 percent disagreed.)
  • 56 percent of students agreed, “The Grader Projects in MyLab IT helped me get familiar with and learn skills in Microsoft Word and PowerPoint.” (22 percent neither agreed nor disagreed; 22 percent disagreed.)

Learning aids/multiple attempts:

  • 48 percent of students always and 30 percent sometimes used the available learning aids (Read, Watch, Practice) for assistance when unable to start or complete a Simulation Training. One student noted, “That’s how I got 100 percent each time.”
  • 52 percent of students said they always or usually completed a second or third attempt on the Grader Projects if they were unhappy with their grade on the first attempt.
  • 63 percent of students always or usually used View Submissions in MyLab IT to help correct their errors if they were unhappy with the Grader Project score received on the first attempt.

Career readiness/skill transfer:

  • 88 percent of students (21 out of 24) said they were able to use the Word skills learned from MyLab IT in real-life situations (another course, a job, internship, etc.) during the course of the semester.
  • 59 percent of students (13 out of 22) said they were able to use the PowerPoint skills learned from MyLab IT in real-life situations (another course, a job, internship, etc.) during the course of the semester.

Overall experience:

  • 41 percent of students would rate their experience with MyLab IT as excellent or good.
  • 44 percent of students agreed that they would recommend MyLab IT to other students taking the course (26 percent neither agreed nor disagreed; 30 percent disagreed).

Selected responses to the question “What did you like most about MyLab IT” include:

  • “What I liked about MyLab IT was the self-study assignments because the learning aids were very useful to learn a lot more about Microsoft and to earn that 100 percent on our self-study assignments.”
  • “What I liked best about MyLab IT was the practice aid. It was very helpful.”
  • “Helped me learn a few things I needed to end up knowing for other classes presentations, projects.”
  • “I liked how I could see my grades right away and see what I did wrong.”

Conclusion

MyLab IT has allowed Umrah to teach in his preferred flipped-style format and has simplified his life with its autograding. The Project Creation Tool has given Umrah new opportunities to customize the course and also standardize project grading. Looking ahead, Umrah is working on a project with the end goal of certifying all first-year Durham College business students in Microsoft Excel. Based on what he’s learned about MyLab IT, Umrah believes “MyLab IT offers a lot in terms of preparation for that exam.”

 

MyITLab is now MyLab IT. New name. Same experience.

0 Comments

Leave a reply

Your email address will not be published. Required fields are marked *

*