MyLab Accounting educator study examines Learning Catalytics and homework scores at University of North Carolina – Greensboro

EDUCATOR STUDY

MyLab Accounting educator study examines Learning Catalytics and homework scores at University of North Carolina – Greensboro

Key Findings

  • Data indicate a very strong, positive correlation between Learning Catalytics scores and MyLab homework scores.
  • Students earning Learning Catalytics scores above average earned higher average MyLab homework, mid-term exam and final exam scores than students who scored below average on Learning Catalytics.
  • A majority of student respondents (80 percent) on an end-of-semester survey agreed that Learning Catalytics encouraged more interactivity in lecture and helped them focus during class.

School name
University of North Carolina, Greensboro, NC

Course name
Financial Accounting

Course format
Face to face

Course materials
MyLab Accounting and Learning Catalytics™ with Horngren’s Financial and Managerial Accounting by Miller-Nobles, Mattison and Matsumura

Timeframe
Fall 2016

Educator
Amanda Cromartie, Lecturer

Results reported by
Candace Cooney, Pearson Customer Outcomes Analytics Manager

 

Setting

  • Locale: large, urban, public, liberal arts, four-year research university, a constituent institution of the University of North Carolina system
  • Enrollment: almost 20,000 students
  • Four-year graduation rate: 32 percent
  • Average freshman retention rate: 76 percent
  • Faculty-student ratio: 18:1
  • Class size: 25 percent with 20 or fewer students
  • Gender: 65 percent female
  • Diversity: 27 percent resident undergrads are African-American

About the Course

Amanda Cromartie has been teaching introductory financial accounting courses for approximately 5 years at UNCG. There are two introductory financial accounting courses at UNCG. Both are one-semester, three-credit hour courses. Financial Statement Preparation and Disclosures is the introductory course required for Accounting majors and enrolls about 300 students annually. Accounting students must complete the course with a grade of C (73 percent) or better. A separate Financial Accounting course is offered as part of the General Education requirement for all other Business majors and minors. This study focuses on the introductory course for Accounting majors, Financial Statement Preparation and Disclosure. It is the first semester of a two semester sequence that includes Managerial Accounting.

Upon completion of the Financial Statement Preparation and Disclosure course, students should be able to:

  • Appreciate the conceptual framework underlying financial accounting;
  • Understand the accounting information system and accounting cycle;
  • Comprehend basic accounting terminology and financial statement preparation, particularly the balance sheet and income statement; and
  • Demonstrate, through completion of a practice set, mastery of the accounting cycle and preparation of financial statements.

Challenges and Goals

In 2012, UNCG adopted MyLab Accounting to accompany their new textbook. Seeking a digital platform that offered more robust support for students with many opportunities for practice and feedback, they also appreciated the various options and different assignments available for homework.

Teaching large sections presents many difficulties, but among the most challenging is encouraging participation and involvement of all students in the class. It is easy for students to hide in a large class, particularly those who are having trouble understanding the lecture content. This led Cromartie to search for an interactive course complement that would help her identify the struggling students. She adopted a personal response system (PRS) but it was costly for students to purchase access to both the MyLab and the PRS. In Spring 2016, Cromartie chose to supplement her MyLab course assignments with Learning Catalytics (LC), a personal response system which coordinates with MyLab. Students no longer need to purchase another device as they could use any personal phone, tablet, or computer to interact with the program.

Additionally, the mid-term and final exams are multiple-choice, but the MyLab homework assignments are calculation-based problems; to bridge the gap between the homework and exam formats, Learning Catalytics responses are phrased as multiple-choice options. Recently, Cromartie was able to integrate MyLab Accounting with their school learning management system (LMS), Canvas, to minimize grade transferring and eliminate the need for multiple gradebooks, saving her time.

Implementation

Learning Management System Integration

Cromartie opted to integrate her MyLab course with Canvas for the following reasons:

  • Grade transfer – grades are easily transferable from MyLab to Canvas and there is one single gradebook for the course;
  • Single sign-in process – students are ready to work in MyLab on the first day of class; and
  • Content linking – ability to link to MyLab directly from Canvas.

Cromartie’s students now have just one access code and a single sign-in process instead of the need to log in to Canvas and additionally sign in to MyLab. This results in a clear-cut way for students to start their MyLab assignments, ensuring that they are ready to work from the first day of class, as indicated on a voluntary, end-of-semester survey (71 percent response rate) in Fall 2016:

  • 98 percent of students strongly agreed or agreed that they were able to access MyLab through Canvas and appreciated not having a separate, second log in and password for MyLab.
  • 100 percent of students strongly agreed or agreed that the registration and log in process for MyLab through Canvas was quick and simple.

MyLab Accounting is required; the program is used primarily by students working at home on a personal computer. Students use MyLab for understanding content, homework assignments, and additional practice. Cromartie’s goals for assigning work in MyLab are to introduce new concepts, provide homework and practice opportunities, and to help students assess their own understanding of the course material and track their progress. As the course instructor, her role is to assign content and homework in MyLab and provide remote support to students using the program at home. Periodically, Cromartie uses the MyLab gradebook to check for a common area of consistent challenge, so she can remediate difficult topics prior to exams.

Cromartie has high expectations for her students; she explains that the material is difficult and that it is a demanding, time-intensive course. Students must complete the course with a final score of 73 percent (equivalent to a letter grade of C) or above to pass. She encourages them to read assigned material before lecture, focusing extra time on chapters 1-4, as they form the key structure of the course. She anticipates that students will spend at least four hours per week working in MyLab. Her students confirmed this on a voluntary, end-of-semester Fall 2016 survey77 percent of students said they spent more than 4 hours per week working in MyLab, while an additional 36 percent of students said they spent 3-4 hours working in the program.

Lecture

Cromartie uses the PowerPoint slides that accompany the textbook as an outline for lecture. She makes them interactive by leaving blank lines and spaces for students to fill in as they progress through class. She also incorporates multi-part problems that simulate what they can expect to complete as part of MyLab homework assignments and follows this up with multiple-choice Learning Catalytics questions (quiz) similar in type to her exam questions. After class, she posts review questions and worked solutions to Canvas so students can review any missed questions. Students may use tutors available through the department or Cromartie’s office hours to fill in any blanks they may have missed in class or to discuss concepts or problems in more detail.

Learning Catalytics

Learning Catalytics (LC), a bring-your-own-device personal response system, complements lecture instruction by using real-time questions and answers to capture student feedback to content questions Cromartie poses. A range of data is captured, allowing her to assess student understanding instantly. Generally, Cromartie offers three to six LC questions per chapter. Questions relate to the current chapter content so students are expected to prepare by reading the chapter before attending class. The questions derive from the Pearson database of LC questions and are crafted around the learning objectives Cromartie is covering in lecture.

Learning Catalytic grades are a combination of participation and in-class quiz (accuracy) scores. To allow students to acclimate to the process, early semester questions are participation-grade only and transition to including accuracy-grading as the semester progresses. Up to half the student grade is based on submitting a response to the question (participation) and up to half of the grade may be based on the accuracy of the response. The lowest LC grade is dropped from calculation of the final LC grade to allow some flexibility to the no in-class make-up quiz policy. Additionally, attendance is taken on random class dates using LC.

Students participating in the end-of-semester survey shared the following:

  • 87 percent of students strongly agreed or agreed that the in-class Learning Catalytics questions helped their understanding of lecture content or helped them identify misconceptions they may have had about the material.
  • 92 percent of students found it helpful when the class scored low on an LC question, Cromartie would explain the concept again and ask a similar question on the same material to confirm students understood the topic before moving on.
  • 82 percent of students strongly agreed or agreed that Learning Catalytics was an important part of the learning process that helped them do well in this course.
  • 80 percent of students strongly agreed or agreed that Learning Catalytics made lecture more interactive and helped them focus on the material being covered.

MyLab Accounting Problem Sets

Cromartie creates her MyLab homework assignments using end-of-chapter problems. There is one assignment per week with a firm due date, comprised of 10 multi-part questions with some matching questions to assess terminology. Students have five attempts at completion and the ‘Help Me Solve This’ learning aid is turned on. On the student survey, 85 percent of students indicated that they always or usually utilized the ‘Help Me Solve This’ when unable to start or complete a homework problem. Many student survey comments acknowledge how beneficial ‘Help Me Solve This’ is: “When I didn’t know how to do something, the ‘Help Me Solve This’ was very helpful and helped me understand better what I was doing” and “‘Help Me Solve This’ was the best part about homework!”

Cromartie encourages her students to use the optional Personal Study Plan and Dynamic Study Modules as preparation for class or review before exams. In fact, she believes the Study Plan is as good as any other MyLab option at gauging mastery. Although voluntary, students on the end-of-semester survey found these tools very helpful:

  • 59 percent of students used the Study Plan or Dynamic Study Modules on their own for practice or to help identify chapter material they were struggling with.

Students shared the following comments on the fall survey:

  • “The Dynamic Study Modules helped me practice and figure out what I needed to go back and study a bit more.”
  • “The Study Plan questions allowed me to see what I did and did not have a proper understanding of. Not only that, recognizing the areas I needed improvement on was easy and allowed me to further develop my understanding.”
  • “The Dynamic Study Modules helped me understand the main concepts by testing me on the same question until I got it right.”
  • “The Study Plan really helped me understand what I didn’t know and to prepare for exams.”

Additional course assessments

  • Practice set – Students complete a comprehensive paper-and-pencil practice set with an Excel component to solidify their understanding of the accounting cycle.
  • Exams – Cromartie assigns four mid-terms and a comprehensive final exam. Mid-terms are comprised of 30-35 multiple-choice questions and are timed to 75 minutes. The final exam is a 50-question multiple-choice exam timed to three hours. All exams are closed book and proctored. There are no make-up exams, so an excused absence from a mid-term will lead to the final exam having a higher weight.

Assessments

  • 60% Mid-term exams (4)
  • 20% Final exam
  • 7.5% Practice set
  • 7.5% MyLab problem sets (12)
  • 5% Learning Catalytics questions and class participation

Results and Data

Figure 1 is a correlation graph; correlations do not imply causation but instead measure the strength of a relationship between two variables, where r is the correlation coefficient. The closer the r value is to 1.0, the stronger the correlation. The corresponding p-value measures the statistical significance/strength of this evidence (the correlation), where a p-value <.05 shows the existence of a positive correlation between these two variables.

  • A strong positive correlation exists between average Learning Catalytics scores and average MyLab homework scores, where r=.77 and p<.05 (figure 1).

The correlation of average Learning Catalytics score to average exam score was moderate, where r=.43. Because the LC score is up to half participation only, we can’t draw any true conclusions from this correlation. However, Cromartie empirically asserts that Learning Catalytics bridges the gap from homework completion to exam success, due to the similarity in question format between the LC questions and exam questions. Her students agree based on this end-of-semester survey response:

  • 88 percent of students strongly agree or agree that the multiple-choice format of the in-class Learning Catalytics questions helped them prepare for the multiple-choice exams.

Correlation between Learning Catalytics score and MyLab homework score

Figure 1. Correlation between Average Learning Catalytics Score and Average MyLab Homework Score, Fall 2016 (n=83)

Students were divided into two groups based on the average Learning Catalytics score. Students who earned higher than average Learning Cataltyics scores earned higher average MyLab homework scores, average mid-term exam scores and average final exam scores than students who scored below average. (figure 2).

  • Average Learning Catalytics score: 96 percent
  • Students earning LC scores above average:
    • Earned MyLab homework scores 19 percentage points higher than students scoring below LC average scores
    • Earned mid-term exam scores 14 percentage points higher than students scoring below LC average scores
    • Earned final exam scores 15 percentage points higher than students scoring below LC average scores
  • 73 percent of students earned an LC score higher than average (n=61)

A t-test, which measures whether the means of two groups are statistically different was used to compare the Learning Catalytics score to average MyLab homework scores, mid-term exam scores and final exam scores. Results of the t-test show that p<0.05 for Learning Catalytics and the three scores noted in figure 2, indicating that this increase is statistically significant.

Relationship between Learning Catalytics score and homework score

Figure 2. Relationship between Average Learning Catalytics Score and Average MyLab Homework Score, Average Mid-term Score and Average Final Exam Score, Fall 2016 (n=83)

The Student Experience

Responses from the Fall 2016 end-of-semester, voluntary survey of Cromartie’s students indicate that the majority of responding students recognize the value of MyLab Accounting.

  • 94 percent of students strongly agree or agree that their understanding of the course material increased as a result of using MyLab.
  • 88 percent of students strongly agree or agree that MyLab provided additional resources that helped me learn more than I would have from traditional pencil-and-paper homework.
  • 86 percent of students strongly agree or agree that the use of MyLab positively impacted their quiz and exam scores.
  • 91 percent of students strongly agree or agree that they would recommend MyLab to another student.

Student survey responses to the question, “What did you like most about MyLab Accounting?” include:

  • “MyLab gave me steps on how the concept works and explained how to work out some questions when I didn’t understand them. I like the many attempts we have to try a different problem, which becomes practice, and practice makes perfect.”
  • “[I liked] the practice questions without needing to do questions out of the book. Thank you modern day homework!”
  • “I love the hands-on experience and practice of the accounting process and I equally appreciate being able to try the problems more than once, for the sake of practice, understanding, and my score.”
  • “I like that if you got an answer wrong, you can see exactly what you got wrong and what needs to be fixed.”
  • “[I liked] the videos on the learning objectives.”

Conclusion

Cromartie had been successfully using MyLab Accounting for several years when she added the Learning Catalytics personal response system to her course. As both a participation and assessment tool, LC enabled her to bring a real-time interactive component to the course that had the added benefit of helping students identify content they were potentially struggling with during lecture. Additionally, the multiple-choice nature of the LC questions helped her bridge the gap between MyLab homework problems that were not multiple-choice and exams that were multiple choice. Cromartie also notes that the CPA exam is a multiple-choice test, so the format of the LC questions allows her to show students how to strategize through the choice options as they will need to on both her course exams and the future CPA exam. To highlight the success of this strategy, as noted above, 88 percent of students strongly agree or agree that the multiple-choice format of the in-class Learning Catalytics questions helped them prepare for the course exams.

Sound advice from Cromartie for new users is to anticipate technical issues to happen! You can expect problems with access codes, temporary access, and integration linking too (technical issues may also arise during the semester but are uncommon). But if you plan ahead and are prepared to make adjustments when issues arise, you’ll be in the position to quickly get and keep things on track.