MyLab Management educator study observes Simulation scores and Personal Inventory Assessment usage at Colorado Mesa University

EDUCATOR STUDY

MyLab Management educator study observes Simulation scores and Personal Inventory Assessment usage at Colorado Mesa University

Key Findings

  • Data indicate a strong positive correlation between MyLab simulation and video exercise scores and average quiz scores in the online section.
  • Students earning higher MyLab simulation scores earned substantially higher average exam scores in both online and face-to-face sections.
  • A majority of students in both sections (70 percent) on an end-of-semester survey agreed that the MyLab simulations helped them practice decision making and apply the chapter content to a real-world business challenge.
  • All survey respondents (100 percent) agreed that the Personal Inventory Assessments in MyLab allowed them to reflect on the management concepts they were learning and apply them to their personal lives.

School name
Colorado Mesa University, Grand Junction, CO

Course name
Organizational Behavior

Course format
Face to face, online, and activity based

Course materials
MyLab Management with Organizational Behavior by Robbins and Judge

Timeframe
Fall 2016

Educator
Dr. Kyle B. Stone, Associate Professor

Results reported by
Candace Cooney, Pearson Customer Outcomes Analytics Manager

Setting

  • Locale: mid-sized, urban, four-year institution in western Colorado
  • Enrollment: more than 10,000 students
  • Undergraduate: 90 percent
  • Student faculty ratio: 22:1
  • First-time students: 23 percent
  • Full-time students: 78.5 percent
  • Freshman retention rate: 66 percent
  • Graduation rate: 36 percent (six-year)
  • Age 18-21: 55 percent
  • Gender: 54 percent female
  • Minority: 26 percent

About the Course

Kyle Stone has been teaching for six years, including four years at Colorado Mesa University (CMU), where Organizational Behavior (OB) is offered to approximately 225 students annually. This is a one-semester, three-credit activity-based course offered face-to-face and online. Organizational Behavior focuses on people and how they behave in the workplace with a key emphasis on improving performance of individuals that directly impact the overall performance of an organization. Students learn how people are motivated (or demotivated), the various factors that affect motivation, how people perform when working in groups or on teams, and how to integrate these concepts with organization structure and their impact on performance.

Upon satisfactory completion of this course, students should be able to:

  • Identify and differentiate between individual behaviors and recognize the influence of these behaviors on group and organizational dynamics;
  • Apply theories of group behavior by designing tools to accomplish group tasks more effectively;
  • Critique the effectiveness of OB tools developed; and
  • Analyze and critique various organizational theories.

Challenges and Goals

When Stone started at CMU four years ago, he and his colleagues reviewed the digital programs offered by several publishers, in anticipation of offering courses, including Organizational Behavior, online. Not having the resources or the bandwidth to create homework assignments and assessments on their own, they ultimately chose to adopt MyLab™ Management for its variety of assignments including video, simulation, and adaptive learning. Stone trusted that MyLab could deliver the content and connect it to learning at the same time, rendering lecture more supplemental. This would offer in-class students more opportunities for experiential learning and real-world application, while presenting online students with a similar experience through the same learning and assessment options.

Implementation

Learning Management System Integration
Stone chose to integrate his MyLab course with Desire2Learn (D2L) for the following reasons:

  • Single sign-in process – students are ready to work in MyLab on the first day of class; and
  • Content linking – ability to link to MyLab directly from D2L.

Stone’s students now have just one access code and a single sign-in process instead of the need to log in to D2L and additionally sign in to MyLab. This results in a clear-cut way for students to start their MyLab assignments, ensuring that they are ready to work from the first day of class, as indicated on a voluntary, end-of-semester survey (47 percent response rate) in Fall 2016:

  • 89 percent of students strongly agreed or agreed that they were able to access MyLab through D2L and appreciated not having a separate, second log in and password for MyLab.
  • 96 percent of students strongly agreed or agreed that the registration and log in process for MyLab through D2L was quick and simple.

MyLab Management is required; the program is used primarily by students working at home on a personal computer. Students use MyLab for understanding content, homework assignments, quizzing, and additional practice. Stone’s goals for assigning work in MyLab are to introduce new concepts, provide homework and practice opportunities, and to help students assess their own understanding of the course material and track their progress. As the course instructor, his role is to assign content and homework in MyLab and provide remote support to students using the program at home. Stone anticipates that students will spend approximately two hours per week working in MyLab, which may include reading the eText, watching videos, simulations, and completing assignments. Based on results of the end-of-semester survey, 75 percent of students in the online section acknowledged that they spent two or more hours using MyLab while just 40 percent of students in the face-to-face section indicated they spent two or more hours working in the program.

MyLab Assignments

Stone covers one chapter per week and students complete one MyLab assignment with several exercises each week, as well. While Stone’s face-to-face sections are only assigned the MyLab simulations and Personal Inventory Assessments, described below, he utilizes a greater variety of MyLab assignments in his online section, including:

  • Decision Making Mini-Simulations: Mini-simulations called ‘Try-It’ put Stone’s students in the role of manager as they make a series of decisions based on a realistic business challenge. The simulations change and branch based on their decisions, creating various scenario paths. Each simulation takes about 15-20 minutes and students have two attempts at completion, with the highest score being recorded as the grade. At the end of each simulation, students receive a grade and a detailed report of the choices they made with the associated consequences included.
  • Video assignments: ‘Watch It’ or ‘Think Like a Manager’ videos explore a variety of business topics related to the theory students are learning in class and assess students’ comprehension of the concepts covered in each video. Videos are less than ten minutes in length and students have two attempts to complete the five multiple-choice question quiz that follows the video. The highest score of the two attempts is recorded as the final grade. Students may work in groups but each individual needs to complete their own assessment.
  • Personal Inventory Assessments (PIA): these self-assessment activities promote self-reflection and help Stone’s students connect personally with management concepts. His students receive a written explanation and graphic display of their results, and Stone uses them to promote class discussion and other activities.
  • Chapter quizzes: MyLab quizzes are comprised of 20-25 multiple-choice, concept-based questions and students have two attempts at completion before checking their work. Quizzes are timed at 30 minutes.

On the end-of-semester survey, students reported the following about MyLab assignments:

  • 70 percent of respondents (both sections) agreed that the simulation exercises helped them practice decision making and apply the chapter content to a real-world business challenge.
  • 88 percent of respondents (both sections) agreed that the detailed report of their answer choices at the end of the simulation exercise allowed them to see the consequences of the decisions they made.
  • 92 percent of respondents (online section only) agreed that the video exercises helped them visualize the chapter contents in action.
  • 100 percent of respondents (both sections) agreed that the PIA allowed them to reflect on the concepts they were learning and apply them in their personal life.
  • 100 percent of respondents (both sections) agreed that the PIA were a fun application of management concepts.

Student comments from the survey about their experience with the PIA include:

  • “I really enjoyed these assignments. They put this class into perspective for me and really got me thinking of how I am personally and how I would handle situations like this.”
  • “I enjoyed the assignments and think that they gave great situational examples that could be relevant to future managers.”
  • “Gave great insight as to where my views were compared to the rest of the world. I thought they were fun and interesting.”
  • “They were overall very positive because I was able to apply what is being learned to myself and see where I would fit into the business world or what can be my weaknesses and strengths.”
  • “I thoroughly enjoyed what these exercises contributed to my learning. The interactive concept is so helpful when it comes to online classes.”

Employability/21st Century Skills

Stone’s use of simulation and video learning in MyLab Management is helping to arm his students with 21st century employability skills. There is a growing body of knowledge about what soft skills are needed to create work opportunities for college graduates today, but these skills themselves can be a challenge for students to develop. The majority of hiring managers place significant value on proficiency in these soft skills, yet less than 38 percent of employers say college graduates are prepared in these areas. The MyLab framework, which includes simulation, video decision making, and written communication, moves away from memorization and content definition and is instead organized to help learners understand, identify, and develop these core soft skills:

  • Communication
  • Decision-making
  • Problem-solving and analytical skills
  • Knowledge application

Additional Course Assignments

Exams: multiple-choice exams are taken online through D2L, comprised of approximately 40 application-oriented questions total, ten per chapter. Once the exam has been opened, it must be done in one sitting, and students have 50 minutes for completion.

Case/Ethical Dilemma assignments (face-to-face section only): students are assigned at least one ethical dilemma or case incident from the end-of-chapter assignments for each chapter and are required to write a short, 150-word essay response to each question. Stone expects responses to be supported by facts, not just opinion, from the textbook, lecture, discussions, or other credible sources. Students may work in groups but each individual must submit his or her own assignment.

Discussion board (online section only): weekly discussion topics centered on chapter readings are posted weekly. Stone’s goal is to enhance student learning through insightful comments and discussion. Students must post a minimum 150-word paragraph to the discussion topic posted by Stone on Monday by end-of-day Wednesday, and also respond to one or two other student posts by end-of-day Sunday. Postings are graded using a rubric posted in D2L.

Manager Profile: students connect with someone in a position of management and develop a Manager Profile of the individual through the following assignments:

  • A structured interview conducted with the chosen manager;
  • A written report using a standard template;
  • Create a video using Panopto on D2L presenting their profile; and
  • Review and comment on three other student-created Manager Profiles.

Ethical Lens activities: CMU is a member of the Daniels Fund Ethics Initiative which provides resources to foster a higher standard of ethics for business school students. As part of a four-part assignment, Stone’s students:

  • Use the Ethical Lens Inventory to determine their preferred ethical perspective and reflect on how their personal core values influence their decisions;
  • Create their own personal code of ethics;
  • Write a two-to-three-page reflection paper working through the Baird Decision Model™ (OB concept) demonstrating strengths, challenges, and opportunities of their ethical lens; and
  • Submit a final version of the report articulating their values and how they inform their ethics code.

Assessments

Online

  • 24% Exams (5)
  • 20% Discussion board assignments
  • 12% MyLab video (Watch-It) assignments (18)
  • 12% MyLab chapter quizzes (18)
  • 12% Manager Profile
  • 8% MyLab simulation (Try-It) assignments (12)
  • 7% Ethical Lens activities
  • 6% MyLab Personal Inventory Assessments (18)

Face to face

  • 30% Exams (5)
  • 30% Case/Ethical Dilemma assignments (18)
  • 15% Manager Profile
  • 10% MyLab simulation (Try-It) assignments (12)
  • 8% Ethical Lens activities
  • 7% MyLab Personal Inventory Assessments (18)

Results and Data

Figures 1 and 2 are correlation graphs; correlations do not imply causation but instead measure the strength of a relationship between two variables, where r is the correlation coefficient. The closer the r value is to 1.0, the stronger the correlation. The corresponding p-value measures the statistical significance/strength of this evidence (the correlation), where a p-value <.05 shows the existence of a positive correlation between these two variables.

  • A strong positive correlation exists between MyLab simulation scores and average exam scores for the face-to-face, where r=.61 and p<.05 (figure 1).
  • A strong positive correlation exists between total MyLab assignment scores (simulation and video exercises) and average quiz scores for the online section, where r=.72 and p<.05 (figure 2).

For Stone’s online section, the correlation of average MyLab quiz score to average exam score was moderately strong, where r=.50, which may be a reflection of the difference in question type and difficulty between quizzes, which are comprised of more definition-based questions, and exams, which are mainly applied questions and likely more challenging. Stone empirically states, however, that students who spend more time working on the MyLab simulations are probably more prepared for exams given the applied nature of both assessments.

Correlation between simulation score and exam score

Figure 1. Correlation between Average MyLab Simulation Score and Average Exam Score, Fall 2016 (n=31)

Correlation between simulation and video score and quiz score

Figure 2. Correlation between Average MyLab Simulation and Video Assignment Score and Average MyLab Quiz Score, Fall 2016 (n=28)

Students were divided into two groups based on the average MyLab simulation score. Students who scored higher than the average simulation score earned higher average exam scores in both sections than students who scored below average on MyLab simulations.

  • Average MyLab simulation score (face-to-face): 84 percent
  • Average MyLab simulation score (online): 82 percent
  • Students earning MyLab simulation scores above average earned average exam scores ten percentage points higher than students scoring below average on MyLab simulations for both sections.
  • 47 percent of total students earned a MyLab simulation score higher than average (n=27)

Relationship between exam scores and simulation scores

Figure 3. Relationship between Average Exam Scores and Average MyLab Simulation Scores, Fall 2016 (face-to-face n=31, online n=26)

The Student Experience

Responses from the Fall 2016 end-of-semester, voluntary survey of Stone’s students indicate that the majority of responding students recognize the value of MyLab Management.

  • 93 percent of students strongly agree or agree that their understanding of the course material increased as a result of using MyLab.
  • 89 percent of students strongly agree or agree that they would recommend MyLab to another student.
  • 78 percent of students strongly agree or agree that the use of MyLab positively impacted their quiz and exam scores.

Student survey responses to the question, “What did you like most about MyLab?” include:

  • The PIA assessments were helpful in finding out more information about myself. It was fun to compare my scores to my classmates.”
  • I liked how there were different ways of learning the material, like the Try It virtual decision making simulations, the videos and the PIAs, instead of just tests and quizzes.”
  • I liked that some of the homework questions gave me several chances to answer correctly. It is helpful to know what you were doing wrong and what the correct answers are. I also like the flash cards [Dynamic Study Modules].”
  • “I liked that assignments put the user into a situation where they had to think critically rather than just know definitions.”

Conclusion

For his online students, Stone sought the support of a digital platform that could provide a meaningful variety of assignments, which would also deliver content and connect it to learning. With MyLab, he is able to assign simulations, videos, personal inventory assessments, quizzes, adaptive learning assignments and other exercises that help students master the content and apply it to the real-world. His face-to-face lecture benefits from the varied content and assignments, as well. Because the MyLab can deliver and assess understanding of content, Stone can spend class time on case analysis, article discussion and other group and experiential activities.

Stone concurs with a Pearson best practice in cautioning new users to take it slowly; because of the many options in MyLab, there is a tendency to try to implement it all and take advantage of every exercise at the start. But he warns that can be overwhelming, for both instructor and student, and suggests that the better option is to start with a specific strategy, trying some features out and layering in new types of assignments as you get more familiar with the program. The variety of assignments is available to cater to each instructor’s course needs and desired learning outcomes, but the quantity might be too extreme if all assignments were used for every chapter, particularly at the outset.