Revel educator study looks at exam scores and student performance at College of Western Idaho

EDUCATOR STUDY

Revel educator study looks at exam scores and student performance at College of Western Idaho

Key Findings

  • Data indicate a strong correlation between the overall percent of Revel points earned and average exam scores.
  • Students who scored above the median on Revel quizzes scored significantly higher on exams than students who scored below the median on Revel quizzes.
  • Prior to using Revel, 26 percent of student survey respondents reported that in the past, they often attended class without having read the textbook. With Revel in use, only 8 percent reported they often attended class without having completed Revel reading.
  • 89 percent of student survey respondents rated their overall experience with Revel as excellent or good.

School name
College of Western Idaho, Nampa, ID

Course name
Introduction to Humanities I

Course format
Online

Course materials
Revel for The Humanities: Culture, Continuity and Change by Sayre

Timeframe
Fall 2015

Educator
Deborah Bonde, Instructor

Results reported by
Stephanie Fritson, Pearson Customer Outcomes Analytics Manager

Setting

  • Locale: a comprehensive community college providing higher education programs to residents of Western Idaho in southwest Idaho’s Treasure Valley
  • Enrollment: approximately 8,400 credit students and 10,000 non-credit students
  • Part-time students: 65 percent
  • In-state students: 92 percent
  • Student–faculty ratio: 24:1
  • Average class size: 17 students
  • Students under age 25: 64 percent
  • Gender: 57 percent female
  • Ethnicity/race: 39 percent minority

About the Course

Introduction to Humanities I (HUMA 101) at College of Western Idaho is an inquiry into the disciplines of the humanities with emphasis on artistic achievements from the beginning of civilization to the Renaissance. Arranged thematically rather than chronologically, the class takes a broad look at the humanities through the arts. The three-credit hour course meets Idaho State Board General Education Matriculation (GEM) competency requirements for GEM 5, Artistic and Humanistic Ways of Knowing. The course is a 16-week online and asynchronous class with all work completed online and no face-to-face meetings on campus.

During the Fall of 2015, 21 students were enrolled in HUMA 101, and all voluntarily completed a course pre-survey. Of those students:

  • 68 percent of student respondents indicated that they attended school full-time in Fall 2015.
  • 44 percent of respondents indicated they were first-generation college students.
  • 96 percent of respondents were able to log in to Revel the first week of class.
  • 26 percent of respondents reported that in the past they often or very often attended class without having read the textbook.

Challenges and Goals

Instructor Bonde’s primary goal of implementing Revel™ was to help students develop their critical thinking skills. Bonde was looking to increase technology use to further engage her students in the course and ensure students read the assigned material. She aimed to prepare students for the course exams through the Revel quizzes. Bonde also hoped that by increasing student engagement with the reading material, the quality of course discussion posts would improve (as a measure of critical thinking).

Implementation

Bonde first implemented Revel in her Fall 2015 HUMA 101 online course. Revel was implemented across several arts courses at College of Western Idaho at this time as a means for using technology to engage students and improve outcomes. The results of another course with Revel implementation, Art History, taught by Dr. Karen Brown, can be read here. Revel registration instructions were included on the course syllabus to highlight the importance of purchasing Revel access. To ensure students accessed the materials in a timely manner, a warning was included in the syllabus stating that students who had not logged into the Revel course, completed Revel quizzes, or made discussion posts during the first three weeks of the semester would be automatically dropped from the course. All of Bonde’s students were successfully registered for Revel by the second week of class.

In order to help students stay current with the readings and retain the most important information from each chapter, weekly Revel module and chapter quizzes were assigned. Students’ total Revel quiz scores were worth 20 percent of the final course grade.  

Course participation through discussion posts comprised 30 percent of the final course grade. Bonde emphasized the importance of course participation, as the humanities are about understanding what it is to be human and communicating ideas about the human condition through the ages. Even though the course was online, a key component of the course included conversation and discussion about the ideas of great minds from other times and other places.

An exam was administered at the end of each unit of study (Ancient, Medieval, and Renaissance), for a total of three exams. In addition, a final exam at the end of the semester was required. The exams were traditional tests with a mix of matching, multiple-choice, short-answer, and essay questions and comprised 30 percent of the final course grade.

Students were required to complete an event review in which they attended a live performance (play, symphony, ballet, or opera) and wrote a review incorporating what they learned in the class, applying that knowledge to what they experienced at the performance for 10 percent of the final course grade. Students also completed a class presentation where they selected a topic of interest and conducted outside research on a work of art, architecture, literature, music, or philosophy and then wrote a paper for 10 percent of the final course grade.

Assessments

  • 30%     Discussion board
  • 30%     Unit exams (3) and final exam
  • 20%     Weekly Revel quizzes
  • 10%     Event review
  • 10%     Class presentation

Results and Data

Bonde aimed to utilize Revel quizzes to improve student preparedness and performance on the unit exams. The correlation between the percent of total Revel quiz points earned and average unit exam score was strong, where r(21)=.70, p<.001. See figure 1.

Percentage of overall Revel quiz points earned correlated with average exam score

Figure 1. Percent of Revel Quiz Points Earned Correlated with Average Exam Score,Fall 2015 (n=21)

A t-test, which measures whether the means of two groups are statistically different was used to compare average unit exam grades for students who scored above the median Revel quiz score with students who scored below the median Revel quiz score. Results indicate that students who scored above the median Revel score earned significantly higher unit exam averages than students who scored below the median Revel score where t(21) = 2.24 and p<0.05, indicating that this difference was statistically significant (figure 2).

Difference in average exam scores between students who scored above and below the median Revel quiz score

Figure 2. Difference in Average Exam Scores between Students who Scored Above the Median Revel Quiz Score and Students who Scored Below the Median Revel Quiz Score, Fall 2015 (n=21)

In addition, Revel was implemented in the course to ensure students were reading the assigned material prior to course discussion posts. Fall 2015 students were asked to complete pre- and post-course surveys to share their feedback on how Revel impacted their learning (100 percent response rate on both surveys). On the pre-course survey, students were asked, “How often have you attended class without having read the textbook?” to which 26 percent answered “often” or “very often.” Following the use of Revel in the course, only 8 percent of students reported attending class (or responding to course discussion posts) often or very often without having read the Revel assignment (figure 3).

Student pre and post course survey responses for attending class without reading

Figure 3. Student Pre- and Post-course Survey Responses for Attending Class without Reading, Fall 2015. Pre-course Survey (n=21); Post-course Survey (n=21)

The Student Experience

In Fall 2015, students completed a voluntary, end-of-semester survey (100 percent response rate) providing valuable insight on their experience using Revel.

  • 96 percent of students were able to log in to Revel the first week of class.
  • 89 percent of students rated their experience with Revel as excellent or good.
  • 89 percent of students rated themselves as very confident or confident in their ability to think critically or analytically after using Revel in the course, compared with 73 percent of students on the pre-course survey.

Student survey comments regarding the use of Revel include:

  • “It has keep me engaged in the material being covered.”
  • “My learning would not be the same without it. If my other classes had it, I’m sure my other grades would be as good as this class.”
  • “The quizzes were really good practice for the test questions.”
  • “I like to listen to the audio and follow along with the print version. Also, it is easier for me to reference something based on feel/general location than trying to remember a specific page number.”
  • “Again, the interactions within the textbook are very beneficial. I liked viewing Plazas with the 360 viewing. The comparisons between the artwork and a 6 foot man really put the art into perspective. I liked the extra links with each unit (like Versailles). Also, I cannot stress enough how important and valuable the audio option was to me. I really hope this option will continue and grow. There have been studies that the more senses used the greater retention there is for a subject. I would say this is quite true. Plus it makes the textbooks less of a dry read.”

Conclusion

Bonde implemented Revel in her HUMA 101 course with the goal of engaging students in the course and holding them accountable for reading the assigned material. She hoped requiring Revel would prepare students for exams and improve mastery of the course objectives, ultimately leading to improved critical thinking skills. A strong correlation was found between the percent of total Revel quiz points earned and average unit exam score, indicating students who performed better on Revel quizzes also had higher scores on the course exams.  Pre- and post-course student survey responses indicated fewer students attended class or responded to course discussion posts without having read the material when Revel was used, and 89 percent of students rated themselves as confident or very confident in their ability to think critically and analytically after using Revel in the course as compared to 73 percent of respondents on the pre-course survey. Bonde also reported improved discussion posts with Revel implementation. She finds these results encouraging and plans to look for additional ways to use Revel (along with other instructional techniques) to continue to improve students’ critical thinking skills and results.