REVEL™ educator study examines student performance and exam scores at Harper College
- When Rimland utilized the REVEL Performance Dashboard to monitor student performance and inform instruction, average student exam and final research paper scores were the highest recorded since data collection began in Fall 2012.
- The correlation between student performance on REVEL assignments and student performance in the course was very strong.
- Students who earned more points in REVEL also had higher average exam scores.
Harper College, Palatine, IL
Face to face
REVEL; Art History by Stokstad and Cothren
Stephany E. Rimland, Associate Professor
Results reported by
Stephanie Fritson, Pearson Customer Outcomes Analytics Manager
- Locale: One of the nation’s largest community colleges, serving students in the suburbs of northwest Chicago
- Enrollment: approximately 40,000
- Full-time students: 64 percent
- In-state students: 99 percent
- Graduation rate: 24 percent
- Gender: 54 percent female
- Ethnicity/race: 46 percent minority
About the Course
Professor Stephany Rimland has been teaching Art 131, Gothic through Romantic Art, for over 10 years, while also serving as the Art Department Chair. Art 131 covers the history of art from the medieval age to the end of the eighteenth century. Focusing on major artistic styles and works of art and monuments, the course follows the historical development of the visual arts produced by western civilizations. A wide range of students take this course, including both traditional and nontraditional-aged students, art majors and nonmajors.
Challenges and Goals
Prior to implementing REVEL, Rimland was using Pearson’s MyArtsLab which included a number of interactive resources. Although she saw improvements in exam and research paper scores when using MyArtsLab in addition to the print textbook, she believed students were still not fully reading and understanding the assigned material. Rimland implemented REVEL in Fall 2015 with the goal of further improving outcomes on exams and the final research paper by ensuring students read the assigned material and complete sufficient writing practice. Results of the the Fall 2015 REVEL implementation indicated students who performed better on REVEL assignments also performed better on exams, and students who completed more practice essays performed better on the final research paper (Harper Fall 2015 Educator Study). However, average exam scores and research paper scores were slightly lower than when MyArtsLab was in use (figures 1 & 2 in Results). In order to continue to improve student reading (as measured by course preparedness and exam performance) and writing (as measured by performance on the required paper) during Spring 2016, Rimland made further adjustments to her REVEL implementation.
In Spring 2016, Rimland increased her use of the REVEL Performance Dashboard, implementing a global perspective of performance monitoring to drive instruction by identifying areas where students were struggling on REVEL quizzes and shaping instruction accordingly. She focused class lecture and discussion content on areas where students had difficulty, identified through the REVEL Performance Dashboard. Rimland also reviewed the dashboard several times a week to identify students who were not completing the readings or performing poorly on REVEL assignments and were determined at risk. She reached out to these at-risk students and believes this more timely intervention helped keep them on track. In addition, Rimland reduced the number of REVEL Writing Space assignments to three auto-graded assignments which allowed students to focus more intensely on writing quality while preparing for the final paper.
To ensure students had read the material before class, all REVEL reading and quiz assignments were due the day the class met, before students came to class. On REVEL quizzes, students were allowed two attempts per question in order to eliminate the possibility of guessing until the correct answer was reached. REVEL auto-graded essay assignments were due after the topic was covered in class and Rimland provided targeted feedback to students on their essays.
REVEL assignments accounted for 20 percent of the overall course grade in both Fall 2015 and Spring 2016. Students’ REVEL grade was determined by their overall REVEL score at the end of the semester. Students who averaged between 90 and 100 percent in REVEL earn the full 100 points, and students who averaged between 80 and 90 percent in REVEL earn 90 out of 100 REVEL points.
- 60% Exams (300 points- 3 exams)
- 20% REVEL quizzes and essays (100 points)
- 20% Final research paper (100 points)
Results and Data
After utilizing the REVEL Performance Dashboard to monitor performance and drive instruction in Spring 2016, average student exam scores were the highest recorded since Rimland began collecting data in Fall 2012 (figure 1). Average exam scores in Spring 2016 were seven percentage points higher than Fall 2015. A t-test, which measures whether the means of two groups are statistically different (average exam scores in Spring 2016 and Fall 2015), was used to compare the average exam scores. Results of the t-test show that students in Spring 2016 (mean = 91 percent) earned higher average exam scores than students in Fall 2015 (mean = 83 percent), where t(49) = -2.29 and p<0.05, indicating that this increase was statistically significant (table 1).
Upon reducing the number of writing assignments to only include the auto-graded essays in REVEL with an increased focus on preparation for the final paper, final research papers scores were also the highest recorded since Fall 2012 (figure 2). Average final research paper scores were three percentage points higher in Spring 2016 than Fall 2015. Rimland believes the reduced writing assignments and increased focus on final paper preparation helped contribute to these improved scores.
Figures 3 and 4 are correlation graphs; correlations do not imply causation but instead measure the strength of a relationship between two variables, where r is the correlation coefficient. The closer the r value is to 1.0, the stronger the correlation. The corresponding p-value measures the statistical significance of this evidence (the correlation), where a p-value shows the existence of a positive correlation between these two variables.
- A strong positive correlation exists between overall REVEL scores and average exam scores, where r=.73 and p<.01.
- A very strong positive correlation exists between overall REVEL scores and overall final course scores, where r=.81 and p<.01 (REVEL counts towards 20 percent of the overall course score, influencing this relationship).
Average exam score
Figure 1. Average Exam Scores, Fall 2012 (n=22), Spring 2013 (n=38), Fall 2013 (n=27), Spring 2014 (n=25), Fall 2014 (n=24), Spring 2015 (n=30), Fall 2015 (n=29), Spring 2016 (n=25)
Average final research paper score
Figure 2. Average Final Research Paper Scores, Fall 2012 (n=22), Spring 2013 (n=38), Fall 2013 (n=27), Spring 2014 (n=25), Fall 2014 (n=24), Spring 2015 (n=30), Fall 2015 (n=29), Spring 2016 (n=25)
Overall REVEL score correlated with average exam score
Figure 3. Correlation between Overall REVEL Score and Average Exam Score, Spring 2016 (n=25)
Overall REVEL score correlated with overall course score
Figure 4. Correlation between Overall REVEL Score and Final Overall Course Score, Spring 2016 (n=25)
Two-sample t-Test assuming equal variances
Table 1. Two-Sample t-Test Assuming Unequal Variances, Fall 2015 Exam Average (n=27) and Spring 2016 Exam Average (n=24)
The Student Experience
Four students chose to provide feedback on their experience using REVEL during the Spring 2016 semester (16 percent survey response rate). Student survey comments regarding the use of REVEL include:
What was your favorite thing about using REVEL?
- “The quizzes to test retention of information.”
Did you prefer using REVEL to a traditional print textbook?
- “Yes, because it cost significantly less for access to more information.”
- “I liked using the E-Book over the physical copy because it’s more accessible reading media. I can pull it up on my laptop that I use for school. ”
When Rimland utilized the REVEL Performance Dashboard to monitor student performance and inform instruction, average student exam and final research paper scores were the highest recorded since data collection began in Fall 2012. Also, data indicate students who earned more overall points in REVEL earned higher average exam and overall course scores. Rimland noted that students are arriving to class more prepared for discussion and with a better understanding of the material since implementing REVEL and utilizing the Performance Dashboard for global performance monitoring.