Revel educator study examines impact on student performance and writing assignments at College of the Sequoias

EDUCATOR STUDY

Revel educator study examines impact on student performance and writing assignments at College of the Sequoias

Key Findings

  • Students earning higher Revel scores also earned significantly higher average participation, average exam, and final course scores.
  • Data indicate a strong correlation between the overall percentage of Revel points earned and average exam scores.
  • 76 percent of student respondents indicated they often or very often used what they learned from Revel assignments to complete a writing assignment.

School name
College of the Sequoias, Visalia, CA

Course name
Introduction to Sociology

Course format
Face to face

Course materials
Revel for Essentials of Sociology: A Down-to-Earth Approach by Henslin

Timeframe
Spring 2017

Educator
Catherine Medrano, Associate Professor

Results reported by
Stephanie Fritson, Pearson Customer Outcomes Analytics Manager

Setting

  • Locale: public, two-year community college located in central California
  • Enrollment: approximately 15,800 students (2016–2017 academic year)
  • Full-time students: 41 percent
  • Age 24 and under: 68 percent
  • Gender: 56 percent female
  • Minority: 65 percent Hispanic, 77 percent total minority
  • First-year retention rate: 68 percent
  • Six-year completion rate: 46 percent

About the Course

Sociology 1 is a three-credit hour, introductory course covering basic concepts, principles, methods, and theoretical perspectives of sociology. Topics include culture, personality, social inequality, diversity, institutions, population and ecology, and social change. Course objectives include the ability to apply sociological principles to everyday life.

Challenges and Goals

Instructor Medrano began using Revel™ in her Introduction to Sociology course at College of the Sequoias in Spring 2015 as an adjunct instructor. At that time, she was teaching Introduction to Sociology courses at both Fresno City College and College of the Sequoias. She found her students were struggling with the course content, and she was curving exam grades in order for students to pass the course. She implemented Revel as a way to require students to read and reflect on their assigned reading, with the goal of increasing exam scores and no longer curving exams. Results of her initial implementation are detailed in this 2015 Revel educator study.

Medrano subsequently accepted an Associate Professor position at College of the Sequoias where she continues to teach Sociology 1. She has incorporated three essay assignments into the course to promote the analysis and application of sociological concepts and has found that students struggle with writing. It is also a challenge for her to grade essays and provide feedback for the 250+ students enrolled in her classes each term. For the Spring 2017 term, Medrano elected to replace one instructor-graded essay with a Revel auto-graded writing assignment. By assigning the auto-graded essay, she hoped to reduce the amount of time she spent reviewing essay assignments and provide students with meaningful feedback in a timely manner to assist in improving their writing skills. She also hoped the Revel quizzes would prepare students for the more summative exams and encourage in-class participation.

Implementation

Sociology 1 met twice a week for a total of two hours and thirty minutes, covering 15 chapters over the course of the Spring semester. Grades for the course were comprised of exams and accompanying study guides, essays, Revel quizzes, and class participation, with a total of 1000 possible points. Final grades were assigned as follows:

A  895–1000 points
B  795–894 points
C  695–794 points
D  595–694 points
F  0–594 points

There were a total of six exams in the course. Exams 1–4 were mandatory and each included a corresponding study guide developed by Medrano. The study guides consisted of a list of keywords for students to define, understand the significance of, and apply. Study guides were required to be handwritten and turned in prior to the exam. Late study guides and typed study guides were not accepted. Each exam with its corresponding study guide was worth 100 points. The exams consisted of multiple-choice questions and/or written responses. Exam 5 was given for extra credit and was worth half the points of a regular exam. However, the study guide for Exam 5 was mandatory. Exam 6 (final exam) was cumulative with no corresponding study guide. If students missed an exam, they could count the final exam score for two exams. For example, if a student missed Exam 2 and scored an 80 percent on the final exam, they received a score of 80 percent on Exam 2 and the final exam. After all submitted work (including extra credit) for the semester was graded, students with an A in the course did not have to take the final exam. The total possible number of points for the exams and study guides was 500.

Students were assigned three essays that were worth 60 points each for a total of 180 points. The essays required students to identify, define, apply, and evaluate key concepts presented in the texts and/or lectures. Essay topics, paper guidelines, and submission instructions were posted on Canvas, the school’s Learning Management System. All essay assignments were submitted online. The first two essays were submitted through Canvas and graded by the instructor. The final essay was an auto-graded essay in Revel.

Revel quizzes were due the Friday of the week assigned at 11:59 p.m. Students were given three attempts to answer each Revel quiz question correctly. Each time an incorrect answer was selected on a Revel quiz, one point was deducted from the score. Students were allowed to read and take quizzes after the due date/time, but did not receive any points for late submissions. Students’ final percentage on Revel was converted to 180 points. (e.g., 90 percent on Revel = 162/180 points).

A total of 160 participation points were assigned. Participation points included two formal group PowerPoint presentations. The 15-minute chapter presentation was worth 25 points and required students to present in groups of three to five students on a section from chapter 3, 5, or 10 of the text. The second presentation, worth 50 points, required students to present for 15 minutes in groups of five on a social problem that they would would like to change for the better—including proposed solutions. The presentations required five sources, three of which were peer reviewed.

The remainder of the participation points consisted of written responses to questions while working in groups during class, and homework that required students to watch a documentary and answer questions.

Assessments

  • 50% Exams (6) and accompanying study guides (5)
  • 18% Essays (3)
  • 18% Revel quizzes
  • 14% Class participation

Results and Data

In implementing Revel, Medrano aimed to assign the Revel readings and quizzes to help prepare students for the essay assignments and exams while encouraging course participation.

The median Revel quiz score was 79 percent. Figure 1 illustrates the relationship between Revel quiz score and participation scores, average exam scores, and final course grades.

  • A t-test, which measures whether the means of two groups are statistically different, was used to compare participation scores of students who scored above and below the median Revel quiz score. Results indicate that students who scored above the median Revel quiz score earned significantly higher participation scores (26 percentage points) than students who scored below the median, where t(99) = 7.67 and p<0.05, indicating that this difference was statistically significant.
  • A t-test was also used to compare average exam scores of students who scored above and below the median Revel quiz score. Results indicate that students who scored above the median Revel quiz score earned significantly higher average exam scores (22 percentage points) than students who scored below the median, where t(93) = 6.23 and p<0.05, indicating that this difference was statistically significant.
  • A final t-test was used to compare final course grades of students who scored above and below the median Revel quiz score. Results indicate that students who scored above the median Revel quiz score earned significantly higher final course grades (31 percentage points) than students who scored below the median, where t(89) = 9.89 and p<0.05, indicating that this difference was statistically significant.

Difference in participation scores, average exam scores, and final grades between students who scored above and below the median Revel quiz score

Figure 1. Difference in Participation Scores, Average Exam Scores, and Final Grades between Students who Scored Above and Below the Median Revel Quiz Score, Spring 2017 (n=251)

Medrano also aimed to use the formative Revel module and chapter quizzes to provide frequent check-in points to help ensure students understood the reading. These quizzes led up to more in-depth, summative assessments (exams).

  • The correlation between the overall percentage of Revel points earned and average exam score was strong (r(251)=.69, p<.05 [figure 2]).

Correlation between total percentage of Revel points earned and average exam score

Figure 2. Correlation between Total Percentage of  Revel Points Earned and Average Exam Score, Spring 2017 (n=251)

Medrano assigned a Revel auto-graded essay as the final essay in order to reduce time spent grading essay assignments and provide timely feedback to students. Results indicate the average score on the Revel auto-graded essay did not differ significantly from the average score on the other two instructor-graded essays, giving Medrano confidence in the technology (figure 3). Essay scores of 0 were not included in the averages for any essays.

Average essay scores

Figure 3. Average Score on Essays* (n=161) *Scores of 0 not included in analysis

The Student Experience

In Spring 2017, students completed a voluntary, end-of-semester survey (69 percent response rate), providing valuable insight on their experience using Revel.

  • Respondents rated their experience with the auto-graded essay in Revel an average of rating of 4.2 with a rating of 1 being poor and a rating of 6 being excellent.
  • 78 percent of respondents rated themselves as confident or extremely confident in their ability to participate in discussions during class after using Revel.
  • 73 percent of respondents rated Revel as good or extremely good value for the money spent.
  • 34 percent of respondents reported they rarely or never read a section of the book when there was not a required Revel quiz for that section.

Student survey comments regarding their Revel experience included:

  • “I think reading the material online in sections, along with the quizzes in between made my experience easier. I didn’t feel overwhelmed, as suppose if we had to read a couple chapters all at once then take multiple quizzes after.”
  • “I think Revel is better than a textbook. I actually read it more and understood the content better.”
  • “Revel is awesome. It’s cheaper and the audio option is super helpful.”
  • “Revel has shown me ways I can improve my writing skills and allows me to think more critically.”
  • “Revel has taught me there are better ways to interact and become engaged in the course.”

Conclusion

Medrano initially implemented Revel in her Sociology 1 course with the goal of increasing participation in class discussion and improving student exam scores. After several semesters of Revel use, she incorporated a Revel auto-graded writing assignment to reduce the amount of time she spent reviewing essay assignments and provide students with meaningful feedback in a timely manner. She hoped requiring Revel would prepare students for summative assessments and improve mastery of the course objectives, ultimately leading to improved critical thinking skills. Students who scored above the median score on Revel quizzes scored an average of 26 percentage points higher on participation scores, 22 percentage points higher on exams, and 31 percentage points higher on final course grades than students who scored below the median. There was also strong correlation between the percentage of total points earned in Revel and average exam score. There was not a significant difference in average scores between the instructor-graded essays and the Revel auto-graded essay. In the future, Medrano would like to assign the auto-graded essay prior to the instructor-graded essays to see if the feedback provided on the auto-graded essay will improve student writing and subsequent essay scores. She will continue to utilize Revel in future terms and make adjustments as necessary based on student results.