Revel educator study examines impact on student exam performance and writing assignments at University of Nebraska Lincoln

EDUCATOR STUDY

Revel educator study examines impact on student performance and writing assignments at University of Nebraska Lincoln

Key Findings

  • Students who scored above the median on Revel quizzes scored significantly higher on exams than students who scored below the median on Revel quizzes.
  • Students who earned a higher percentage of total Revel points also earned higher average course grades.
  • Students who earned higher overall Revel scores also earned higher Media Analysis Paper scores.
  • Correlations between percentage of total Revel points earned and exam performance, Media Analysis Paper, and final course scores were stronger than correlations between percentage of total recitation points earned and those same three metrics.
  • 91% of student survey respondents reported that Revel helped them to be better prepared for class.
  • 65% of student survey respondents reported that completing work in Revel made class discussions more interesting.

School name
University of Nebraska, Lincoln, NE

Course name
Introduction to Sociology

Course format
Face to face

Course materials
Revel for Sociology: A Down to Earth Approach by Henslin

Timeframe
Fall 2016

Educator
Larry Gibbs, Post Doc Research Associate Professor

Results reported by
Stephanie Fritson, Pearson Customer Outcomes Analytics Manager

Setting

The University of Nebraska-Lincoln (UNL) is a land-grant university that serves as a public research institution in the Big 10 Conference. UNL is Nebraska’s oldest university and the largest in the University of Nebraska system. Nearly 26,000 students were enrolled at UNL during the 2015-2016 academic year, with 20,833 of those enrolled in undergraduate programs. Eighty-five percent of students attended UNL full-time and 48 percent of those students were female. Students who identified themselves as Caucasian comprised 74 percent of the student population. Sixty-five percent of students enrolled at UNL were residents of the state of Nebraska and the average age of undergraduate students was 20.4 years. The six-year retention rate for the fall of 2010 freshman class was 82 percent with 67 percent of those students graduating within six years of enrolling at UNL.

About the Course

Larry Gibbs is a Post-Doctoral Research Associate Professor at UNL, coordinating the Introduction to Sociology course (SOCI 101). SOCI 101 is a three-credit hour course that investigates society and human interactions. Drawing on C. Wright Mill’s “sociological imagination”, students identify the connections between personal experiences and social environment. The course emphasizes three broad theoretical frameworks (Functionalism, Symbolic Interactionism, and Conflict Theory), data interpretation, and social science concepts and terminology. Students are encouraged to think critically and sociologically on a vast array of societal issues concerning crime, families, health, race/ethnicity, gender and education. Course assignments are delivered through Blackboard and Revel™ is integrated with Blackboard for seamless access to course materials.

Challenges and Goals

Gibbs elected to move to Revel from another publisher’s textbook because he was seeking a product that offered eBook access with online integration so students could complete assignments directly within the reading. He also liked Revel’s seamless integration with Blackboard and the option for students to purchase a loose leaf print text. Gibbs aimed to increase student preparedness for class and improve in-class discussion through implementing Revel, along with improving critical thinking skills (as measured during class discussion and on the Media Analysis Paper) and increasing grades on multiple choice unit exams and writing assignments.

Implementation

Professor Gibb’s fall 2016 SOCI 101 course enrolled nearly 250 students in an interactive lecture/recitation format with lectures on Mondays and Wednesdays, and a recitation class on Fridays.  The purpose of the recitation was to reinforce the topics discussed in lectures through small group discussions and topic-based activities. Seven instructors (Graduate Teaching Assistants) led recitation sections and students were assigned to a section by default upon registering for the course. There were a total of 14 recitation session activities/assignments worth ten points each for a total of 14 percent of the final course grade.

Along with Revel, Pearson’s Learning Catalytics was used during the interactive lecture sessions. Learning Catalytics is student response tool that encourages participation based learning by using students’ smartphones, tablets, or laptops to engage them in interactive tasks and thinking. Students in SOCI 101 responded to questions during class through Learning Catalytics. Students were required to bring electronic devices to class for Learning Catalytics activities and could use them to reference Revel content and take notes. Students were also provided the option of purchasing a loose leaf print copy of the text in addition to the required Revel access.

To ensure students completed the reading assignments and arrived at class prepared, Revel chapter quizzes comprised 19.5 percent of the overall course grade and were due the night before the topic was covered in class. Students were also assigned six Revel shared essays that were graded by their recitation instructors and two Revel auto-graded essays.

A final four-to-six page Media Analysis Paper (MAP) was required for 15 percent of the total course grade. The MAP required students to apply journal articles and chapter content to a film (movie), radio broadcast, or video presentation. The paper assessed critical thinking and students’ ability to comprehensively answer a question about the topic, including application of sociological theory and personal reflection about how the media depicts the concepts and perspectives of sociology.

In addition to the recitation activities, Revel assignments, and MAP, students also completed four multiple choice unit exams. The exams were administered online in a campus testing center. Students were given a seven-day window to take each exam and had the option of retaking each exam. Student were allowed four attempts on exam 1, three attempts on exam 2, two attempts on exam 3, and 2 attempts on exam four. The highest score for each exam was recorded as the final score. Students also were required to attend one office visit with the course instructor or recitation instructor for 15 points or 1.5 percent of the course grade.

Assessments

  • 40%     4 unit exams
  • 19.5% Revel chapter quizzes
  • 15%     Media Analysis Paper (MAP)
  • 14%     Recitation activities
  • 6%       Revel brief shared essays
  • 4%       Revel auto-graded essays
  • 1.5%    Office visit

Results and Data

Gibbs aimed to utilize Revel quizzes to improve student preparedness and performance on the multiple choice unit exams. The correlation between the percentage of total Revel quiz points earned and average unit exam score was very strong (r(246)=.75, p<.001 [figure 1]).

Correlation between percentage of total Revel points earned and average exam score

Figure 1. Correlation between Percentage of Total Revel Points Earned and Average Exam Score, Fall 2016 (n=246)

A t-test, which measures whether the means of two groups are statistically different was used to compare students who scored above the median Revel score with students who scored below the median Revel score and average unit exam grades. Results indicate that students who scored above the median Revel score earned significantly higher unit exam averages than students who scored below the median Revel score where t(246) = 8.61 and p<0.01, indicating that this difference was statistically significant (figure 2).

Average exam scores of students scoring above and below the median Revel score

Figure 2. Difference in Average Exam Scores between Students who Scored Above the Median Revel Score and Students who Scored Below the Median Revel Score, Fall 2016 (n=246)

In addition, a very strong correlation was found between the percentage of total Revel points earned and final course grade (r(246)=.91, p<.001 [figure 3]). It should be noted that Revel assignments made up 19.5 percent of the final course grade, thereby influencing this relationship.

Correlation between percentage of total Revel points earned and course grade

Figure 3. Correlation between Percentage of Total Revel Points Earned and Course Grade, Fall 2016 (n=246)

Gibbs implemented Revel with the goal of improving students’ critical thinking and writing skills as measured by the MAP assignment. The correlation between the percentage of total Revel quiz points earned and MAP score was strong (r(246)=.70, p<.001 [figure 1]), indicating students who performed better on Revel essay assignments also earned higher scores on the MAP assignment.

Percentage of total Revel points earned correlated with MAP score

 

Figure 4. Correlation between Percentage of Total Revel Points Earned and MAP Score, Fall 2016 (n=224)

Table 1 reports on correlations between the percentage of total Revel points earned and the percentage of total recitation points earned and student performance measures in the course. Correlations do not imply causation but instead measure the strength of a relationship between two variables, where r is the correlation coefficient. The closer the r-value is to 1.0 or -1.0, the stronger the correlation. The corresponding p-value measures the statistical significance/strength of this evidence (the correlation). When a p-value is less than .05, the correlation is deemed significant. All correlations reported in Table 1 had p-values less than .001.  Stronger correlations were found between the percentage of total Revel points earned and each performance measure than between the percentage of total recitation points earned and each performance measure.

Table 1. Correlations between Percentage of Total Revel Points Earned and Percentage of Total Recitation Points Earned to MAP Score, Average Exam Score, and Overall Course Score (n=246)

The Student Experience

In Fall 2016, 53 percent of students enrolled in SOCI 101 completed a voluntary, end-of-semester survey providing valuable insight on their experience using Revel in the course.

  • 97 percent of student respondents reported they were able to log in to Revel the first week of class (only one student reported not being able to log in because of technical issues with Revel and/or Blackboard).
  • 91 percent of student respondents reported that Revel helped them to be better prepared for class.
  • 88 percent of student respondents strongly agreed or agreed that Revel kept them more engaged in the content of the course.
  • 86 percent of student respondents rated their overall experience with Revel as good or excellent.
  • 86 percent of student respondents strongly agreed or agreed that Revel gave them greater access to a variety of learning tools.
  • 81 percent of student respondents rated their experience with Revel as easy or extremely easy to use.
  • 77 percent of student respondents strongly agreed or agreed they would recommend their instructor use Revel for other courses.
  • 65 percent of student respondents strongly agreed or agreed that Revel made class discussions more interesting.

Student survey comments regarding their Revel experience:

  • “It kept me prepared for class and helped me learn the material.”
  • “It forced me to read the entire chapter.”
  • “I stayed more focused with an online book and I liked being quizzed throughout the chapters.”
  • “It gives you videos to better understand the content.”
  • “It required us to read so we would do well on the quizzes.”
  • “It helped me better understand the topic we were learning about and I could often use that information to help me study for upcoming exams.”
  • “You can access it on any mobile device and complete your homework even if you do not have your textbook.”
  • “Revel gave lots of examples for things you needed to know to help you better understand the material.”

Conclusion

Gibbs chose to use Revel in SOCI 101 with the goal of increasing student preparedness for class and improving class discussion. He wanted to provide students with a streamlined technology experience and to utilize Revel quizzing to better prepare students for assessments. Gibbs also hoped to improve students’ critical thinking skills, measured in the MAP assignment and class discussions. Student survey results indicate 91 percent of respondents strongly agreed or agreed that Revel made them more prepared for class and 65 percent strongly agreed or agreed that Revel made class discussions more interesting, while 97 percent of respondents strongly agreed or agreed they were able to log in to Revel the first week of class and Revel was easy to use. In terms of Revel as an exam preparation tool, a strong correlation between average Revel quiz scores and average unit exam scores was found. Students who scored above the median Revel score performed significantly better on unit exams than students who scored below the median Revel score. A strong correlation between Revel essay scores and the MAP assignment were also found. The findings of this study corroborate Gibbs’ observations and he plans to continue using Revel integrated with Canvas as UNL moves to the new learning management system.