Learning Catalytics educator study reports on a flipped classroom redesign at North Idaho College


Learning Catalytics educator study reports on a flipped classroom redesign at North Idaho College

Key Findings

  • Redesigning the course to a flipped format with Learning Catalytics increased student engagement in class and provided just-in-time student feedback to identify gaps in understanding.
  • There was a strong positive correlation (p=.82) between Learning Catalytics scores and exam averages.
  • Exam averages increased after a flipped format was implemented using Learning Catalytics.
  • The instructor identified several best practices for implementing in-class activities and Learning Catalytics to enhance efficiency and learning.

School name
North Idaho College, Coeur d’Alene, ID

Course name
Physical Geology

Course format
Face to face, flipped

Course materials
Learning Catalytics and Mastering Geology with Earth: An Introduction to Physical Geology, by Tarbuck, Lutgens and Tasa

Spring 2014Spring 2015

Bill Richards, Lecturer

Results reported by
Betsy Nixon, Pearson Customer Outcomes Analytics Manager


  • Location: Rural, population of approximately 46,000
  • Founded: 1933
  • Type: Community college (one of three in Idaho)
  • Enrollment: 6,000 students in credit classes, more than 4,400 in non-credit courses
  • Transfer policies: Students obtaining an associate degree may transfer with junior standing to all other Idaho public colleges and universities.
  • Admissions: Open-door admissions policy defined as providing all eligible students with access to appropriate educational offerings at the college.
  • Age: Approximately 31 percent older than 25

About the Course

Bill Richards has been an instructor at the college for 33 years. He teaches both Physical Geology and Physical Geography in a face-to-face format. Physical Geology is the study of the origin and development of the earth. It includes the detailed study of the development of the Earth’s crust, its minerals, rocks, volcanoes, glaciers, mountains, and continents. This course provides an understanding of the natural and physical processes of planet Earth and an appreciation for the impact geology has on everyday life. Richards typically has a small lecture of up to 70 students, but always less than 100. The course is a three-credit lecture taken by both science majors and students seeking to fulfill a general education science requirement. The course outcomes include:

  • Describe, classify and explain the origin and uses of various earth materials.
  • Identify the major geologic conditions conducive to development of the major natural resources used by modern nations.
  • Describe the structure and composition of Earth from the inside out.
  • Give examples of landforms that are evidence of various geologic forces that shape the earth’s surface and explain how those landforms were formed.

In addition, general education abilities emphasized in this course include:

  • Communication skills;
  • Critical and creative thinking and problem-solving;
  • Historical, cultural, environmental and global awareness;
  • Information literacy; and
  • Mathematical, scientific and symbolic reasoning.

Challenges and Goals

Richards has spent the majority of his teaching career utilizing a traditional lecture format. He said, “When you teach for a long period of time, your teaching either gets stale and stays stale, or you look at what you can do to change and engage students. The changes in technology and students are significant enough that we have to change.” He believes that if students are involved in the learning process, they will learn, and he emphasizes that the answer is not to just add more technology, but rather to understand how to use the technology to motivate students to enhance learning.

Since Richards is continually looking for ways to increase student success without compromising content or assessment rigor, he has researched teaching methods and became interested in the flipped classroom model. In this format, students prepare by studying course concepts prior to lecture, and in-class lecture time is then spent with students participating in some form of activity designed to measure and enhance their level of conceptual understanding. Activities vary and can be conducted with students answering by traditional methods (paper and pencil), or utilizing digital technologies students are already familiar with, such as smartphones, tablets or laptops. Richards felt that a move to more active learning and a flipped format would enhance engagement and learning. In fact, various studies have been done supporting this and presenting results on increased engagement with flipped learning. A meta-analysis of 225 studies on active learning published by Scott Freeman found “that active learning leads to increases in examination performance that would raise average grades by a half a letter, and that failure rates under traditional lecturing increase by 55% over the rates observed under active learning.”1


Because Richards believed a flipped format would facilitate his course goals, he piloted the approach at the end of the Fall 2014 semester. Richards was using Mastering Geology and added Learning Catalytics (LC) to assess students and provide data to evaluate performance on the in-class activities.  Students could use LC on any device such as their smart phones or tablet, so there was no need to purchase an additional device. A few tablets were available in the classroom for any student that did not have a web-enabled device.

Students in all semesters were required to complete the same Mastering Geology homework assignments, took exams with the same assessment criteria, and used the same edition of the textbook. In Fall 2014, LC was added to the fourth unit, but not used during the first three units of that semester. LC was then used for the full Spring 2015 semester.  Students were required to complete a reading for the upcoming class as well as participate in class activities. LC was used to answer questions during the class period.  After the course was fully redesigned to a flipped format, it consisted of the following components:

Exams: There were a total of five one-hour exams including a comprehensive final. Test material was derived from a combination of class discussions, assignments, and the textbook readings. Tests were objective and included multiple-choice, fill-in-the-blank, true-false, and/or matching questions answered using a student response device in class (not LC). A summary report was returned to the student after each test. The actual test remained in the possession of the instructor but students had access to review their exams by meeting with Richards. Students were not allowed to make up exams. For purposes of calculating a final grade, only the four highest scores of the five exams were used. If students were happy with the first four exam grades, they did not need to take the final exam.

Mastering Geology:  The Mastering Geology homework consisted of four multi-part assignments per semester intended to give students a way to study and evaluate their mastery of the material. The four homework assignments each consisted of three subparts. The first subpart contained 30 questions. The second subpart contained the same 30 questions from the first part plus an additional 30 questions. The last subpart contained the same 60 questions from the first two parts along with 30 additional questions. Richards believed the repetition in questions reinforced the concepts since students continued to review the material and test their knowledge. If they missed any questions, it provided another opportunity to learn the material.

Learning Catalytics: Learning Catalytics (LC), an automated, cloud-based, classroom response system, was utilized in each class session with students providing their own web-enabled device. LC responses were recorded automatically in the Mastering gradebook. The in-class engagement consisted primarily of questions based on previously assigned textbook readings and included approximately four to six questions for each class period. LC activity was scored based on 20 percent credit for participation and 80 percent for correctness.

During each in-class session, students worked in groups of two to three answering questions via LC designed to assess their understanding of course concepts. Richards did not have a prepared lecture for class time, but did have PowerPoints ready for topics that students often found problematic. He would stop group discussions if he found a high rate of incorrect LC responses and cover the topic with the large group. In-class activities often focused on regions and directions and included multiple choice, sketching, and short answer questions. Richards used some questions from Pearson’s content, but often created his own using old test questions and information from lecture slides.

Lecture attendance was required and participation was recorded via LC activities. Students arriving late were not permitted to participate in quizzes, LC activities, or other assignments that had been given at the beginning of class.

Paper-and-pencil homework: A mix of homework was assigned in the form of paper and pencil quizzes (both take-home and in-class), vocabulary worksheets, and/or other evaluative materials. There were 12 weekly assignments and students were not allowed to make up any missed assignments. The lowest quiz or assignment score was dropped.

Semester project: This was scheduled to follow the midterm exam. Students participated in a collaborative, role-play project with other group members. The project score was based equally on participation and peer evaluation.

After redesigning the class to a flipped format, Richards identified the following best practices that have helped enhance learning in his course:

  • On the syllabus, include a statement about the flipped model explaining what to expect during class.
  • Help students understand the daily expectations to be successful in the class.
  • Clearly list the daily topics in the syllabus.
  • Assign material in chunks and identify what they are responsible for understanding for each class.
  • Ensure students are working with different students during class activities. Richards set up a system so students drew a golf ball with a seat number for each class period as they entered the room and returned the golf ball at the end of class. He feels mixing up groups is important since students would often end up working with the same students. LC includes a seat mapping program that will group students based on their question response so instructors can use the automated system which is helpful for large enrollment classes.
  • Facilitate movement around the classroom so time is not wasted. Richard told students not to put their backpacks on the floor, but instead place them against the wall to keep aisles clear.
  • Use a tablet to control the LC questions so it’s easier to walk around the classroom during the activity periods and observe group conversations.
  • Utilize a variety of resources and question types, such as a mix of visual or text-based questions or photos.
  • Include more interactive questions, such as sketching, a capability with LC. Students can view all of the sketches submitted in class and they can be used for discussion pointing out why someone may have drawn something one way or another to highlight common errors.
  • Start the class session with an easy or repeat question from the prior class period. Richards suggests this will keep students from getting frustrated at the start of class if they don’t know something, so starting out with an easier activity or question can help get students engaged early in class.


Spring 2014

  • 50%        Exams
  • 25%        Lab
  • 10%        Weekly quizzes and assignments
  • 10%        Mastering Geology homework
  • 5%          Semester project

Fall 2014

  • 50%        Exams
  • 25%        Lab
  • 10%        Weekly quizzes and assignments
  • 10%        Mastering Geology homework
  • 2.5%      Learning Catalytics (implemented for last unit)
  • 2.5%      Semester project

Spring 2015

  • 45%        Exams
  • 25%        Lab
  • 10%        Weekly quizzes and assignments
  • 10%        Mastering Geology homework
  • 10%        Learning Catalytics

Results and Data

Richards included data from the Spring and Fall 2014 and Spring 2015 semesters in the analysis. The Fall 2015 data was not available at the time of the analysis. Students were first grouped by quartile each semester based on their overall exam average. There was an increase in exam averages for each quartile of students for Fall 2014 and Spring 2015 with LC (see figure 1).  In addition, figure 2 presents final score distributions based on exam averages. The percent of students who earned an 80 percent or higher exam average increased after the flipped format with LC was implemented. In Spring 2015, 60 percent of students had an exam average of 70 percent or higher, with 51 percent of students prior to LC earning 70 percent of higher.

Finally, although not itself a direct indicator of improved success for one semester over another, the correlation between scores from LC in-class engagement activities and the exam average shows a strong positive correlation, with r=.82 (figure 3). Correlations do not imply causation but instead measure the strength of a relationship between two variables, where r is the correlation coefficient. The closer the r value is to 1.0, the stronger the correlation.

Exam average by quartile, Spring 2014Spring 2015

Figure 1. Spring 2014 (n= 38); Fall 2014 (n= 40); Spring 2015 (n= 28)

Final score distributions by semester by exam average, Spring 2014Spring 2015

Figure 2. Spring 2014 (n= 38); Fall 2014 (n= 40); Spring 2015 (n=28)

Correlation between Learning Catalytics and exam averages, Spring 2015

Figure 3. Spring 2015 (n=28)


The Student Experience

The feedback Richards received from students after piloting Learning Catalytics in Fall 2014 for only the last unit was that they would have wanted to participate in a flipped format class with LC for the full semester. Now that Richards has been teaching a flipped class using Mastering Geology and Learning Catalytics for a few semesters, word has spread  about what students do during class time in his courses. Instructor names are listed on the schedule, and students tend to look to see who is teaching when they register. Richards has found that his class consistently fills up each semester, and he feels students register knowing that it will be an active classroom environment.


Richards stated, “Even though changing to the flipped model of instruction was a significant investment in time and a significant shift in classroom operational modality, an improved rate of student success does seem to exist.” He feels that by flipping the classroom and implementing the Learning Catalytics technology in each class, students experience a different way of managing their time to study and do homework. They can no longer wait until the last minute before the exam, but rather they are required to prepare on a regular basis before coming to class since they are being assessed regularly during in-class activities. That means that the students who keep up with the work are engaging with the course content outside of class more frequently, and Richards is able to utilize Learning Catalytics during class to assess student understanding and identify gaps in learning. This allows him to cover those concepts more fully.

Richards said that since students know what kind of class he conducts before they register, he now finds that students enrolled in his class are ready to engage, understand the work that will be required to succeed, and are prepared to actively participate and collaborate with other students.


1Freeman, Scott, Eddya, Sarah L., McDonough, Miles, Smith, Michelle K., Okoroafora, Nnadozie,  Jordta, Hannah, and Wendorotha, Mary Pat, (2015), Active learning increases student performance in science, engineering, and mathematics. Proceedings of the National Academy of Science, www.pnas.org/cgi/doi/10.1073/pnas.1319030111


Note: The results reported were first presented by Richards at the 2015 Geological Society of America Annual Meeting, Baltimore, MD.


Leave a reply

Your email address will not be published. Required fields are marked *