Mastering Physics University North Georgia

EDUCATOR STUDY

Mastering Physics educator study reports on redesign of calc-based physics course at University of North Georgia

Key Findings

  • The course was redesigned over a period of time to enhance learning and develop students’ problem-solving skills to increase achievement in this and in higher-level courses.
  • FCI normalized gains were significantly higher for students taking the flipped format with Mastering Physics.
  • The Mastering homework implementation was designed to discourage students from cheating and to motivate them to do extra practice without looking up solutions.
  • The instructor used Learning Catalytics to evaluate understanding and determine student misconceptions during class.

School name
University of North Georgia, Dahlonega, GA

Course name
Principles of Physics I

Course format
Face to face

Course materials
Mastering Physics for University Physics with Modern Physics by Young and Freedman

Timeframe
2015–2018

Educator
Sarah Formica, Associate Professor

Results reported by
Betsy Nixon, Pearson Results Manager

Setting

In early 2012, the University System of Georgia’s Board of Regents recommended the consolidation of North Georgia College & State University, founded in 1873 in Dahlonega, Georgia, and Gainesville State College, founded in 1964 in Gainesville, Georgia. The consolidation became official on January 8, 2013, creating the University of North Georgia (UNG).

  • Location: Five campuses across north Georgia
  • Type: Public university and one of six senior military colleges in the United States
  • Enrollment: More than 18,000 students
  • Gender: 57% female; 43% male
  • Full-time students: 68%
  • Pell Grant recipients: 28%
  • Race/ethnicity: 75% White; 12.4% Hispanic; 4.4% Black or African American; 3.7% Asian, Native Hawaiian, or Pacific Islander; 3.3% Multiracial; 1% Unknown; 0.2% American Indian or Alaskan Native

About the Course

Associate Professor Sarah Formica has been at UNG for 13 years. She teaches physics and interdisciplinary STEM courses, including Principles of Physics I, a calculus-based introductory course which covers material from mechanics, thermodynamics, and waves. The course includes three lecture recitations per week, with a required concurrent lab. Students are required to have completed Calculus I with a C or better or permission of the instructor. It is the first course in a two-semester sequence with an optional third semester of Modern Physics. The majority of students who take Principles of Physics are Physics, Chemistry, or Math majors, with many planning to transfer to Georgia Tech to pursue a degree in Engineering. Formica’s course goals are for students to learn how to approach, solve, and understand a wide variety of qualitative and quantitative physics problems, to relate classroom physics to real-world applications, and to develop and retain conceptual understanding and problem-solving skills required to do well in future courses.

Challenges and Goals

Formica has taught all levels of physics and has found that when students get to upper-level courses, they often cannot remember fundamental concepts, are not able to extract meaning and apply what was taught in the early courses, and tend to struggle to write out full solutions. In addition, she finds it frustrating to see some students getting high homework scores when they were likely cheating on the homework. She conducted a webinar, How Mastering can help your students learn to think like a Physicist, discussing her course goals and design. During the webinar, she cited an MIT study that recorded wrist-sensor readings of a student that showed regular, strong spikes during periods of study, lab work, and homework, but the readout flatlined during two activities: attending class and watching TV (figure 1).1

Because lecture is a passive activity, there has been a growing interest in active classrooms and the flipped model as a way to improve performance in STEM courses, and several studies have been published that support this initiative. Published in 2014 in the Proceedings of the National Academy of Sciences, “Active learning increases student performance in science, engineering, and mathematics” summarized findings from 225 studies. The conclusion was that active learning leads to increases in examination performance that raised average grades by a half a letter, and that failure rates under traditional lecturing increase by 55% over the rates observed under active learning.”2 Formica was driven by an interest in Eric Mazur’s peer instruction and the research around active learning, so she began a redesign of her course to address the issues stated above. She now uses activities to engage and motivate students, and to help students learn to think like physicists so they can apply the concepts, solve problems, and develop the necessary skills in the introductory course to do better in future courses.

Electrodermal Activity

Figure 1. Electrodermal Activity: Measure of Sympathetic Arousal Associated with Emotion, Cognition, and Attention3

Implementation

Formica’s redesign of the Principles of Physics course took place over a 10-year period. As she made changes, she evaluated the effectiveness and impact, and then made adjustments accordingly. The first step was to add a pre-class warm-up assignment to encourage reading the textbook so students were prepared for the added in-class activities focused on peer learning. Pre-lecture videos were later added, and Mastering™ Physics (MP) was adopted in 2014 after using another publisher’s online program. The adoption of MP resulted in the use of Learning Catalytics™ (LC) during class time, and in Fall 2017, she made a further change to the homework format to address cheating. To assess learning, students take the Force Concept Inventory (FCI) pre- and post-tests, and Formica tracks normalized gains to better understand learning and performance. In Spring 2018, her course was comprised of the following assignments and assessments:

Quizzes and exams for course credit

  • Daily in-class quiz: A two-minute, paper-and-pencil quiz on a physics formula was administered at the start of each class.
  • Weekly in-class quiz: A ten-minute, paper-and-pencil, one-problem quiz was administered at the start of the first class of the week. The quiz problem is one of the ten problems assigned in the extra credit MP practice quiz (below).
  • Midterm exam: A fifty-minute, paper-and-pencil exam with five problems was administered in class.
  • Final exam: A two-hour, cumulative, paper-and-pencil exam with ten problems was administered in class at the end of the semester.

Required homework for course credit

  • Textbook readings: Textbook readings were assigned using Perusall, a free, collaborative eText platform, and were due by 7:00 a.m. before each class, with a required minimum of five annotations.
  • Discussions: Discussions using Perusall about the weekly problems were due by 7:00 a.m. on the first class of the week.
  • Mastering Physics: One daily MP problem was assigned at the end of each class and due before the start of the following class. 

Extra credit homework
The following items were assigned to students as optional, extra-credit work:

  • Two types of Mastering pre-lecture videos:
    1. Conceptual videos (7–9 minutes) that contain pause-and-predict questions with no calculations during the video, with another 3–5 questions at the end
    2. Videos with quantitative information and pause-and-predict questions, along with a few problem-based questions at the end. Formica developed these videos herself, and Pearson has acquired them and added them to Mastering for use by all adopters.

Initially when Formica added pre-lecture videos, she found learning gains decreased. After researching the issue, she believes a likely reason is that the type of videos she provided at the time allowed students to passively watch without any participation, so students weren’t engaged. She now uses interactive videos to engage students and encourage them to stop and think while viewing. To better understand the impact of different types of videos, see the online post, Khan Academy and the Effectiveness of Science Videos.4

  • Mastering Dynamic Study Modules (DSMs): Multiple-choice question sets that adapt to each student’s performance and offer personalized, targeted feedback to help master key concepts were due at the start of the first class of the week.
  • Learning Catalytics: Students participated in class activities answering via LC by phone, tablet, or computer. The results were recorded in Mastering. Formica would start off with a slightly easier question to get students engaged and not frustrated. She would ask 3–5 questions during class and assess the responses to determine what to do following the question, i.e, provide more explanation or move to another activity. If Formica used LC questions as part of a group activity, she told students to find someone with a different answer to enhance peer instruction. She has small class sizes which makes this feasible, but LC also has a function that automatically groups students based on their response. Questions for LC activities generally came from Eric Mazur’s content and other instructors who have shared peer instruction resources.
  • Mastering tutorials: Ten to twenty tutorial, conceptual, and quantitative problems were due at the start of the first class of each week. Tutorial problems contain hints to help guide the students in the learning process.
  • Mastering practice quiz: Ten MP quantitative problems were due at the start of the first class of each week for practice to develop problem-solving skills. One of the problems was used for the weekly for-credit, in-class quiz.

To help students develop their problem-solving skills, Formica requires written paper-and-pencil solutions on the for-credit, in-class quizzes. The solutions follow the ISEE format used in the textbook, which requires students to:

  • Identify the relevant concepts
  • Set up the problem
  • Execute the solution
  • Evaluate the answer

Prior to the Spring 2018 semester, required for-credit homework included 10–15 problems. Student feedback indicated it was too much work, and Formica felt that students would tend to do a few problems, get frustrated and stressed because of the amount of work, and then try to find solutions online. To address this issue in Spring 2018, she changed her homework and quiz implementation as follows:

  1. Students were required to do one for-credit MP problem per class meeting. With just one required problem, students tend not to get overwhelmed, so they do their own work to solve it. However, Formica recognized that one problem was not enough practice for most students to become proficient at problem solving, so she assigned the optional weekly MP practice quiz for extra credit.
  2. The weekly MP practice quiz contained ten end-of-chapter problems to give students more practice. Since it was optional, students could work as many problems as they needed or wanted. To encourage them to work more of the problems to improve their skills, Formica used one of those ten problems as the in-class, for-credit quiz.
  3. The one-problem, in-class, for-credit quiz was worth 25% of the total course grade. Students were required to turn in the written solution to the problem. If students successfully worked and understood the ten practice quiz problems, they would likely do well on the in-class quiz. Formica felt this format motivated students to do the optional practice without looking up solutions, so they would do better on the graded quiz and develop stronger problem-solving skills.

Class time tended to follow a format of:

  • One Learning Catalytics question from the reading
  • A mini-lecture lasting about ten minutes
  • A group problem or activity for peer learning
  • A few Learning Catalytics questions with some additional peer instruction

In addition, Formica generally demonstrated working through one problem that was similar to an end-of-chapter problem so students could see the written solution. During class, students worked on problems and simulated experiments to learn how to apply different concepts and problem-solving methods. They collaborated with classmates but submitted their own responses for credit through Learning Catalytics (LC). Grading for LC activities was a half of a point for submitting a response and an additional half of a point for submitting the correct response. Class attendance was checked and played a large role in the successful completion of course requirements. Formica reserved the right to drop a student with a W/F if six or more classes were missed. While students were encouraged to collaborate on homework outside of class, Formica recommended they start the problem themselves and write up their own solutions.

Formica did not permit use of note cards or formula sheets during exams. She believes that if students don’t learn the formulas in the initial course, they won’t be able to do the work in upper-level courses. To help students learn the formulas, she worked on them during class and quizzed students on the formulas. She wanted students to know them well enough that they focused on thinking about how to solve problems and not just “plug and chug” into a formula. Formica plans to continue to track student performance in her course, and the UNG physics department instructors plan to track performance in upper-level courses to understand the long-term impact that changes in the introductory course have made.

Assessments

  • 25% Weekly quizzes
  • 20% Midterm exam
  • 20% Final exam
  • 10% Mastering Physics daily problems
  • 10% Perusall reading
  • 10% Perusall weekly discussion problems
  • 5% Daily quizzes

Results and Data

All students taking this course at UNG take the pre- and post-FCI, a 30-minute, multiple-choice national standardized test measuring mastery of concepts commonly taught in a first-semester physics course. The FCI was designed to assess student understanding of Newtonian physics using everyday language and common-sense distractors.

The normalized gain is used to evaluate student performance. It was introduced by Hake in 1998 as “a rough measure of the effectiveness of a course in promoting conceptual understanding,” and has become the standard measure for reporting scores on research-based concept inventories. Hake defined the average normalized gain as: <g> = (<Post> – <Pre>)/(100 – <Pre>) where brackets indicate class averages. This measure is commonly described as “the amount students learned divided by the amount they could have learned.”5 Hake advocated using normalized gain because this measure strongly differentiated between teaching methods, but allowed for “a consistent analysis over diverse student populations with widely varying initial knowledge states.” That is, it appeared to be independent of population or pre-test scores, allowing instructors to compare their students’ learning to other students at very different kinds of institutions.6

Formica compared her Principles of Physics I normalized gains to another section of Principles of Physics I (not using MP) at UNG and students in the algebra-based physics course. Figure 2 shows the results during the period that MP was in use. The students in Formica’s section from 2015 through 2018 had significantly higher normalized gains than the other groups in this analysis.

Normalized change on Forced Concept Inventory

Figure 2. For N-count, See Table 1

Table 1. N-count for Normalized Change, 2015–2018

The Student Experience

Formica has received positive feedback from students about the different activities in the course. Students reported that Dynamic Study Modules helped them learn course concepts. They also said that they enjoyed the in-class activities using Learning Catalytics, and Formica has noticed how students get excited and more engaged during these activities. On course evaluations, students were asked to give feedback around the course design. They reported that the course organization helped them know what to expect and when to expect it, allowing them to schedule their time and stay current with their work. Formica also has heard from some students that the amount of work was high, and a small number left the course due to the workload. However, the majority of students did the work in a timely manner and, by the end of the semester, tended to appreciate the rigor because they realized how much they had learned.

Conclusion

In an effort to help students learn to think like physicists, Formica has focused on changing her course to include more active learning and technology to address student needs, enhance learning, and help prepare students for upper-level physics courses by improving their problem-solving skills and critical thinking. Results show that her students are performing better compared to students in other groups, and she has heard positive feedback about their experience in the course.

Formica said she feels that with the focus on practice and active learning, along with the use of diverse tools and resources in Mastering Physics and in the course, students are getting better at problem solving. She recalls that she has given some context-rich problems that required students to use concepts learned in prior chapters, including a problem that required them to recall concepts from six chapters back, and she was pleased that many were able to solve it, saying, “for an instructor, that was a good day.” She plans to continue to monitor student performance in her course, but also will be tracking it in upper-level physics courses to see if students are retaining the skills and how well they are applying what they learned in Principles of Physics to advanced courses.

 

1Poh, Swenson, Picard, “A Wearable Sensor for Unobtrusive, Long-Term Assessment of Electrodermal Activity,” IEE Transactions on Biomedical Engineering Vol. 57, No. 5

2Freeman, et al., “Active learning increases student performance in science, engineering, and mathematics,” PNAS, Vol. 11, No. 23, Retrieved from http://www.pnas.org/content/pnas/111/23/8410.full.pdf

3Poh, Swenson, Picard, “A Wearable Sensor for Unobtrusive, Long-Term Assessment of Electrodermal Activity,” IEE Transactions on Biomedical Engineering Vol. 57, No. 5

4Muller, Derek, “Khan Academy and the Effectiveness of Science Videos,”Action-Reaction: Reflections on the dynamics of teaching

5McKagan, Sayre, Madsen, “Normalized gain: What is it and when and how should I use it?” PhysPort

6Ibid