Mastering Engineering University Iowa

EDUCATOR STUDY

Mastering Engineering educator study from the University of Iowa summarizes published research paper findings

Key Findings

  • The outcomes of lecture-based and student-centered instruction were compared in concurrent, 140+ student sections of an electrical circuits course.
  • The student-centered section used Mastering Engineering to provide students opportunities for practicing problem solving during and outside class.
  • The student-centered section’s second and final exam averages were higher than the lecture-based section after controlling for students’ GPAs and ACT scores.
  • Student surveys indicated that the student-centered section’s students spent, on average, more time preparing for class and missed fewer classes than the lecture-based students.
  • Read the full study, “Large Lecture Transformation: Improving Student Engagement and Performance through In-class Practice in an Electrical Circuits Course” in the ASEE’s Advances in Engineering Education journal.

School name
University of Iowa, Iowa City, IA

Course name
Electrical Circuits

Course format
Face to face

Course materials
Mastering Engineering for Electric Circuits by Nilsson and Riedell

Timeframe
Fall 2014

Educator
Mark Andersland, Associate Professor

Results reported by
Betsy Nixon, Pearson Customer Outcomes Analytics Manager

Setting

Enrollment: 24,475 undergraduate; 7,535 graduate
College of Engineering enrollment: 2,202 undergraduate (Fall 2017)
First-year retention rate: 86%
First-generation students: 23% (class of 2020)
Minority students: 19% (class of 2020)

About the Course

Mark S. Andersland, Ph.D., is an associate professor of Electrical and Computer Engineering at the University of Iowa (Iowa). In Fall 2014, Andersland began a research study with Jae-Eun Russell, Ph.D., an instructional and research specialist in the Office of Teaching, Learning and Technology at Iowa, with support from Sam Van Horne, Ph.D., assessment director in the Office of Teaching, Learning and Technology at Iowa; John Gikonyo, a graduate student at Iowa; and Logan Sloan, an undergraduate student at Iowa. The study’s results were published in a 2017 paper entitled, “Large Lecture Transformation: Improving Student Engagement and Performance through In-class Practice in an Electrical Circuits Course,” in the ASEE’s journal, Advances in Engineering Education.1 A summary of the study and its results follows. Full results can be found in the published paper.

The study examined student performance in the three-credit electrical circuits course taught by Andersland that is required for all engineering majors at Iowa. After completing a calculus-based physics course, students usually take the electrical circuits course during their sophomore year along with a differential equations course. Iowa generally offers three to four sections of electrical circuits each year with 140 to 180 enrolled in each section. “The objectives of the course are to explore fundamental electrical quantities, components, and concepts, and to develop ad hoc and systematic tools for circuit analysis. The topics covered include: circuit variables; sources, resistors, and Ohm’s law; ad hoc, nodal, and loop analysis; scaling and superposition; the Thévenin-Norton and maximum power theorems; operational amplifiers; capacitors, inductors, and mutual inductance; first-order transient analysis; phasor analysis; and sinusoidal power analysis. The course has no laboratory component but includes several direct-current transient, and alternating-current, circuit simulation (Multisim) assignments.”2

Challenges and Goals

Problem solving is a key component in the electrical circuits course and a skill fundamental to success in engineering. Traditionally instructors teaching Iowa’s electrical circuits course had lectured on concepts and problem-solving strategies during class time and assigned homework problems for practice outside of class. To develop good problem-solving skills, however, students need to solve more than just assigned homework problems. Unfortunately, due to student time constraints, too often students were not solving extra problems for practice when they got stuck, and consequently, were never working enough problems to become truly proficient problem solvers.

In Spring 2014, the semester prior to the study, Andersland decided to make changes to the course. He “wanted to find a way for every student to take ownership of what he or she was learning.”3 He hypothesized that by adopting a student-centered approach that encouraged problem solving practice, students’ scores on problem solving exams could be improved. The purpose of the study was to investigate whether “even in an auditorium-taught course, students’ learning outcomes (exam scores) could be improved through in-class practice and immediate feedback in collaborative learning environments.”4

Implementation

Prior to the study, all electrical circuits sections at Iowa had been taught in the traditional lecture-based format with paper-and-pencil problems assigned for completion outside of class. During the Fall 2014 study, one section was taught as a traditional lecture-based course, while the second section adopted a student-centered model in which lectures were moved out of class to free up class time for in-class, problem-solving practice. Students were not informed of the sections’ differences before they registered. A week before classes began, Andersland sent an email informing registered students of the differences, the study, and the requirement that students bring a smart device to every class. Students were then permitted to switch sections and opt in to the study. Ultimately 79% of the total enrollment of both sections — 127 from the lecture-based section and 116 from the student-centered section — opted in. “Of the 243 study participants, 74.9% were male and 72.4% were sophomores. The percentage of male and sophomore students was not significantly different between the two sections.”5

Students in both sections attended three, weekly, 50-minute instructor-led class sessions and one 50-minute teaching assistant (TA)-led discussion each week. The traditional instructor-led class sessions were devoted solely to lecture. Changes made to the student-centered, instructor-led sessions were designed to give students more opportunities to practice problem solving and receive support while practicing. The changes included:

  • Adoption of a student-centered classroom model that moved lectures outside of class to free up class time for collaborative, in-class, instructor-assisted, problem-solving activities and practice
  • Use of Mastering™ Engineering to more efficiently distribute different versions of the shared, interactive, and auto-graded problems underlying the class activities to each student
  • Development of problems that could be completed in class that introduced concepts and tested understanding in steps and would interest and challenge students of varied abilities
  • Support for multiple levels of feedback as students worked problems including: correct/incorrect indications and hints, and on-demand assistance from classmates and instructors
  • ​Access to Mastering and required instructor-led active learning training for TAs to enable them them to more effectively guide and coach student learning

Both the lecture-based and student-centered sections’ students were expected to solve approximately 75 paper-and-pencil homework problems over the course of the semester. The lecture-based section’s submissions were hand-graded by the section’s TAs. The student-centered section’s problems were distributed and graded via Mastering Engineering, allowing the parameters of each student’s assignment to be varied, and freeing time for the TAs to assist students during the instructor-led sessions.

By design, students in the student-centered section were also given the opportunity to solve 442 additional problems, 52% (232) of which were to be worked in class and 48% (210) of which were made available for optional pre-class practice. Like the homework problems the in-class and pre-class practice problems were distributed and graded via Mastering Engineering. On average, at least 71% of the in-class problems were attempted. Consequently, by the end of the semester, student-centered section students had, on average, “attempted at least twice as many problems as those in the lecture-based section.”6

To try to avoid reducing course content or increasing the amount of work outside of class to accommodate the in-class activities, the following additional course-design decisions were made for the student-centered section.

  • Lectures were limited to a 10-minute concept summary, freeing 40 minutes for problem solving
  • Omitted details were covered in videos posted to the course website to ensure availability
  • Problems were delivered via Mastering Engineering for efficiency and to provide answer feedback
  • Problem parameters were randomized to encourage strategy as opposed to answer sharing
  • Answers to in-class problems were graded for credit to encourage attendance and problem ownership
  • Multiple, penalty-free, answer attempts were allowed to help students identify misconceptions
  • Collaboration and requesting instructor help were encouraged to more quickly resolve misconceptions
  • A curated website of readings, videos, and extra problems was created to aid pre-class study
  • The coverage and difficulty of the lecture section’s assignments was mirrored for continuity7

The course designs for the lecture-based and student-centered sections are summarized in table 1.

Course design in lecture-based and student-centered sections

Table 1. Lecture-based vs. student-centered course structure8

To address the need in the student-centered section to efficiently deliver and score a large number of problems in a short period of time, Mastering Engineering was adopted as the section’s problem delivery platform. Andersland felt it was the best available problem delivery platform because:

  1. It came with a large library of editable circuits problems which reduced startup overhead
  2. It supported symbolic and randomized problems, enabling different parameterizations of the same problems to be assigned to each student to focus students’ discussions on the processes by which problems could be solved, as opposed to specific answers
  3. Its problems are typeset in TeX, the language commonly used by STEM professionals to typeset technical documents with significant math content which reduced typesetting overhead
  4. Its response to inputs was quicker than that of other problem delivery platforms
  5. The student interface was easy to learn and supported integration with Iowa’s LMS (Canvas)

All homework, in-class, and pre-class problems distributed in the student-centered section were delivered and scored using Mastering Engineering. So, not only was the problem distribution and grading efficient, but students also received immediate correct/incorrect feedback and hints that helped them to more quickly recognize misconceptions in their understanding and work. “Students were limited to three answer submissions per problem without penalty in class and were penalized 20% per answer submission per problem on homework problems. No limits or penalties were imposed on the pre-class problems since they were optional.”9 Table 2 summarizes the implementation of the Mastering Engineering problems in the student-centered section.

Implementation of Mastering in student-centered section

Table 2. Mastering Implementation for the Student-centered Section

Assessments

Grades for the two sections were assessed as outlined in table 1. In the lecture-based section, students’ best 10 of 12 homework scores accounted for 20% of their grade; their scores on two, two-hour midterm examinations 50%; and their score on a two-hour final examination accounted for the remaining 30%. In the student-centered section, students’ scores on the best 37 of 43 in-class assignments accounted for 10% of their grade; their scores on the best 10 of 12 homeworks 15%, their scores on two, two-hour midterm examinations 45%; and their score on a two-hour final examination, the remaining 30%.

Results and Data

The study examined student performance based solely on the students’ scores on their two midterm exams and their finals, which corresponded to 80% of the course grade in the lecture-based section and 75% of the course grade in the student-centered section. The exams covered all course learning objectives and, in every case, students had to solve circuits problems to determine their answers. All exams were two hours long and consisted of 18, five-point questions. The two sections shared 15 common questions on the first midterm, 14 on the second, and 15 on the final. Only the common questions were included in the study.

“[R]esults of regression analyses indicated that the students in the student-centered section achieved significantly higher test scores on the matched exam items of the second midterm and final exam than the students in the lecture-based section after controlling for their prior learning outcomes (cumulative GPA before taking the circuits course and ACT math scores).”10 The student-centered students’ first midterm was also higher, but not statistically so. Detailed data and findings, descriptive statistics for the test scores for matched items, Cumulative GPAs, and ACT math scores, results of the linear regressions predicting the exam scores, and conjectures as to why the student-centered approach seemed to have less impact on students’ first midterm scores, can be found in the published paper. “Although the study provided robust evidence of the benefits of a student-centered instructional approach compared to lecture-based teaching in a large Electrical Circuits course, its limitations prevent identification of the origins of these benefits. Further research is suggested to determine which specific activities and constructs of student-centered instruction significantly impact learning, which of these activities are transferable to other courses, and whether the same student-centered instructional approach would achieve better learning outcomes in active learning classrooms than in auditoriums.”11

The Student Experience

Participating students from both sections were given three surveys during the study. Each respondent answered questions concerning their demographics, prior learning outcomes, and course outcomes. Participation was voluntary, so respondents were given a small cash compensation for their time. The surveys were as follows:

  • Survey 1: Administered during the second week of the semester; assessed students’ initial self-efficacy, intrinsic value, and test anxiety
  • Survey 2: Administered during the seventh week of the semester following the students’ first midterm exam; assessed students’ perceptions of the course and its technology
  • Survey 3: Administered at the end of the semester; assessed students’ final self-efficacy, intrinsic value, and test anxiety, as well as students’ final perceptions of the course and its technology12

Findings included:

  • That all three types of engagement were consistently higher in the student-centered section than in the lecture-based section
  • That the post self-efficacy and intrinsic value in the student-centered section were significantly higher than those in the lecture-based section
  • That the post-test anxiety between the sections was not significantly different
  • That students in the student-centered section reported that they were significantly better prepared for class, and learned more from their mistakes and the feedback received, than the students in the lecture-based section
  • That students in the student-centered section reported significantly higher perceptions of the improvement of these skills than the students in the lecture-based section
  • That in both sections, students’ satisfaction with the course was relatively low13

Details of these findings can be found in table 5 of the published paper.14 Additionally, respondents from the student-centered section reported attending class more frequently throughout the semester and spending more time studying each week than students in the lecture-based section. They also reported that solving many in-class problems with peers and the instructional staff was helpful to their learning even though it was stressful to prepare for each class and to complete the assigned problems in the allotted time during class. In both sections, students responded that among the most helpful aspects of the course were the example problems, homework problems, and help from the TAs and their peers. Specific student-centered section comments included:

  • “Completing so many problems helps me understand the concepts much better.”15
  • “The most helpful aspect is that I get to work on more problems and get immediate feedback.”16

Conclusion

There is growing interest in using active classrooms and student-centered instruction to improve performance in the STEM disciplines, and several studies supporting these approaches have been published. The paper, ”Active learning increases student performance in science, engineering, and mathematics,” published in 2014 in the Proceedings of the National Academy of Sciences, summarized findings from 225 published studies. It found “that active learning leads to increases in examination performance that would raise average grades by a half a letter, and that failure rates under traditional lecturing increase by 55% over the rates observed under active learning.”17

Since the Iowa study’s conclusion, Andersland reports that the specific problems assigned and the classrooms in which the course has been offered have evolved, but that the student-centered course format implemented as part of the study has remained largely the same and is now offered almost every semester. The Fall 2014 student-centered section was offered in a conventional auditorium seating 150+ students. Spring 2015–Fall 2017, the course was offered in a technology-rich, active-learning classroom containing only round tables. In Spring 2018, the course was offered in a new hybrid auditorium/active-learning classroom containing rows of well-spaced rectangular tables. According to Andersland, “The classroom environment makes a difference in administering the format.”18 Iowa is continuing to conduct studies of this issue.

Mastering Engineering has continued to be the platform used for problem solving both in and out of the class. Andersland emphasized, as a best practice, that, “Parameterizing the students’ problems is important. Students collaborate because they are working the same problems, but their discussions will then focus on the problem-solving approaches they are using because their answers will be different.”19

Finally, Andersland pointed out that, “There is overhead to consider when moving to this format, as curriculum must be created for both in and out-of-class activities. The in-class curriculum involves minute-by-minute planning as opposed to the traditional class-by-class or week-by-week planning. Consequently, continued support from college administrators and educational publishers will, at least early on, be essential.”20 The Iowa study, conducted over a seven-month period, was a joint effort involving the course instructor and an instructional designer from the university’s Office of Teaching, Learning and Technology, supported by the University’s Large Lecture Transformation project. It included course planning, design, and development phases.

Andersland summed up the classroom experience for the student-centered approach by observing that, in the moment when students are working problems and get stuck, “They are ready to learn, and that’s what is very interesting about the approach. They get immediate feedback. They can turn to their neighbor and get instantaneous help from someone working on the same problem, so it’s fixed and it sticks, because they have an a-ha moment.”21

_______________________________________________________________

1 Russell, Andersland, Van Horne, Gikonyo, and Sloan, “Large Lecture Transformation: Improving Student Engagement and Performance through In-class Practice in an Electrical Circuits Course,” Journal of Advances in Engineering Education

2Ibid, p. 5

3 The University of Iowa Engineering, Iowa Engineer: Reinventing the Classroom, Volume 2016, #2, p. 3 Retrieved from: http://www.pageturnpro.com/University-of-Iowa-College-of-Engineering/73203-Iowa-Engineer-Magazine,-2016-No-2/default.html#page/3

4 Russell, Andersland, Van Horne, Gikonyo, and Sloan, “Large Lecture Transformation: Improving Student Engagement and Performance through In-class Practice in an Electrical Circuits Course,” Journal of Advances in Engineering Education, p. 9

5 Ibid, p. 10

6 Ibid, p. 7

7 Ibid, p. 6

8 Ibid, p. 7

9 Ibid, p. 8

10Ibid, p. 20

11Ibid, p. 20

12Ibid, p. 10

13 Ibid, pp. 13–15

14 Ibid, p. 13

15 Ibid, p. 16

16 Ibid, p. 17

17 Mark Andersland, personal communication.

18 Andersland, personal communication.

19 Andersland, personal communication.

20 Freeman, et al., “Active learning increases student performance in science, engineering, and mathematics” PNAS, Vol. 11, no. 23, Retrieved from: http://www.pnas.org/content/pnas/111/23/8410.full.pdf

21The University of Iowa Engineering, Iowa Engineer: Reinventing the Classroom, Volume 2016, #2, p. 2 Retrieved from: http://www.pageturnpro.com/University-of-Iowa-College-of-Engineering/73203-Iowa-Engineer-Magazine,-2016-No-2/default.html#page/3