MyWritingLab educator study explores impact of instructional technology on academic achievement in developmental English course at California State University Bakersfield
MyWritingLab educator study explores impact of instructional technology on academic achievement in developmental English course at California State University Bakersfield
- Students who passed the course spent significantly more time using MyWritingLab and also completed more of the topics that were assigned to them.
- The correlation between the number of topics completed in MyWritingLab and students’ final essay score was moderately strong.
- Weekly progress reports exported from MyWritingLab inform class time, help instructors keep tabs on students’ progress, and alert instructors to individual students who may need extra help or encouragement.
California State University, Bakersfield, CA
Reading and Writing (English 80)
Face to face with optional lab time
MyWritingLab Paragraph to Essay; Pearson Writer; Quick Access by Troyka or Mosaics: Reading and Writing Essays by Flachmann (used by most English 80 instructors) and a nonfiction novel
Abby Flachmann and Valerie Turner, Instructors
Randi Brummett and Brooke Hughes,
Coordinators of CSUB MyWritingLab Headquarters
Results reported by
Sara Owen, Pearson Customer Outcomes Analytics Manager
Christine Leow, Manager, Impact Evaluation, Pearson Higher Ed Courseware
Yun Jin Rho, Director, Impact Evaluation, Pearson Higher Ed Courseware
California State University Bakersfield (CSUB) is part of the California State University public university system, the largest public university system in the United States, with 23 campuses and eight off-campus centers enrolling over 460,000 students. In 2015, CSUB served a total of 9,230 full- and part-time students; 87 percent of CSUB students are undergraduate students, and 13 percent are graduate/post-baccalaureate students. Fall 2015 data for CSUB undergraduates show that 25 percent of students are under age 25; 62 percent are female; and 52 percent identify as Hispanic. CSUB currently offers more than 50 different bachelor’s and master’s degree programs.
About this Study
Improving academic achievement and graduation rates for first-generation and low-income students is one of the greatest challenges faced by community colleges and universities. The following educator study explores how Pearson’s MyWritingLab supports students’ academic achievement in developmental education courses. To gather more detailed information about how MyWritingLab was used at California State University Bakersfield, a study was designed to understand more precisely the factors that influence how students use MyWritingLab and the impact of using MyWritingLab on students’ performance on final essays.
About the Course
Reading and Writing is a 10-week developmental course focused on writing development from paragraph to essay. The course is designed to improve reasoning, reading, writing, and research skills, helping students to build a strong foundation for success in college. Annual enrollment is approximately 450 students, with roughly 13 sections running in the Fall, 4–5 sections in Winter, and one in Spring. Sections are capped at 25 students. The course may be taken for five credits toward full-time status, but does not count toward the baccalaureate degree because it is a remedial course. Students are placed into the course based on their English Placement Test (EPT) results. EPT scores are reported on a scale from a lowest possible score of 120 to a highest possible score of 180. Students scoring 147 or higher are deemed ready for college-level English. Students scoring below 147 are required to enroll in preparatory instruction.
Challenges and Goals
A decade ago, MyWritingLab was selected for use at CSUB because instructors felt it would provide a consistent, comprehensive resource for a multiplicity of purposes in a multitude of courses and disciplines. Read a comprehensive 2015 study outlining 10 years of MyWritingLab results and implementation at CSUB. MyWritingLab could address students’ individual learning needs, despite large class sizes. It could be used to track and measure student learner outcomes through mastery of topics and gains from pre- to post-assessment. In addition, the myriad topics of study available in MyWritingLab, from basic grammar to how to avoid plagiarism, would offer instructors the opportunity to tailor its use.
MyWritingLab is currently required across dozens of courses at CSUB, but for English 80 specifically, its overall purpose is to help students improve their grammar and writing skills outside of class. Instructors don’t have time to address basic and sentence-level grammar instruction in the classroom, so assigning outside work in MyWritingLab frees up instructors to address critical thinking, development, and other important topics during class time. To legitimize it as a requirement, MyWritingLab counts toward 10 percent of the final grade to ensure that students are held accountable for the grammar work.
CSUB coordinators set up the MyWritingLab courses in advance for all instructors so that implementation is consistent across all sections. An introduction to MyWritingLab is given to students at the beginning of the term in a computer lab. Occasionally instructors may give students class time to work on MyWritingLab, but primarily students work on MyWritingLab on their own time, outside of class. Says Flachmann, “I don’t have to spend time on direct grammar instruction. I do give a short overview and provide examples of grammar in class, but sometimes I run out of time, so assigning MyWritingLab homework does free up a lot of direct instruction time.” Students may work on MyWritingLab in the MyWritingLabPlus Headquarters, other computer labs on campus, or their own personal computer.
Students start by taking the multiple-choice Path Builder diagnostic test which covers content from 10 topics. Path Builder results give instructors a general idea of the strengths of the class as a whole and each individual student. Following the Path Builder assessment, students complete 10 required topics. Each topic includes an Overview (read), Animation (watch), Recall 1 and 2 (multiple choice), and Apply (multiple choice). The Post-test is not included. Pre-requisites are set so students must work through each topic in sequence. They have three attempts per assignment; mastery is set at 80 percent. Students finish by taking the Mastery Check post-assessment. Neither the Path Builder nor the Mastery Check are timed nor proctored. There is no specific score required on the Mastery Check; Flachmann tells her students they need to at least score higher on the Mastery Check than the Path Builder. Mastery Check results are intended to serve as a baseline for comparison with Path Builder results so instructors can see the progress students have made.
Students are advised to master approximately one topic a week in MyWritingLab in order to stay on track. Data from Fall 2015 showed that the average total time students spent working in MyWritingLab during the quarter was six hours. All topics are due at the end of the semester. Instructors may choose from three grading options:
- Option 1: Give students credit/no credit for completing the topics, Path Builder, and Mastery Check.
- Option 2: Give students points for each topic and deduct points for late work.
- Option 3: At the end of the quarter, give students their overall average score out of 100 for MyWritingLab topics/Path Builder/Mastery Check, and deduct points for late work.
Throughout the quarter, CSUB’s two MyWritingLab coordinators distribute weekly reports to each instructor with information on their students’ progress. Many instructors use these reports to check in with their students. Says Flachmann, “I address the class as a whole to remind students to work on MyWritingLab, but if there are individual students who are falling behind, I pull them aside and talk to them in person to make sure they’re keeping up with their work. I also give praise for students who are doing well and completing topics as they should.” Flachmann acknowledges that, “MyWritingLab work does inform how I spend class time.” If she sees on a report that students are getting stuck on a topic, or if students are exhausting a topic, she will spend more time on that topic in class. Or if students have already completed a topic, but she sees that it’s not translating to their work, she will refer them back to the topic in MyWritingLab.
In addition to MyWritingLab, students use Pearson Writer, specifically the Automatic Writing Review component, to help identify and fix grammar errors in their writing. Students may also be required to use Pearson Tutor Services and submit up to six pieces of writing to an online tutor for a detailed writing review. Some instructors require students to print out the Pearson Tutor Services response and turn it in with their final draft for credit. Turnitin.com is required across the department, and Blackboard is the Learning Management System in use at CSUB.
In class, students work on reading or writing strategies with a focus on current writing. Instructors spend more time at the beginning of the quarter on lecture and lessons, but later, class time is devoted to self- and peer-evaluations, group activities and discussion, revision workshops, and student presentations. All incoming freshmen read the same book, and students typically write an essay about this book.
All students in Reading 80 complete a synthesis as their final written assignment, approximately three pages in length. This final essay requires a synthesis of summary, paraphrasing, and quotes, using two to three nonfiction sources provided by the instructor. Using familiar sources helps the instructor and the students readily identify and address issues of plagiarism.
There is tremendous MyWritingLab support for both students and instructors at CSUB. The MyWritingLabsPlus Headquarters opened in 2012 to enhance instructor training and student support. The Headquarters has 21 student computers, a sign-in computer, and a projector for demonstrations and presentations. Formal MyWritingLab training is provided for new instructors every year, and the Headquarter coordinators, tutors, and student assistants are always available to answer questions.
Writing Resource Center
In the first weeks of English 80, instructors walk students over to the Writing Resource Center (WRC) which offers writing tutoring across any discipline. This familiarizes students with its location and with the services it provides. Instructors may enforce a student visit to the WRC with a referral form that needs to be signed and returned after a session. Turner believes that it is unlikely students will go to these services on their own (they need a nudge to get there), but says, “once they go, the impact is quite positive.” Flachmann requires her students to attend tutoring and has found that if students go once, they will continue to go back.
Since 2013, a Summer boot camp entitled Early Start is required across all California State University campuses for any student who places into remedial classes via the English Placement Test. This ends up being a majority of incoming freshman (18 sections ran in Summer 2015 at CSUB). Early Start is offered as soon as high school gets out, and a few sessions run simultaneously. Students stay in dorms at CSUB during this time and, for two weeks, spend four hours a day, Monday through Thursday, completing the program.
Accounting for about 20 percent of the Early Start workload, students are required to master three topics in MyWritingLab. Tutors visit for one or two days to speak to and work with students. Students write two essays, one of which is an in-class essay, written on the second-to-last day of class. The last day is spent conferencing and advising with students. At the end of the two weeks, if students’ final writing assignments are adequate, they can potentially skip a level or two of remedial English. This is the incentive to get them to try their best.
Because Early Start introduces many students to the format of MyWritingLab, instructors have found they have fewer troubleshooting issues when the Fall term begins because students already have a solid foundation on how MyWritingLab works.
- 80% Reading and writing assignments, including:
- Out-of-class essays (2)
- In-class essays (2)
- Final essay (worth 15–20%)
- Pearson Tutor Services
- Peer editing
- In-class assignments
- Discussion board/journal/blogs
- Participation and homework
- 10% MyWritingLab
- 10% Instructor’s discretion
Results and Data
The purpose of this study’s data analysis was to isolate the unique contribution that MyWritingLab had on student academic performance in the course after controlling for other factors that also impact student performance (students’ self-reported confidence levels on a scale of 1–6; students’ comfort levels with technology on a scale of 1–10; prior ability) to determine if MyWritingLab still had an impact on student performance in the course. Students were surveyed at the beginning and at the end of the course.
Hierarchical linear modeling techniques were used to analyze the individual contribution of each set of variables, including impacts on achievement that could be due to differences in:
- The way the course was taught by each teacher;
- Demographic characteristics and goals of students;
- Students’ levels of confidence and comfort with both writing and using technology-based instructional products;
- Prior ability as measured by the Path Builder assessment;
- Obligations that have been shown to impact college students’ abilities to focus on school work;
- Level of academic support students received;
- Students’ self-reported level of engagement;
- Students’ self-reported study behaviors within the writing course; and
- Students’ perceptions of the utility of MyWritingLab.
Figure 1 is a correlation graph; correlations do not imply causation but instead measure the strength of a relationship between two variables, where r is the correlation coefficient. The closer the r value is to 1.0, the stronger the correlation. The corresponding p-value measures the statistical significance/strength of this evidence (the correlation), where a p-value <.05 shows the existence of a positive correlation between these two variables.
- A moderately strong positive correlation exists between the number of MyWritingLab topics completed and final essay scores, where r=.50 and p<.01.
- The average final essay score was 82 percent; the average number of MyWritingLab topics completed was seven.
- Note: eight students were removed from the correlation due to having a zero score on the final essay or zero MyWritingLab topics completed.
Correlation between number of MyWritingLab topics completed and final essay score
Figure 1. Correlation between Final Essay Score and Number of Topics Completed in MyWritingLab, Fall 2015 (n=106)
Preliminary results from Hierarchical Linear Modeling (HLM) analysis (not shown in this study) suggest the impact of using MyWritingLab is additive. For every MyWritingLab topic the students worked, the average gain in the final essay score is about two percentage points. On average, students’ use of MyWritingLab was associated with a 14 percentage point total gain in final essay scores. These data cannot be captured by a correlation analysis. However, figure 2 illustrates average final essay scores based on the number of topics completed in MyWritingLab.
Additionally, preliminary Hierarchical Linear Modeling analysis shows that after controlling for instructor and student variables, students’ reported perception of the effectiveness of MyWritingLab (from the student survey) was positively related to final essay (p<.05). For more information about the HLM analysis/methodology, please email firstname.lastname@example.org.
Average final essay score based on number of topics completed in MyWritingLab
Figure 2. Average Final Essay Score Per Number of Topics Completed in MyWritingLab, Fall 2015; Total (N=114); 0 topics (n=3); 1 topic (n=5); 2 topics (n=7); 3 topics (n=2); 4 topics (n=3); 5 topics (n=3); 6 topics (n=7); 7 topics (n=10); 8 topics (n=14); 9 topics (n=13); 10 topics (n=47)
Figure 3 shows that students who passed the course spent significantly more time using MyWritingLab and also completed more of the topics that were assigned to them. Between students who passed and students who did not pass, the differences are significant for time spent (hours) in MyWritingLab (p<.05) and number of topics covered in MyWritingLab (p<.0001). Out of 169 students enrolled in Reading 80 in Fall and Winter 2015, 137 students took a pre-survey, and 106 students took a post-survey. The pre- and post-survey were both taken by the same 94 students, but 19 of these students were removed due to skipping questions in the surveys or missing grade data. This leaves an analytic sample of 75 students for figure 3.
While the amount of time that students spend using MyWritingLab is important, data from the HLM analysis suggest that the number of topics that a student completes within MyWritingLab is most important. If one student spends six hours completing one topic in MyWritingLab and another spends six hours completing two topics in MyWritingLab, it is the student who completes two topics that is likely to achieve a higher grade.
Passing the course and MyWritingLab usage
Figure 3. Time Spent (Hours) in MyWritingLab and Number of Topics Covered in MyWritingLab; Overall Analytic Sample (n=75); Students Who Passed (n=69); Students Who Did Not Pass (n=6)
The Student Experience
Out of 169 students enrolled in Fall and Winter 2015, a pre-survey was taken by 137 students (81 percent response rate), and a post-survey was taken by 106 students (63 percent response rate). Of the 94 students who had both pre- and post-survey results, 19 were removed from the analytic sample due to skipping questions in the surveys or missing grade data. The final sample consists of 75 students. Table 2 lists self-reported data from the student survey.
Self-reported student survey results
Table 2. Self-reported Student Post-survey Results, Fall 2015 (n=75)
Students were asked both multiple-choice and open-ended questions on the post-survey. The following summary responses are related directly to MyWritingLab use.
When using the MyWritingLab in this course:
67 percent of respondents agree: “I had access to a greater variety of learning materials.”
64 percent of respondents agree: “I was able to use different approaches to help me learn.”
58 percent of respondents agree: “I was more engaged in the learning experience.”
45 percent of respondents agree: “I came to class better prepared.”
Selected student responses to the question, “What do you think are the benefits of using MyWritingLab?”:
- “You gain better understanding on what you need to focus on.”
- “Applying what you know about grammar into your essays”
- “If you are unclear on a topic in class, MyWritingLab will give a chance to go over the material again.”
- “By practicing the homework and quizzes online, once you get to class you already know what’s going on in the lesson.”
- “The material we learn will be refreshing our minds with the material we may have forgotten about.”
- “It gives an example of the topic, explains why you got an answer wrong.”
- “To more effectively edit your own and other’s papers”
- “It helps us be prepared when writing essays.”
- “Greater access to a variety of learning materials”
- “You get to visually see and hear the lesson.”
- “It helps those who are afraid to ask questions.”
- “You can do the work at home.”
Selected student responses to the question, “How has MyLab impacted your learning in this course?”:
- “I can write much more effectively.”
- “MyWritingLab has impacted my learning because I have a better understanding of what fragments and comma splice are.”
- “Now I am more aware of my problem areas, and I know how to fix them.”
- “It taught me a lot of things I should have known during high school.”
- “I’ve noticed that my writing has improved greatly.”
- “MyWritingLabPlus has made learning and the course a little more fun I guess. It gave me motivation when I knew what I was doing.”
- “I get to work on my own pace with no pressure.”
- “Helped me strengthen my writing skills”
- “I can now see my grammar mistakes.”
Selected student responses to the question, “What have been the challenges (if any) in using MyLab?”:
- “The Apply part was always challenging.”
- “Getting the work done because I don’t have a computer.”
- “Getting it done and remembering that I had it”
- “A challenge in using MyWritingLabPlus would be passing the quizzes at times. I would need have to sometimes need them to unlock the quizzes to have more tries.”
Students are asked to complete the same MyWritingLab topics in English 80 (Reading and Writing), 99 (Critical Thinking and Writing), and 110 (Writing and Research) because instructors believe students need the reinforcement. Says Flachmann, “It’s repeated so they get better.” By the time students are in English 110, additional non-grammar topics are assigned in MyWritingLab.
With MyWritingLab, instructors Flachmann and Turner are looking for the transfer of grammar and basic writing skills learned from MyWritingLab. They share, “You see a bit of transfer in English 80, a little more in English 99, and then really you see it the most at the end of English 110. It takes time for grammar skills to transfer to writing.”