Effective Use of Data Goes Beyond the Digitally Generated Kind
Q. Please tell us about your General Biology course and students?
Sandra: My General Biology course is designed for non-majors, which means any student at Rochester Institute of Technology (RIT) who has not declared Biology as a major. The class is open to all types of students, and I believe is a strong reflection of what we are seeing across our universities.
I have more transfer students (internal and external to RIT) than I have seen over the past decade. In particular, I had a conversation with a student recently who told me that RIT is her FIFTH college! She has no degree, she has only completed 60 credits in total, and is entering RIT with freshmen status because she does not have enough credits to align with a program. Further, our between program “transfer” students are struggling to find their way, fulfill credits, and actually complete a degree.
I have quite a few first generation and international students in my courses. In many ways, these students are not different from my transfer students – they too struggle finding their way.
Q. Professors are trained to collect and understand data. What do you think is the most important data to collect to understand your students?
Sandra: For my class, knowing the primary majors of my students and being able to gauge their level of commitment to the course materials are first and foremost data points. Before the first day of class, I do a “roster analysis” and determine the demographics of my class. As I teach 250 students in a section, knowing this information is crucial to making connections, and earning respect and trust from day one. How many freshmen vs. fifth year students? How many computer science majors vs. biomedical science majors? I share these data with my class so that they have a better understanding of why some days we talk about weird applications of information (e.g. the evolution of computer programming and the tensile strength of fibers used in prosthetics) instead of just talking about biology. And they have a better understanding of why I say on day 1 “I expect you to know nothing about biology!”. When the class knows as a whole that I am here to support their learning and engagement to the utmost of my ability, we all start on better footing.
Q. How do you use data–quantitative and qualitative–to understand your students and their progress or lack thereof?
Sandra: After the course starts, I collect “time on task” data from Mastering Biology and my course management system. I produce many plots of “time” vs. “attendance” vs. “grades” to show the students where they stand in the class. Not coming to all of the classes? Spending the least amount of time on assignments as compared to other students? Then they should not be surprised that they are earning grades in the bottom of the class. This makes a very strong statement that is clear for the students to see. I am not looking to develop a highly competitive atmosphere – but I am all about giving them the tools and information to make a difference in themselves. This information does just that. And, yes, students will change – not all – but enough to make this information extremely powerful.
I also run discussion boards in my course management system. The first discussion is always – Introduce Yourself. This introduction is to the virtual space (considered to be safe space in which to hide) for many students – but it allows them to meet other people in the class without fear of rejection, which is extremely common among students today. They might learn that someone graduated from high school in their neighboring town, or that some one else really likes to collect unicorns, or they have all of the same classes as several other people in the class this semester. After making these realizations, I see them finding one another in class – the “real” meeting! They now have made a connection to someone in the class (other than myself!), they form study groups, and they all improve – academically and personally. Since I have implemented the discussion boards, the students report stronger connections to the class and to the materials. This link may be coincidental – but if it even appears to be working in education – leave it alone!
Q. Using MasteringBiology, what data is most important to you? Why?
Sandra: To me, time on task and assignment grades are the most important superficial data to collect from Mastering. As I stated – it is a telling sign for students that if you want to improve, if you are not happy with your grades, a simple solution may be to just put in more time.
What happens then to the students who put in a lot of time and still struggle? This is when you need to dig. Are they answering questions in Mastering without hints – if so, why won’t they use them? Are they always missing the Mastering questions on the first attempts? If so, why? Are they rushing through the materials or do they really not know? If the students are not doing well on the assignments, are they completing the follow-ups? If not, why? If yes, are they improving? Sitting down with struggling students and showing them all of this information can be very eye opening – and, again, you are giving them information to make decisions about their pathway to success in your course.
Q. How do you use student progress data on homework, and exams to help improve your course?
Sandra: Students will infrequently use data on their own. Big picture data, such as my plots of “time” vs. “attendance” vs. “grades”, take class time to convey. You have to put it out there and take time to talk about it. You have to be willing to sacrifice that class time to make improvements. I have never regretted spending 5-10 minutes of class and saying – “Look at these data about our class. What do you think? How can we improve as a class?” Sometimes the solutions are targeted at me – better information on what they should study, etc. But, often, the fingers point back at themselves – how can they help one another to improve? Maybe study groups? Maybe increasing attendance at review sessions by all going in to split a pizza for dinner? These things have all come out of our discussions – and they are helping students to form a learning community in the classroom that benefits many more students than if I went in and tried to “incentivize” anything. Bonus points do not improve overall grades – they make students feel good temporarily. A true sense of a learning community in the classroom truly improves grades (and morale!) for everyone across the board.
Q. How do you make sense of all of the data you collect? You call some data points easy ones and others difficult, but you find them both important. Why?
Sandra: If you want to analyze course data – it is easy to look at the average score of an exam in year 1 and compare it to year 2. Did the average change? If so, why? Did you change something in class, give more assignments, or are the students just a different group? All are reasonable explanations. But these are data that are easy to come by, even if they are not always the easiest to directly interpret.
There are also data that are very hard to collect, but easy to interpret. My students come to class early. They rarely miss class. They sit (relatively quietly!) until the bitter end of class – even when I go over by a few minutes. Students leave my class with smiles on their faces. Why do they do these things? I have no idea! I cannot pin a data point on it and say “AH HA! I have the magic key to student satisfaction!” I have tried to find it and I wish I could! But, there must be something. My colleagues teaching the same materials in the same semesters with an equally odd proportion of majors and ages in their classes do not report the same qualitative results. Why are they not seeing the same things? They are using the same tools (learning management system, textbook, MasteringBio) – even the slides that we use are almost identical (mine originally). So what is the difference? I can’t tell you for certain. The only thing that the students consistently state in person and on my evaluations is “She loves her job!” Perhaps that is all it takes. If I could help every person to love their job – love teaching – love whatever it is that they do – I would! Why? Because then they would never “work” a day of their lives and everyone around them would be “infected” with that enthusiasm and desire to be better at everything that they do too. In the meantime, I will keep digging for that “magic key to student satisfaction”!
Q. What do you think is the most important aspect about data? And the most important data for course improvement?
Sandra: The most important thing about the data is that they must always be used with the intention to improve. No, not all data will demonstrate improvement. That is not what I am suggesting. Exam grades often bounce all over the place. Sometimes a homework assignment just wasn’t all that you had dreamed it could be! Sometimes we have to face the results of “failed” applications and “failed” implementations. But, we don’t have to stay down. Fix it! Use these data that are suggesting shortcomings as a reason to try a new direction – or, sometimes, go back to what you were doing! You tried something (more than once, please!), and it just doesn’t work for you, or the students, or the material. Fine! Try something else, or go back to what you know works for you, the students, and the material. It is ok to fail in our attempts to educate – but it is not ok to continue to fail when there are other options!
For me, the most important data to convey to my administration have been class enrollment numbers. My class is an elective for most of my students (>65%). The increase in my class size has far exceeded the rate of increase in the whole student body. My class has more than tripled in size in nine years. To me, this enforces that we are doing something right. We are making biology accessible – and enjoyable – for the masses, or we would not see these results.
Secondarily, my class averages are way up! No, I have not started writing easier tests (a claim of my colleagues)! I have given the students a vast toolbox from which to work. You don’t use a hammer when you need calipers. The students shouldn’t be forced to only take notes in class when they have difficulties multitasking. “One and done” doesn’t work for most of us! My students have at least five options for learning any given topic – my face-to-face lecture, my lecture outlines posted online, MasteringBio (assignments and study area), my voice-over video lectures, and their textbook. This doesn’t include all of the review materials that I post, review sessions, office hours, or tutors/TAs. The students need to find the approach to the materials that works best for them. Yes, they may struggle to find what works, but if they don’t have options, how can they ever find their path to success?
Q. Would you like to add anything else?
Sandra: So often we are lulled into the notion that we must try everything new, shiny, and the latest buzzword in education. Stop it! Yes, some of them may work – some of them may not – but they are certainly not going to work for everyone, every day, in every class, for every subject, or for every student. You are allowed to be picky about what you use in your class, you are allowed to say “NO!” to the latest and “greatest” thing in education, and you are allowed to say “Let me see how this works in someone else’s class first.” You have permission! Let me know if you need me to send you to school with a note saying so!
If you want to learn more about collecting data using MasteringBiology, watch the recorded webinar presented by Sandra Connelly.
About Sandra Connelly, Ph.D.
Dr. Sandra J. Connelly joined RIT as a lecturer in 2007 and became an assistant professor in 2009. She earned her BS in biology/forensic science at Juniata College, MS in ecology and evolutionary biology from the University of Buffalo, and Ph.D. in zoology from Miami University of Ohio.
Every semester Connelly teaches foundational biology lectures in the traditional, flipped, and online formats, engaging up to 450 students with diverse interests from all colleges at RIT, freshmen through seniors. Bringing her research into the classroom, Connelly also offers a course entitled “Molecular Ecology”, co-instructed with Dr. Andre’ Hudson, and she teaches “Evolutionary Biology” with the core evolution team. She is designing a course entitled “Forensic Ecology” with Criminal Justice faculty for students interested in applying their knowledge of criminology and environment to a real-world scenario that will be offered in the coming semesters.