My past few blog posts have been concerned with the usefulness of peer grading. Steve Joordens showed that peer grading was pedagogically useful for first-year psych students…but what about computer science students? Would they learn from it? Would they become better programmers?
We don’t know.
Maybe it’s time to find out.
It’s pretty simple, actually.
I have two groups of students. Let’s call them groups A and B.
For each student in A, have them complete a simple programming assignment (call it P1). Once they’re finished, have them complete a second simple programming assignment (call it P2).
For each student in B, have them complete P1. Once that’s done, have them view 5 or 6 different mocked up submissions, also for P1. For each submission, have the students fill out a rubric and assign a grade. Once finished, the students then complete P2.
Then, I get some fellow graduate students to mark my mocked up submissions, the group A P1/P2 submissions, and the group B P1/P2 submissions.
If grading made the students better programmers, we should see an increase in the number of marks given to the students in group B for P2.
Bonuses, and Other Concerns
This experiment is nice and simple. And, besides showing if peer grading makes students better programmers, it gives us a couple of bonuses:
- It tells us if graduate students tend to agree on what marks to give to submissions. If they don’t agree, and the marks wildly differ…we might have a problem
- It tells us if some number of students can, on average, approximate the grade a TA would give on a submission
- It can tell us the average inspection rate for both students and TAs
I’ll have to do randomization here and there to eliminate ordering effects – for example, randomizing the criteria on the rubric, randomizing which assignments go first and second, randomizing the order in which the mock-up submissions are shown, etc.
One thing to consider though: what effect does simply seeing the rubric have on students?
I’ve been in courses where I’ve not been allowed to see the marking rubric for some assignment. It’s frustrating. Seeing the rubric helps me focus on the areas that I’ll be marked on.
So what if just seeing the rubric makes the students “better programmers”? One way to counteract this would be to have the rubric for the second assignment be quite a bit different than the one for the first assignment.
Oh yeah. Stats. Not my strongest subject. I’m going to have to brush up on this (and probably enlist some help within the department) if I’m going to do this properly. I’m probably not going to get as many participants as I think I will…so I have to accommodate small N. Hrm.
Anyhow, this is where my summer experiment seems to heading. What do you think? I’m all ears.