Research Question Idea #3

When we started using ReviewBoard with MarkUs a few months back, all of a sudden, commits to the repository seemed to slow down: we would take more time cleaning up our code, and polishing it for others to see.

Our commits were usually quite large too.  This is because we were all working on different sections of the code, and we wanted to commit stuff that “instantly worked” and was “instantly perfect”.  So after days of silence, 1000 lines of code would suddenly go up for review…and as Jason Cohen can probably tell you, the number of defects found during review decreases as the amount of code to look at increases.  So, the reviewer would skip through 1000 lines, assume most of it was OK, and give it the Ship It.

Yeah, I know.  Awful.  I wonder if this is a standard newbie mistake for student groups just starting out with code review…

So, study idea:

Have two separate groups working on some assignment.  Have Group 1 commit to their repository without any review process.  Have Group 2 do pre-commit reviews using a tool like ReviewBoard.

Now check out the size, frequency, and readability of the repository diffs of each group.  Might generate some interesting data.

Anyhow, in our defence, we seem to have calmed down on MarkUs.  Diffs up for review are pretty small, and get posted relatively frequently.  Using ReviewBoard on MarkUs has made me a believer.  Testify!

Research Question Idea #2

(Read this if you have no idea what I’m talking about)

Why not go right for the throat?

How about I just round up all of the instructors who teach courses with group assignments, and ask them why code review tools aren’t provided or encouraged.  Or maybe they’ve tried, but they ran into a stumbling block.  Or perhaps the whole idea of using code review tools flies in the face of some important teaching method.

I won’t know until I ask.  So why not just ask?

It might not be a quick, sharp, clever scientific study, but it sure might generate some interesting material for examination.

Coming Up with Interesting Research Questions, and Idea #1

So my supervisor Greg Wilson has challenged myself and fellow grad student Zuzel to try to come up with one idea for a study per day until our next meeting.

I’ve been researching the use of code reviews in CS undergraduate classes, and this is what my ideas will center around.

My first idea is a knock-off of one that Jorge Aranda performed a while back:

Take a group of students, and tell them that they will all be working together on an assignment. Give them a spec for their assignment.  Get a time estimate in hours as to how long they think they will need to complete the assignment.

Take a second group of students (who were not present when the first group was around), and tell them that they will be working together on an assignment.  Give them the same spec that the first group had.  Tell them that they will need to use a peer code review tool like ReviewBoard for every commit.  Get a time estimate on how long they will need to complete the assignment.

Compare the two sets of estimates.

I predict that the second set will have a higher range of values.  I wouldn’t find that surprising.  I’m more interested in how much higher the estimates are.

I remember my first reaction to using ReviewBoard on MarkUs:  This’ll slow us down.  We don’t have time for this.

I’m curious if others feel the same way.

alertCheck Grows Up

Remember that Firefox add-on I wrote up a while back – the one that allows you to suppress annoying alert popups?

Some updates:

  • alertCheck just went public on Mozilla Add-ons after being reviewed by an add-ons editor
  • As of this writing, alertCheck has 740 downloads
  • As of this writing, alertCheck has 4 reviews – all positive

Not bad for my first add-on!

Learning to Read Things, and Code as Art

A while back, I went on a little rant about how CS students don’t learn how to read and critique code, and that because of this, they’re missing out on a huge opportunity for learning.  They’re not exercising their abilities to comprehend other people’s code.

But let’s step back for a second.  Forget code for a minute, and let’s just think about reading in general.

Check out this article that Greg Wilson forwarded to Zuzel and I.

The take home message:  reading is not a skill that can be sharpened in a vacuum.  One of the keys to increasing reading comprehension is to increase the variety of reading material to practice with.  This will help build the foundation needed to understand more sophisticated material.

Actually, this really doesn’t come as a surprise to me.  In the Drama department, before diving into a new script, we are encouraged to briefly study the time period (slang, accents, and turns of phrase in particular) and the subject matter of the play.  This builds a foundation of context, so that the script makes some sense the first time it is read.  And the first time a script is read is very important, because for me,  it lays down many of the assumptions that I carry throughout my time working with the play.  Every subsequent reading of that play builds off of the first reading.

And have you ever read T.S. Elliot’s Wasteland?  Good luck making sense of that unless you have a study guide, because that guy references a ton of stuff, and expects you to fill the gaps.

So let’s direct this back to reading source code.

Taking that article (and this video) under consideration, it seems that just slapping some code up on a projector, and getting people to talk about it might not be good enough.

Something like this might be a better idea:  check out this post by Jason Montojo.  If his CS art history course ever takes off, I’ll be first in line to sign up.

The idea of studying code as art is intruiging…maybe I’m not the only one who thinks of it like sculpture.