Tag Archives: game

Turning Peer Code Review into a Game

A little while back, I wrote about an idea that a few of us had been bouncing around:  peer code review achievements.

It started out as a bit of Twitter fun – but now it has evolved, and actually become a contender for my Masters research.

So I’ve been reading up on reputation and achievement systems, and it’s been keeping me up at night.  I’ve been tossing and turning, trying to figure out a way of applying these concepts to something like ReviewBoard.  Is there a model that will encourage users to post review requests early and often?  Is there a model that will encourage more thorough reviews from other developers?

An idea eventually sprung to mind…

Idea 1:  2 Week Games

In a speech he gave a few years ago, Danc of The Last Garden placed the following bet:

  • If an activity can be learned…
  • If the player’s performance can be measured…
  • If the player can be rewarded or punished in a timely fashion…
  • Then any activity that meets these criteria can be turned into a game.

Let’s work off of this premise.

Modeled on the idea of a sprint or iteration, let’s say that ReviewBoard has “games” that last 2 weeks.

In a game, users score points in the following way:

  • Posting a review request that eventually gets committed gives the author 1 point
  • A review request that is given a ship-it, without a single defect found, gives the author The 1.5 Multiplier on their total points.  The 1.5 Multiplier can be stolen by another player if they post a review request that also gets a ship-it without any defects being found.
  • Any user can find/file defects on a review request
  • A defect must be “confirmed” by the author, or “withdrawn” by the defect-finder.
  • After a diff has been updated, “confirmed” defects can be “fixed”.  Each fixed defect gives the defect-finder and author 1 point each.

After two weeks, a final tally is made, achievements / badges are doled out, and the scores are reset.  A new game begins.  Users can view their point history and track their performance over time.

Granted, this game is open to cheating.  But so is Monopoly.  I can reach into the Monopoly bank and grab $500 without anybody noticing.  It’s up to me not to do that, because it invalidates the game.  In this case, cheating would only result in bad morale and a poorer piece of software.  And since scores are reset every two weeks, what’s the real incentive to cheat?

Idea 2:  Track My Performance

I’ve never built a reputation system before – but Randy Farmer and Bryce Glass have.  They’ve even written a book about it.

Just browsing through their site, I’m finding quotes that suggests that there are some potential problems with my two week game idea.  In particular, I have not considered the potentially harmful effects of displaying “points” publicly on a leader-board.

According to Farmer / Glass:

It’s still too early to speak in absolutes about the design of social-media sites, but one fact is becoming abundantly clear: ranking the members of your community-and pitting them one-against-the-other in a competitive fashion-is typically a bad idea. Like the fabled djinni of yore, leaderboards on your site promise riches (comparisons! incentives! user engagement!!) but often lead to undesired consequences.

They go into more detail here.

Ok, so let’s say that they’re right.  Then how about instead of pitting the reviewers against one another, I have the reviewers compete against themselves?

Ever played Wii Sports?  It tracks player performance on various games and displays it on a chart.  It’s really easy to see / track progress over time.  It’s also an incentive to keep performance up – because nobody wants to go below the “Pro” line.

So how about we just show users a report of their performance over fixed time intervals…with fancy jQuery charts, etc?

So what?

Are either of these ideas useful?  Would they increase the number of defects found per review request?  Would they increase the frequency and speed of reviews?  Would they improve user perception of peer code review?  Would it be ignored?  Or could it harm a team of developers?  What are the benefits and drawbacks?

If anything, it’d give ReviewBoard some ability to record metrics, which is handy when you want to show the big boss how much money you’re saving with code review.

Might be worth looking into.  Thoughts?

A Few Things Drama Can Bring to Computer Science

So, yesterday I wrote:

[W]hat can Drama bring to Computer Science?

The easy one is presentation/communication skills.  A CS student might be brilliant, but that doesn’t mean they can present or communicate.  And if an idea can’t be communicated, it’s worthless.

But what else?  Any ideas?  I’m going to think about this for a bit, and I’ll see if I can come up with any more.

I posted the question on Twitter, and on my Facebook.  I was quite surprised by the amount of feedback I got back – apparently, quite a few people are interested in this topic.

Thanks for everybody who posted, or who came up to talk to me about this!  Let me summarize what I heard back:

  • Without a doubt, work in Drama hones movement/body senses.  It also trains us to use and take care of our body, and voice, like a musician would take care of a musical instrument.  Spending too much time hunkered over a keyboard can have detrimental effects on the body over time – I can personally admit to having absolutely awful shoulder tension, no doubt to my constant typing.  I only became aware of this tension, and how to deal with it, thanks to my work in Drama.  The dichotomy between body and mind is, in my humble opinion, a Western myth, and when you stop separating them, and get them to work together, amazing things can happen.  Just ask any contact improviser.
  • Drama is also emotional work.  No, this doesn’t mean we sit in a big circle and cry, and get credit for it.  Emotions are something that we study – how to mimic them, how to summon them out of ourselves, how to describe them, and abstractly represent them.  This is where Psychology, Drama, and Human-Computer Interaction might have some overlap.  In particular, it must be remembered that theatre is a communications medium between the actor(s) on stage, and the audience.  A webpage is also a communications medium.  Perhaps the theatre can teach a website a thing or two about communication.  I wonder what Marshall McLuhan would have to say on all of this…
  • Drama folk are creative, and are used to doing impossible, unreasonable things.  If you ask them to fly, they’ll figure out a way of doing it.  It’ll probably be abstract, and involve crazy lighting effects, but they’ll do it.  Production Managers are used to getting crazy, impossible requests from Directors all the time.  In my opinion, that’s what Directors are for!  Sometimes (usually due to time constraints), the Production Manager just says no to the Director – usually, though, they just go ahead and make impossible things happen – like building a triple layered reflection box.  This thing was a beast, and used a ton of computing power for live, context sensitive visual effects. I’m proud to have been a part of that.
  • In Drama, if the project is no fun, the end result suffers.  I’m pretty sure the same goes for software.  Drama students have a way of finding the “game”, the “jeu”, and the “play” (that’s why it’s called a “play”, people!) in what they’re doing.  The best actors are the ones who are clearly having a great time on stage, and are sharing this with the audience.  I believe this is applicable to software development…
  • If you want to think about complex systems, think about the stage.  At any given moment, n actors are on stage, interacting with various bits of set or props, interacting with each other – and each has their own motivation and personal story.  It can’t be a coincidence that the I* modeling language orients itself around terms like “actors” and “goals”.  It also can’t be a coincidence that many adventure game engines refer to in-game sprites as actors…

But now I want to hit the big one.  There is one thing that I really think Drama can bring to Computer Science.  Drama students are very good at it.  From what I can tell, Computer Science students rarely get exposed to it.

That thing is collaboration skills.

I already know that a few of my fellow Drama students will laugh at that – and say, “there are plenty of people in this department without collaboration skills”.  Yes, this is true.  But they tend not to do very well, or produce anything too interesting.

For me, the best, most exciting stuff comes when I’m with a group, and we’re not sure where we’re going with a project, but we just try things. We all throw a bunch of ideas in the middle, and try to put them on their feet.  The most unexpected things can happen.

Two years ago, I took a course in Experimental Theatre.  We were broken down into groups of 3 or 4 right at the beginning of the term, and given this challenge – show us what you like to see in theatre.  Show us what you think good theatre looks like.

That was it.  A blank canvas.  No script.  No “spec”.  Just each other.  It felt hopeless at first – we’d improv things, trying to get a feel for what our group wanted to do.  Nothing would happen, it’d fall flat.  We were lost.

But slowly, something started to piece itself together.  We found some material that we wanted to play with (The Wizard of Oz), and a subject that we liked – “home”.  What it means to be home, why people leave their homes, why we miss home, why we can’t stand home, what if we can’t get home, etc.  We divided the work up into 4 sections – 1 for each of us:  Dorothy, Cowardly Lion, Scarecrow, Tin Man.

It’s really hard to describe what we did.  The characters and structure from The Wizard of Oz was just a playground for a huge meditation on what “home” meant to different people.

And, wouldn’t you know it, the Robert Dziekanski Taser Incident happened just a week or so before we were to present.  It integrated perfectly into our piece.

When we finally presented it, some people were incredulous, others nauseous, others outraged.  Some were crying.  We had a huge class debate on whether or not it was appropriate to include the film clip of the Taser Incident in our piece.

But a lot of people really got something out of it.  And I believe a bunch of people from that class went to a protest rally about the incident that took place only a few days later.  I heard a lot of really positive things.  We were so excited by it that we almost took it to the Toronto Fringe Festival.

In my opinion, that was one of the most interesting, educational, horrifying, and rewarding art pieces I’d ever been involved in.  And it all started from nothing.

When are Computer Science students grouped up, and told to make whatever they want?  When are they given total freedom to just go crazy, and come up with something beautiful?  Something unique?  When are they given the frightening prospect of a blank canvas?  Maybe I’m being naive – but where are the collaborative creativity assignments in computer science education?

Now, I can imagine someone shouting – “but what about those group assignments!  What about CSC318, or CSC301?  Those were collaborative!”.

My friend, thanks for trying, but there’s a distinct difference between group problem solving, and collaborative creation.  In my mind, for collaborative creation at its best, the ensemble starts with nothing and must create something from it.  It’s the difference between having a script to toy with, and not having a script at all.

And don’t just tell me that an independent study fits the bill.  The word “independent” sabotages the whole idea – the key word is collaborate.

Oh, and did I mention that Artful Making sounds like an excellent book? Why don’t you go to their website, and read the forward by Google’s own Dr. Eric Schmidt.  I found it very illuminating.  I think this is going to the top of my to-read list.

Thanks to Blake Winton, Veronica Wong, Cam Gorrie, Jorge Aranda, Neil Ernst, Peter Freund, Jennifer Dowding, and Yev Falkovich for their input on this.  Yes, those little conversations made an impact!