Code Spelunking for Students

Last Friday, I went to the FSOSS conference at Seneca@York Campus with Zuzel and Greg.

One of the talks I attended was an open discussion about getting students involved in open source software.

I’m going off of memory there, but I believe one of the speakers at that talk said something like:

Students generally don’t have to deal with large code-bases in their school assignments…1000 lines of code is really nothing.  When students work on an open source project, they get dropped into a massive code-base with only a fork, a spoon, and a compass.  They have to find their way around, and that’s where the real challenge and learning is.  This is a skill that most students just don’t get with normal school assignments.

Again, I’m paraphrasing.

So is this true?  Hm.

During my undergraduate career, I’ve certainly had to explore strange code that someone else has written.  But nothing even close to the size of, say, the Mozilla Firefox code-base.  Or the Chromium code-base.  I mean, these are massive wads of code.  This is not a criticism of my teachers or the UofT CS program by any means – it’s just an observation.

But some students explore these large code-bases on their own in their free time.  During my (admittedly brief) break before summer work began, I started poking around the Firefox code.  I made two discoveries:

  1. The code that I saw was, in my opinion, very well written
  2. I was completely lost, and didn’t know where to start

I still haven’t worked on any software that is nearly as large as Firefox.  Not even close.  MarkUs is a nice chunk of code, but minuscule in comparison.

So just go with me on this for a second.  Let’s assume that a large code base is intimidating and difficult for students to wrap their heads around, and this is one of the main challenges in getting those students to contribute to open source software.

Again, I only have my own experience to back up that claim.  Looking at Firefox, I didn’t know where to start.  I didn’t know where to go.  I didn’t know which way was up.  I was lost.

So how can students get a better grasp on a mountain of code?  A few ideas:

  1. Write tests for the code, starting small and going big.  This is a relatively easy way to play with the code without having to change it. This assumes, of course, that the software has been designed to be easy to test…
  2. Ask someone else.  Go into the appropriate IRC channel and ask around.  This, of course, has it’s own problems.
  3. Read up on the developer documentation.  Let’s just hope it’s up to date and relevant…
  4. Read up on someone else’s experiences exploring the same code base.  Good luck finding those.

Not Quite Blueprints

I’ve always thought of computer software as being like an invisible machine inside my computer.

And to me, the source code is a bit like the description of the blueprints for that invisible machine.  It’s not the top-down crystal-clear cutaways that a blueprint provides…it’s a flat, textual interpretation of those blueprints.  And it takes quite a bit of reading before those descriptions sink in, and the “personality” of the machine becomes clearer.

In his article “Code Spelunking Redux“, George V. Neville-Neil says:

Working in this way is a bit like trying to understand the United States by staring at a street sign in New York City. The ability to look at a high-level representation of the underlying system without the fine details would be perhaps the best tool for the code spelunker. Being able to think of software as a map that can be navigated in different ways—for example, by class relations and call graphs—would make code spelunkers far more productive.

I was thinking a lot about that on my ride home from FSOSS.  When I got home to my computer, I found out that there are some really cool alternative ways of viewing software.  Here are three that I found quite interesting:

Codecity

Imagine that you’re curious about developing on Firefox.  You can wade through the swaths of source code…

or you can stroll through a city that represents the software:

CodeCity

CodeCity

CodeCity is an integrated environment for software analysis, in which software systems are visualized as interactive, navigable 3D cities. The classes are represented as buildings in the city, while the packages are depicted as the districts in which the buildings reside. The visible properties of the city artifacts depict a set of chosen software metrics, as in the polymetric views of CodeCrawler.

Imagine virtually driving around that city, hearing a guided tour through your headphones…you can walk into buildings, check out the different floors…check the plumbing.  Interesting idea.

Cocoviz

Or how about a neighbourhood…

CocoViz

CocoViz

CocoViz address software comprehension by a combination of visualization and audio. It uses common place metaphors (like houses) for an intuitive understanding of software structures and evolution.

For each source code entity, evolution and structural aspects are mapped to such metaphors and annotated with different audio, to represent concepts such as design erosion, code smells or evolution metrics.

The tool is used in the software evolution analysis domain but offers DB-, XML-importer and a plugin architecture to extend its use into other domains.

EvoSpaces

Another attempt at using the architecture/neighbourhood metaphor.  This one does a neat job of displaying execution traces though – check out the video demo.

These are cool ideas.

But are they useful? Are they usable?  Do they work? Could they help students get a firm grasp on a large code-base?  Can they help visualize the evolution of software?

Has anyone actually used any of these?

Augmenting Code Review Tools: Screen Age and Retina Burn…

Two more ideas to augment the code review process:

Screen Age

Imagine you’re reviewing a piece of code.  There’s something like…500 lines to look at.  So you’re scrolling through the diff, checking things here, checking things there…

And in the background, the review software is recording which sections of the code you’re looking at.

Hm.

I wonder if it’d be a useful metric to know what portions of the code were being looked at the most during a review?  I believe it’s possible (at least within Firefox) to determine the scroll position of a window.  I believe it’s also possible to determine the dimensions of the entire diff page, and the offset coordinates of elements on that page.

So shouldn’t it be possible to determine which elements were on the screen, and for how long?

Imagine seeing a “heat map” over the diff, showing where the reviewer spent most of their time looking.

Jason Cohen wrote that code reviews should take at least 5 minutes, and at most 60-90 minutes.  I wonder if collecting this information would help determine whether or not a review was performed carefully enough.

Now, granted, there are plenty of ways to add noise to the data.  If I’m doing a review, and stand up to go get a sandwich, and then my neighbour visits, etc…my computer sits there, gazing at those elements, and the data is more or less worthless…

Which brings me to:

Retina Burn

Eye tracking is not a new technology.  One of my roommates works at a lab where they do experiments with eye tracking tools.  From what he tells me, the eye tracking gear in the lab is pretty expensive and heavy-weight.

But check this out:

and this:

and this:

So it looks like a webcam and some clever software can do the eye tracking trick too.  This stuff is probably far less accurate than what my roommate uses – but it’s lighter, and cheaper, and therefore more likely to find its way into homes.  So it looks like the day is coming where I’ll eventually be able to use my eyes to control my mouse cursor.

But, more interestingly, this more or less solves the problem with my Screen Age idea:  this technology can tell when you’re looking at the screen.  And it can give a pretty good guess about what area of the screen you’re looking at.

I wonder if collecting this information from code reviewers would be useful – what exact parts of the code have they looked at?  And for how long?  What have they missed?  What did they gloss over?

UPDATE: Karen Reid has also brought to my attention the possibility of using this technology to see how TAs grade assignments by tracking what they’re looking at.  Hm…

Limits of the i>Clicker

This post is from an idea that Karen Reid has posed to me for potential research…

i>Clickers have been around for a few years.  I’ve never had to buy or use one in any of my classes, but it seems like more and more courses are starting to find it useful.

So what is this i>Clicker thing?

An i>Clicker is a handheld wireless device that essentially brings the “ask the audience” portion from Who Wants to Be A Millionaire, into the classroom.

So, each student buys his or her own personal i>Clicker, and registers it for any classes that require it.  During one of those classes, the instructor could throw up a slide that quizzes the students on what was just taught.  Students key in their responses on the i>Clicker, and the results are then displayed up on the screen.

From what I can tell, the idea is that the i>Clicker should encourage more class participation because:

  1. Students answer simultaneously – so instead of instructors choosing a raised hand from the class, the entire class gets polled
  2. Results displayed to the class can be anonymous.  So, instead of remaining silent among your peers out of fear of being publicly wrong, all students can submit an answer, and get feedback that helps them learn

The i>Clicker can also be used to poll students and give the instructor feedback. For example, an instructor could put up a slide that says “How was my lecture today?” and get some anonymous feedback there.

Well, not exactly anonymous.  See, the instructor has the ability to see who submitted what, and when…so if you repeatedly answer quiz questions incorrectly, the instructor can probably detect that you’re misunderstanding, guessing, or just don’t care.

Anyhow, that’s the basic idea behind the i>Clicker.  It’s used in a few classes here at UofT, and I know people who’ve had to purchase ($35+) and use them.

Click here to visit the i>Clicker website

The Limits of the i>Clicker

The amount of data that students can provide through the i>Clicker is pretty limited.  Here’s a photo of a the device:

The iClicker.

Ta da.

Students have a maximum of 5 choices that they can make while being polled.  Instructors are restricted to multiple-choice questions.

Hm.  Can’t we do any better?

Turning Smartphones into an i>Clicking Device

Wifi-enabled “smartphones” are becoming part of everyday life.  It seems like I can’t walk half a block without seeing somebody whip out their iPhone and do something really freakin’ cool with it:

So it’s not really a far fetched idea to imagine that, some day, every student will possess one of these things.

Certainly, something like the iPhone could act as a multiple choice interface.  But is there a way of turning some of that cool touch/gesture/accelerometer stuff into useful polling feedback for students and instructors?

Some Ideas

  1. Instructor puts a graphic up on the board, and asks the students “what’s wrong with this picture?”.  Students look at the picture on their SmartPhone, and use their finger to indicate the portion of the picture that they’re interested in.  After a few seconds, the instructor displays the results – which is a semi-transparent overlay on the image, showing all of the areas that students indicated.  Areas that are of interest to more students are emphasized.I can see this being useful for code reading classes.  The instructor splats a piece of code up on the screen, and asks the students to indicate where the bug is.
  2. Students were asked to mock-up a paper prototype for an interface that they are designing.  The instructor asks all of the students to take a picture of their paper prototype, and submit it on their SmartPhone.  The instructor is then able to put all of the photos up on the screen for discussion.  This could nicely tie in with idea #1.
  3. Students are polled to see how many years they have been programming for.  Students simply type in the number of years, and submit it.  While the i>Clicker restricts the answer to such a question to 5 ranges, the SmartPhone submits the actual answer.  Once collected, the submissions could be displayed on a histogram to give students an accurate impression of the level of experience in the classroom.

Any other ideas?

Code Reviews and Predictive Impact Analysis

A few posts ago, I mentioned what I think of as the Achilles’ Heel of light-weight code review:  the lack of feedback over dependencies that can/will be impacted by a posted change.  I believe this lack of feedback can potentially give software developers the false impression that proposed code is sound, and thus allow bugs to slip through the review process.  This has happened more than once with the MarkUs project, where we are using ReviewBoard.

Wouldn’t it be nice…

Imagine that you’re working on a “Library” Rails project.  Imagine that you’re about to make an update to the Book model within the MVC framework:  you’ve added a more efficient static class method to the Book model that allows you to check out large quantities of Books from the Library all at once, rather than one at a time.  Cool.  You update the BookController to use the new method, run your regression tests (which are admittedly incomplete, and pass with flying colours), and post your code for review.

Your code review tool takes the change you’re suggesting, and notices a trend:  in the past, when the “checkout” methods in the Book model have been updated, the BookController is usually changed, and a particular area in en.yml locale file is usually updated too.  The code review tool notices that in this latest change, nothing has been changed in en.yml.

The code review tool raises its eyebrow.  “I wonder if they forgot something…”, it ponders.

Now imagine that someone logs in to review the code.  Along with the proposed changes, the code review tool suggests that the reviewer also take a peek at en.yml just in case the submitter has missed something.  The reviewer notices that, yes, a translation string for an error message in en.yml no longer makes sense with the new method.  The reviewer writes a comment about it, and submits the review.

The reviewee looks at the review and goes, “Of course!  How could I forget that?”, and updates the en.yml before updating the diff under review.

Hm.  It’s like a recommendation engine for code reviews…”by the way, have you checked…?”

I wonder if this would be useful…

Mining Repositories for Predicting Impact Analysis

This area of research is really new to me, so bear with me as I stumble through it.

It seems like it should be possible to predict what methods/files have dependencies on other files based on static analysis, as well as VCS repository mining.  I believe this has been tried in various forms.

But I don’t think anything like this has been integrated into a code review tool.  Certainly not any of the ones that I listed earlier.

I wonder if such a tool would be accurate…  and, again, would it be useful?  Could it help catch more of the bugs that the standard light-weight code review process misses?

Thoughts?

Teaching Your Dog Tricks Through a Glass Window: The 10/Gui Interface as an Adventure Game Controller

Adventure game developers have tried a bunch of ways of making it easy to control characters in 3d environments.  Myst stuck with the basic point and click.  Grim Fandango used the numpad on the keyboard.  Gabriel Knight 3 used both the mouse and keyboard:  the mouse moved the player, and the keyboard controlled the camera.

With all of these…it’s always felt a little bit…restrictive.

A few weeks ago, Ben Croshaw reviewed the new Monkey Island game.  Here’s the review.  Pay particular attention to 1:50 when he talks about the mouse controlling a character in a 3d environment.

NSFW (language):

That’s exactly right.  He’s hit the nail on the head.  It’s like trying to teach your dog something through a glass window.

The 2D interface of a mouse or keyboard makes actions in a 3d world awkward.

Multi-touch improves on 2d interfaces by giving us more bandwidth (multiple fingers = multiple cursors).  If this is the next step in desktop computing:

What does that mean for 3d adventure games?  How can 3d adventure games leverage multi-touch?

For that matter, how can 2d adventure games leverage multi-touch?  Think of all of the puzzles…