Mike Conley's Blog

Last Friday, I went to the FSOSS conference at Seneca@York Campus with Zuzel and Greg.

One of the talks I attended was an open discussion about getting students involved in open source software.

I’m going off of memory there, but I believe one of the speakers at that talk said something like:

Students generally don’t have to deal with large code-bases in their school assignments…1000 lines of code is really nothing.  When students work on an open source project, they get dropped into a massive code-base with only a fork, a spoon, and a compass.  They have to find their way around, and that’s where the real challenge and learning is.  This is a skill that most students just don’t get with normal school assignments.

Again, I’m paraphrasing.

So is this true?  Hm.

During my undergraduate career, I’ve certainly had to explore strange code that someone else has written.  But nothing even close to the size of, say, the Mozilla Firefox code-base.  Or the Chromium code-base.  I mean, these are massive wads of code.  This is not a criticism of my teachers or the UofT CS program by any means – it’s just an observation.

But some students explore these large code-bases on their own in their free time.  During my (admittedly brief) break before summer work began, I started poking around the Firefox code.  I made two discoveries:

1. The code that I saw was, in my opinion, very well written
2. I was completely lost, and didn’t know where to start

I still haven’t worked on any software that is nearly as large as Firefox.  Not even close.  MarkUs is a nice chunk of code, but minuscule in comparison.

So just go with me on this for a second.  Let’s assume that a large code base is intimidating and difficult for students to wrap their heads around, and this is one of the main challenges in getting those students to contribute to open source software.

Again, I only have my own experience to back up that claim.  Looking at Firefox, I didn’t know where to start.  I didn’t know where to go.  I didn’t know which way was up.  I was lost.

So how can students get a better grasp on a mountain of code?  A few ideas:

1. Write tests for the code, starting small and going big.  This is a relatively easy way to play with the code without having to change it. This assumes, of course, that the software has been designed to be easy to test…
2. Ask someone else.  Go into the appropriate IRC channel and ask around.  This, of course, has it’s own problems.
3. Read up on the developer documentation.  Let’s just hope it’s up to date and relevant…
4. Read up on someone else’s experiences exploring the same code base.  Good luck finding those.

Not Quite Blueprints

I’ve always thought of computer software as being like an invisible machine inside my computer.

And to me, the source code is a bit like the description of the blueprints for that invisible machine.  It’s not the top-down crystal-clear cutaways that a blueprint provides…it’s a flat, textual interpretation of those blueprints.  And it takes quite a bit of reading before those descriptions sink in, and the “personality” of the machine becomes clearer.

In his article “Code Spelunking Redux“, George V. Neville-Neil says:

Working in this way is a bit like trying to understand the United States by staring at a street sign in New York City. The ability to look at a high-level representation of the underlying system without the fine details would be perhaps the best tool for the code spelunker. Being able to think of software as a map that can be navigated in different ways—for example, by class relations and call graphs—would make code spelunkers far more productive.

I was thinking a lot about that on my ride home from FSOSS.  When I got home to my computer, I found out that there are some really cool alternative ways of viewing software.  Here are three that I found quite interesting:

Codecity

Imagine that you’re curious about developing on Firefox.  You can wade through the swaths of source code…

or you can stroll through a city that represents the software:

CodeCity

CodeCity is an integrated environment for software analysis, in which software systems are visualized as interactive, navigable 3D cities. The classes are represented as buildings in the city, while the packages are depicted as the districts in which the buildings reside. The visible properties of the city artifacts depict a set of chosen software metrics, as in the polymetric views of CodeCrawler.

Imagine virtually driving around that city, hearing a guided tour through your headphones…you can walk into buildings, check out the different floors…check the plumbing.  Interesting idea.

Cocoviz

Or how about a neighbourhood…

CocoViz

CocoViz address software comprehension by a combination of visualization and audio. It uses common place metaphors (like houses) for an intuitive understanding of software structures and evolution.

For each source code entity, evolution and structural aspects are mapped to such metaphors and annotated with different audio, to represent concepts such as design erosion, code smells or evolution metrics.

The tool is used in the software evolution analysis domain but offers DB-, XML-importer and a plugin architecture to extend its use into other domains.

EvoSpaces

Another attempt at using the architecture/neighbourhood metaphor.  This one does a neat job of displaying execution traces though – check out the video demo.

These are cool ideas.

But are they useful? Are they usable?  Do they work? Could they help students get a firm grasp on a large code-base?  Can they help visualize the evolution of software?

Has anyone actually used any of these?

Remember that Firefox add-on I wrote up a while back – the one that allows you to suppress annoying alert popups?

Some updates:

• alertCheck just went public on Mozilla Add-ons after being reviewed by an add-ons editor
• As of this writing, alertCheck has 740 downloads
• As of this writing, alertCheck has 4 reviews – all positive

Not bad for my first add-on!

Remember alertCheck – that Firefox Extension I wrote a while back, that helped suppress annoying popups?

I finally decided to post it on Mozilla Addons.  Now compatible with Firefox 3.5.*.

Here you go.

So, if you recall, I was asked to write a Firefox extension that would do word counting on websites.

Originally, when I started this project, I set a goal for myself:  I copied the text from Project Gutenberg’s First Folio version of Shakespeare’s Hamlet into OpenOffice Writer, recorded the word/line/character count statistics, and set that as my projected goal for my first iteration of my extension.

But there’s a problem with this approach:  I’m supposed to be copying the behaviour of Unix’s wc, not OpenOffice Writer’s word count.  Normally, this wouldn’t be a problem – a word count is a word count, a line count is a line count, and Writer should pump out the same numbers as wc.

Not so.

In my last post, I wrote:

According to OpenOffice Writer, this text has 32230 words, 173543 characters, and 4257 lines.

However, upon passing the same text (saved in the textfile “count.txt”) through wc, I got the following output:

5302 32230 178845 count.txt

Writer and wc agree on the number of words, but disagree on the number of lines – 5302 (wc) vs 4257 (Writer).  It’s a disagreement of about a thousand lines.

Brutal.

Anyhow, I’m going to focus on wc’s approach to line counting – simply returning the number of newline characters in the file.

And guess what…it works.  For Hamlet, my extension pumps out:

Document statistics:

Word Count:  32230
Line Count:  5302
Character Count:  178845
Character Count (no spaces):  142368

Nice.

Hamlet’s just the simple case though.  There are plenty of other cases to consider, but this is a start.

Anyhow, download here.

In this version, I’m using Mozilla’s TreeWalker implementation to stitch together the page text.  So far it seems to be working alright, but if it somehow ends up falling through, I might end up using something like Andrew Trusty’s code with the jQuery library to do the text stitching.

So there it is.  Maybe I’ll keep working on this, pretty it up a bit, etc.  However, work starts on Monday, and that’ll probably take up most of my technical attention.

We’ll see though.

It didn’t take long for another Firefox extension idea to come along.

Prof. Greg Wilson recently sent me an email, saying the following:

I’d like a Firefox plugin that does ‘wc’, i.e., counts characters, words, and lines on the current web page, and displays the results in the status bar.

Cool, I thought.  No problem.  That doesn’t sound too hard.

But I’ve been mulling and chewing this around in my head, and it’s actually a harder problem than it first sounds.

“wc“, short for word-count, is a small, simple, yet extraordinarily useful Unix utility that reads in some file, and spits out the number of words, characters, and lines for that file.

So what’s the problem?  What’s so hard about coding something like this for web pages?

Well, for starters, users of this proposed extension are probably only interested in the visible, readable text on a web page.  That means filtering out all of the HTML tags, all of the JavaScript, etc.  Also, many modern web pages make use of IFRAME’s, hidden DIV’s, etc.  Not to mention, most browsers do automatic word-wrapping, which could throw off the “line” counting.  How should I treat these cases?

I certainly don’t think this is an impossible task, just harder than it first sounded.

So here’s what I’m going to do:

First, I’m going to take care of the base case.  I’m going to take care of the case where users are viewing a page of all text, with almost zero HTML.

My test page will be an “etext” copy of Shakespeare’s Hamlet (first folio), hosted by Project Gutenberg.

According to OpenOffice Writer, this text has 32230 words, 173543 characters, and 4257 lines.

So that’s my target.  I’m going to create an extension that sits as a button on the status bar.  When the button is clicked, an alert will pop up with the statistics.  If all goes well, the numbers will match.

Sure, it’s not the most elegant interface, but it’ll do for now.

I’ll post more as it comes.

So, I think I’m more or less done the extension.

Someday, when I’ve got more extension development experience under my belt, I’ll probably come back to this and fix it up.  Until then, this will have to do.

Click here to download.

If you’re interested in looking at the source, just change the file extension from “.xpi” to “.zip”, and decompress.  It’s all there.

There’s no license on this thing, no GPL, MIT, nothing.  Use it however you want.  If you find it useful though, I’d love to hear from you – send me email, post a comment, Facebook, Twitter, whichever.

Whew.  I think I’m going to reward myself with some orange sherbet.  Om nom nom…

Here’s a really annoying website to test the extension with.  I really don’t recommend that you visit it without my extension installed.

The window hops around a bit, so just double click on the location bar, and type in something like “http://www.google.ca”.  This will start up the flood of alerts, and (hopefully) you’ll be able to suppress them after the first one hits.

Here’s the site.  Visit at your own risk.

UPDATE:

I’ve moved the extension to Mozilla Addons, and added Firefox 3.5 compatibility.

I’ve updated alertCheck.xpi so that it’ll play nice with Firefox 3.0b5, and hopefully Firefox 3.1.*.  Let me know if there are any behaviour foulups, and I’ll do my best to fix them.

Wow.  I think I got it.  I’ve got a Firefox plugin that can suppress all alert() dialogs on a page if the user checks a “suppress” box on the second alert() dialog.

The trick, was not to rely on the DOMContentLoaded event to fire to do the override.  Instead, I used the DOMWillOpenModalDialog to detect the first alert().  After detection, I overrode with an alertCheck which asked the user whether or not to “suppress more dialogs”.  If the user answers in the affirmative, alert() is simply overwritten with an empty function.

Piece of cake.

A couple of issues though…

Security

In order to override the alert() function, I have to write to document.getElementById(‘content’).contentWindow.wrappedJSObject.alert.

Remember how I mentioned the distance between the Extension JavaScript, and the inline content JavaScript?  I said it felt like a security layer.

I was totally right.

Check this out. I’ll quote:

You should be aware of XPCNativeWrappers when working with untrusted content. With XPCNativeWrappers turned on (which is the default in Firefox 1.5+), your extension can safely access the DOM of the content document, but not the content JavaScript. Bypassing XPCNativeWrapper to work with content JavaScript directly can lead to security problems.

Hrmph.  So I seem to be violating some security rules here.  So maybe my approach isn’t the greatest idea.  “Mook” from irc.mozilla.net #extdev suggested looking into commonDialog.xul…but I can’t seem to wrap my head around that just yet.

Imperfections

Not sure why yet, but while I can suppress dialog floods like this:

for (i = 0; i < 10; ++i) {
  alert(i);
}

It seems to fail on this:

for (i = 0; i < 10; ++i) {
  alert(i);
  confirm(i);
}

For some reason, regardless of whether or not I choose to suppress the dialogs, they just keep coming.  It works fine when I swap out the confirm() for a second alert().  Not exactly sure why.  Yet.

Ok, so I’m going to clean the code up, and post it soon.  I’ll also post a link to a real, brutally annoying website where you can test the alertCheck extension.  Just give me a bit.

Ok, so there have been some developments.

Before I go into this though, I just want to make it clear that I have very little knowledge or experience working with XUL, or writing Firefox extensions in general.  I’ve dabbled, but I’m mostly ignorant.  So all you Firefox aficionados out there…go easy on me.

So, developments:

Using this site, I created a skeleton for my extension, calling it “alertCheck”.

And then I hit a brick wall right away, trying to find the right “alert” to override.

Let me explain.  Mozilla extension development is strange for me because of all of the various layers to Firefox, and the distance between the JavaScript in the extension, and any page that is loaded in the browser.  It feels like a security layer – and it makes sense:  you really don’t want the Javascript on a website to monkey around with the internals of your browser.

Thankfully, this forum and this forum provided some help.

Now, after the appcontent is loaded, I wait for the DOMContentLoaded event to fire, and then write my new alert function to here:

document.getElementById('content').contentWindow.wrappedJSObject.alert

The problem is, if I wait for the DOMContentLoaded event to fire, the alert override happens (obviously) after all of the DOM Content is loaded into the browser.  This is only useful if the alert’s that we want to capture are fired using <body onLoad> or some other function that detects when the DOM has been loaded.  It’s no good for inline Javascript alerts, since these commands are parsed and executed line by line as the HTML is processed.

Phew.  Big mouthful.

So, where does that leave me?  Well, I was just hanging out in irc.mozilla.net in the #extdev channel.  Here’s a chunk of what was said:

<mike_conley> Hey all - I'm trying to write an extension to override window.alert, with an alertCheck that allows users to disable future alerts.
<mike_conley> So far, I'm able to override alert by writing to document.getElementById('content').contentWindow.wrappedJSObject.alert
<mike_conley> however, now it's a matter of timing - I'm doing the override after the DOMContentLoaded event is fired
<mike_conley> But this means that it doesn't catch alerts that are fired using inline javascript.
<mike_conley> So my question is, when should I do the override?
<Mook> I suspect just overriding commonDialog.xul is easier
<Mook> trying to poke the content JS's definition is going to be full of pain
<mike_conley> Cool, I'll look into that - thanks!
<Mook> (the content JS starts around http://mxr.mozilla.org/mozilla/source/dom/src/base/nsGlobalWindow.cpp#4022 and goes the the prompt service which pokes that xul)
<mike_conley> Excellent - I appreciate it.

So maybe there’s another approach that I should be taking – I’m going to look into overriding commonDialog.xul… I’ll write more when I have it.

When I don’t have work to do, I get antsy.

And right now, I’ve got no work to do.

So I’ve come up with a project for myself:  remember how I created a Firefox Plugin a few months back?  I’d like to make another one – but this one will actually serve a useful purpose.

Have you ever been to a page that suddenly started spewing window.alert boxes at you?

If you haven’t, open up Firebug, and paste this into the console:

for(i = 0; i < 10; ++i){ alert(i); }

Now imagine if instead of 10 alert boxes, it spewed hundreds…or thousands….or god forbid, it uses a while(true) loop, and throws infinity alert boxes at you.

It totally cripples Firefox. It’s a super simple browser DoS attack.

Mozilla knows this, but so far, no solution except for killing the Firefox process, or disabling Javascript manually, or with NoScript (a plugin that I highly recommend).

Google Chrome has solved this problem by providing a checkbox on alert dialogs that allow a user to disable future popups from the current site.

Cool.  I want Firefox to have the same feature.

So, this summer, I’m going to try to build a Firefox Plugin that will override the standard window.alert function, with one that provides a checkbox, letting the user disable future alerts.

I don’t even know if this is possible, but I’m looking into it.

I’ll blog my research and progress as I go along, and share my code / final plugin when it’s all finished (or when I abandon it…hey, it happens).

So stay tuned.

It’s a race.

I’m going to attempt to create a simple Firefox extension that will display the DOM ID of an element that my mouse cursor is hovering over in the status bar.

There are probably a ton of Firefox extensions that will do that already, but I want to give it a shot as a project.

It’s 3:30PM right now, and I want to try to get this done by 5:00PM.  I’m going to be using Ubuntu 8.04, gEdit, and Google to get me started.

And I’m going to record my progress here in this blog post.

Note: After I’m done, I’m going to edit my sporadic notes so that they make more sense.  So if you’re wondering just how I managed to stay so cool, calm, and collected in my prose under such time pressure, and why the publish date on this article is after 5PM, now you know.

3:35PM:

Gonna start with Google:  “building a firefox extension”

Ok, found an article about how to create a Firefox extension.

Apparently, the first thing I want to do is try setting up a development profile in Firefox.

3:41PM

Finished setting up my dev profile by opening up FF with this command:

firefox -no-remote -P

Then created a profile called “Development”.  After that, I typed “about:config” in the URL bar, and changed some settings as instructed on this site.

3:49PM

According to that last article, I can create a skeleton Extension project using this site.  Done – calling the project DOM ID Displayer

3:53PM

Installed this Extension - apparently, it’ll be some help.  Will let me reload Firefox’s chrome  shtuff without restarting the browser each time.  Useful.

3:55PM

Found this article on making a status bar extension in XUL.  Easy as pie.

4:05PM

Ok, I’ve coded something in XUL that should display a new panel in the status bar.

<?xml version="1.0" encoding="UTF-8"?>
<?xml-stylesheet href="chrome://domiddisplayer/skin/overlay.css" type="text/css"?>
<!DOCTYPE overlay SYSTEM "chrome://domiddisplayer/locale/domiddisplayer.dtd">

<overlay id="domiddisplayer-overlay" xmlns="http://www.mozilla.org/keymaster/gatekeeper/there.is.only.xul">
<script src="overlay.js"/>

<!-- Firefox -->
<statusbar id="status-bar">
  <statusbarpanel id="domiddisplayer" label="Hello, World!" tooltiptext="Dom ID Displayer" />
</statusbar>
</overlay>

I put that in the “overlay.js” file in my ~/Experiments/Extensions/domidinspector/content folder that was created using that Wizard from 3:49.

Now, to get this thing to run in my Development profile, I create a symbolic link to it in the Development profile’s extensions directory

ln -s ~/Projects/Experiments/Extensions/domidinspector ~/.mozilla/firefox/nzuzbdpz.Development/extensions/domiddisplayer@mike.conley

Open Firefox with Development profile:

firefox -P Development &

4:16PM

Looks like I can access and relabel the XULElement that I’ve ID’d as “domiddisplayer” using this:

domiddisplayer.updateDisplay = function(event) {
var dom_element_id = event.relatedTarget.id;
document.getElementById('domiddisplayer').setAttribute('label', dom_element_id);
}

Cool – I can now change my text in the Firefox status window.  Now I just need to capture any time a mouse moves over a DOM element….yikes, that might be tricky.

4:33PM

Been tinkering with this as a way of putting a mouseover event listener on everything in the window:

window.addEventListener("mouseover", function(e) {
  domiddisplayer.updateDisplay(e);
}, false);

4:35PM

Seems to only be capturing mouseover/mouseout events on Chrome elements – so I can get the ID’s of the statusbar, etc.  These are XUL Elements, not the DOM elements of a web page…

So I’m close.

4:44PM

This page is super helpful…

Apparently, I need to wait for the content of the page to load before I can attach observers to all of its sub-elements.  Makes total sense.

So, in my domiddisplayer.onLoad function, I write this:

var appcontent = document.getElementById("appcontent");
if(appcontent)
  appcontent.addEventListener("DOMContentLoaded", domiddisplayer.onPageLoad, true);

And now, I create a function called onPageLoad, which looks like this:

domiddisplayer.onPageLoad = function(aEvent) {
//Runs when the page is loaded
  window.content.document.addEventListener("mouseover",
  function(e) {
    domiddisplayer.updateDisplay(e);
  }, false);
}

5:03PM

Done.  I’m over time, but I’ve finished a (relatively) working extension.

Here, it’s a mess, but you can download the whole thing right here if you want to tinker with what I did.

Download domiddisplayer.zip