Author Archives: Mike Conley

MoMo All-Hands. In Hawaii.

It’s been a little while since I posted.  Well, I’ve been busy.

In Hawaii.

That’s right.  Hawaii.  Mozilla Messaging just sent the entire team to Hawaii for an all-hands meeting.  And, believe it or not, we got a hell of a lot done.  It’s amazing how productive people can be in shorts, Hawaiian shirts, and sandals!  No joke!

It was also an opportunity for me to meet my new teammates.  It’s a fantastic group, and a very warm welcome.  They’re smart, committed, quirky, and hilarious.  I think I’m really going to enjoy working with this team.

Anyhow, I had a great time, and learned a lot.

And I took notes.

Here’s part 1…

This is my last “goodbye school” post, I swear.

So a few days ago, my official grades for my M. Sc. rolled in.  That same day, I went to the Bahen Centre, turned in my desk keys, got my keycard authorization revoked, and scheduled my computer for erasure.

It felt like some pretty big steps.  There was a palpable sense of finality.  I was out.  It was over.

The University has played a big role in my development, and despite all of my moaning and complaining over the years, I’m glad I went, and I’d do it again.

But not right now.

Graduate school almost didn’t happen for me, and I have two very important people to thank for making that happen:  Karen Reid and Greg Wilson.

I still fondly remember when you cornered me during that codesprint in 2009, and convinced me to try graduate school.  I don’t regret it.  It was the right decision.  So thank you both so much for convincing me, and giving me the chance, and thank you Greg for supervising, and guiding me through.

I learned lots.  I had fun.  🙂

Shakespeare wrote that brevity is the soul of wit.  Well, I

Day 3: Stream of consciousness

So, I just started my day off with a big bowl of Apple Cinnamon Cheerios and some fresh orange juice, c/o Mozilla Messaging.  Thanks, team.  😀

Another great thing about Mozilla is that, since its spread out around the world, somebody is awake and working pretty much 24 hours a day.  That means if I write a bunch of code and go to bed, there’s a chance that when I wake up the next morning, the code review is done and I have some feedback on my next steps.

And that’s more or less what’s happened at the start of today.  That add-ons manager grouping feature I was working on for Firefox got looked at in the night, and I got some feedback on some changes I can make.  Awesome.

So here’s the scoop:

First off, a conversation got started regarding where add-ons with pending installs or uninstalls go.  The answer:  pending installs should go into the group they’ll be in once the install is complete, and pending uninstalls should stay in the group that they were in when the install happened (in order to prevent the add-on from jumping around in the list).  That’s good – that makes my job easier, I think.

With regards to implementation, I don’t think my nifty Shwarzian Transform is gonna fly:

I don’t think that this is the right approach to take here. Instead the more straightforward way is to just make sortElements accept an array for aSortBy (and update all callers to pass one) of fields to sort by in order of preference. If the elements match by the first field then move onto the second and so on. For the non-search list views use a specially named field “uiState” and then use your function as the comparison function for that field.

(from one of my reviewers)

So this approach involves me changing a bit more code.  See, my original approach was to try to change as little as possible.  I guess that’s just me being the “new guy”, and trying not to rock the boat.  But clearly, they want me to go deeper.  I’m happy to oblige!

But there’s a bit more complication.  According to the Bugzilla page, this bug I’m working on depends on this other bug, where somebody else is also tinkering with the sorting functions.  That means I have to be uuber careful, and make sure to base my work off of their patches.

In particular, it looks like this patch is going to be altering the sorting tests, and removing the ability for Firefox users to sort add-ons by anything besides the add-on name.

So what I’m going to do is download and apply this patch, and then start basing my work off of it.  Each time the patch is updated in Bug 623207, I’ll just re-base my work off of it.  Nice.

Ok, so first of all, I wipe out my old work using hg strip (using Blake’s handy script to find the HEAD revision number of that branch).  Next, I grab the patch for 623207, and use patch to apply it.  Then, I use pnew to create a new pbranch with that change, and then create another pbranch on top of that.  That second pbranch is where I’ll do my work.  If/when the patch to 623207 gets updated, I’ll update the first pbranch, and then merge it into the second.  Awesome-sauce.

Argh.  It seems I have to recompile in order to get that 623207 patch to work.  And not an incremental compile either, since “make” doesn’t seem to do much in toolkit/mozapps/extensions.  *sigh*…compiling…

Just stopped by UofT to drop off my old keys, and get my old computer wiped.  Stopped by and talked to Karen, and helped two new MarkUs students.  Kind of bitter sweet moment.  Goodbye school.  It was a long battle.  Well played.

I’ve finished a draft of my add-on grouper/stratifier.  Now I’m going to check out these tests.  Without my change, the tests in browser_sorting.js all pass.  With my patch, 3 fail.  Ok, so I think I’ve found the right tests.  Lets see whats up…

Ok, it looks like the tests were trying to call my new sorter, and didn’t know that it had to pass an Array instead of a string.  Fixed that, and all tests are passing.  Sweet.

Now let me try all of the extension tests…ok, without my patch, 10 fail.  With my patch… OH SHIT.  194 failures.  That’s a big deal.

Ok, I think I’ve fixed those tests.  But now when I run the extension tests, one of the tests seems to take forever…what’s the deal?  Turns out, this test does this periodically, even without my patch.  Hrm. So I guess I don’t have to worry about it.

After a few runs, I’ve got the same number of passing tests as there were before my patch:  only 10 fail.  Nice.

So now I have to try to write some new tests.

Suite!  Tests written.

It takes me a while to run the Firefox Mochitests on my machine.  If I were to do them all, it’d probably take at least an hour.  Running the add-ons manager tests takes about 5 minutes.  And in either case, I can’t use my machine, because Mochitest needs me to keep focus on Firefox while its running the tests.  Basically, this means if I want to run the tests, I give up my machine, and go snack on something in the kitchen.  That was cool at first, but after a while, giving up my machine for 5 minutes seemed pretty lame.

So, luckily, there’s a machine in the office called TheFlash, and, as its name suggests, it’s SUPER fast.  Like, lightning speed.  Compiling Thunderbird from scratch?  7 minutes flat.  Jeebus.  Anyhow, Blake got me an account on TheFlash, and I’ve pushed my changes to a Firefox instance over there.  Firefox is compiling, and then I’ll try out my tests over there.  Awesome.

Tests pass!  Lovely.  And Blake just took a look at my code and showed me a neat trick:

So I’ve got an Array of uiState’s, like so:

const UISTATE_ORDER = ["enabled", "incompatible", "disabled", "blocked"]

And I wanted to sort a collection of these values in order that they appear in that Array. So something like:

["disabled", "disabled", "blocked", "incompatible", "disabled", "enabled", "enabled", "blocked", "disabled"]

# Would become

["enabled", "enabled", "incompatible", "disabled", "disabled", "disabled", "disabled", "blocked", "blocked"]

(this sort of thing is useful if each of those original entries is associated with something like, I don’t know, a Firefox add-on…)

In Javascript, we use a sort command, and we pass a function to do comparisons.  Normally, I’d do something like:

function uiStateCompare(a, b) {
 if(UISTATE_ORDER.indexOf(a) < UISTATE_ORDER.indexOf(b))
   return -1;
 if(UISTATE_ORDER.indexOf(a) > UISTATE_ORDER.indexOf(b))
   return 1;
 return 0;
}

But there’s a more concise way to say this:

function uiStateCompare(a, b) {
  return (UISTATE_ORDER.indexOf(a) - UISTATE_ORDER.indexOf(b));
}

Which makes total sense.  If UISTATE_ORDER.indexOf(a) < UISTATE_ORDER.indexOf(b), then of course the difference is less than 1.  Anyhow, I thought this was a pretty neat trick.  Thanks Blake.

Alright, patch is scrubbed and ready for posting on Bugzilla….here goes!

Ok, patch posted.  Home time.

Day 2 at Mozilla Messaging: pbranch, testing, teleconferencing, intranet, and more testing

Ok, today was my second day at Mozilla Messaging.  Another good day.  Here are some highlights:

Today, I started off my day by wanting to learn a few things:

  1. How to use pbranch to locally commit my Firefox patch from yesterday
  2. How to write tests for my patch using Mochitest

I started with the first one.

So, for the most part, Mozilla uses Mercurial as its distributed version control system.  I’ve been using Git (arguably Mercurial’s main competitor) since last summer with both MarkUs and Review Board.  Mercurial is something quite different.  Quite different indeed.

pbranch is a tool that lets me have a patch queue.  Basically, organization, and re-organization of any changes I make to the Mozilla code-base is a lot easier using something like pbranch.

So I spent a few minutes going through the pbranch tutorial.  Eventually, I think I got the hang of it – basically, for my extension changes-sets, I create a new branch using hg pnew, and commit to that branch.  I’ll keep committing to that, and when I’m all done, I’ll dump my patch to a Bugzilla attachment.  After I pass code review, someone will merge my patch.  Then I’ll remove my local branch and pull in my changes.  Sweet!

Ok, so at this point, I think I got the workflow.  Next, I needed to figure out how to write a Mochitest.  Thankfully, there’s this tutorial.

Looking through the documentation, I was reminded of Selenium a little bit.  I think it’s sort of the same idea.

So how do I write a test for my changes?  Unfortunately, the documentation on how to write a Mochitest is a little thin.  So I started hunting around, looking for examples to extrapolate from.

At some point, I found myself staring at this code.  Wow!  A full-blown API for manipulating the add-ons manager!  Great!  But it turns out that this is for Mozmill tests, and not Mochitests.  But a quick search through MXR showed that nobody was using that add-on manager API.  Argh.

Was I barking up the wrong tree here?  Where the hell were the add-on manager tests?

I quickly swallowed my pride, and decided to talk to an expert.  I used Mercurial’s log function to determine who had changed extensions.js the most.  The name Dave Townsend came up.  According to his site, he’s “Mossop” on IRC, but he wasn’t online.  The log function also mentioned the name Blair McBride.  On IRC, he’s “Unfocused”.  He was online, but unavailable.

Argh.

It was at this point that Blake told me that the Mozilla Messaging weekly meeting was going to take place.  Apparently this happens every Tuesday at 9:30AM PST.  So we marched over to a conference room, hooked up this super-advanced phone (the phone had a boot-up screen, and then showed the Mozilla logo…whoa!).  A little while later, the meeting began.  The meeting was super fast, and super efficient – especially considering the teams are spread out across the globe.  One person led the meeting, and called the different teams up to give their weekly status.  I also got to introduce myself to the team.  I rambled off something about Thunderbird+Unity and code review, and then stumbled back to my chair.  Cool times.  Anyhow, teleconferencing is going to take some getting used to.

So, with the meeting over, and still no word from Unfocused, I decided to clean up my code a bit, and then posted my patch up on Bugzilla.  I asked Dave Townsend for a code review, and said that if testing needs to happen, hopefully he’d let me know and advise me.

It didn’t take long for a response to come back.  Apparently, there are indeed tests for the add-ons manager, and they’re right here in front of my face.  Crap, I should have known.  :/

So I dove into those tests…wow, there were a lot of them.  And I didn’t have a clue as to how to run any of them.

Following the Mochitest instructions, I eventually tried this:

TEST_PATH=toolkit/mozapps/extensions/test/ make -C $OBJDIR mochitest-plain

(where the TEST_PATH is set to the folder of tests I want to run, and $OBJDIR is an environment variable that points to the objdir compilation folder for Firefox.)

But this only ran a single test out of the bunch, and there were hundreds in there.  So what was the deal?

It turned out that the tests I wanted to run were with higher privileges than your average Mochitest test.  A basic Mochitest test is run using mochitest-plain.  Apparently, I needed to use mochitest-browser-chrome.  Took me a good half-hour to figure that one out.  :/

Anyhow, BAM, I had it – the tests were running.  The bad news:  I had a bunch of failing tests.  The good news:  the same tests failed without any of my changes.  So…great…I guess.

It was at this point in the day that I was given access to the Mozilla Messaging Intranet (the internal wiki).  There was plenty to read there, including something along the lines of “So you’re a new Mozilla Messaging hire”…I gave that a read.  Very interesting.

After that, I subscribed to a few internal mailing lists, and submitted my Mozilla-centric blog feed to be added to Planet Mozilla and Planet Mozilla Messaging.  Woop!

Finally, I got back to testing.  After digging through those add-ons manager tests, I finally found this:  PAYDIRT.

Sweet!  Tons of stuff for free in there:  MockProvider, createAddons… writing tests in there looked like it’d be cake.

But then it was home time.  More tomorrow.

Today, I want to learn a few things:

  1. How to use pbranch to locally commit my Firefox patch from last night
  2. How to write tests for my patch using Mochitest

Lets start with 1:

So Mozilla uses Mercurial as its distributed version control system.  I’ve been using Git since last summer with both MarkUs and Review Board.  Mercurial is something quite different.

Started going through pbranch tutorial.

So for my extension changes, here’s what I’m going to do.  I create a new branch using pnew, and commit to that.  I’ll keep committing to that.  When I’m all done, dump my patch to a Bugzilla attachment.  Someone will merge my patch.  Then I’ll remove my local branch and pull in my changes.  Sweet!

Ok, so I’ve got the workflow (I think).  Next, I need to figure out how to write a Mochitest.  Thankfully, there’s this:  https://developer.mozilla.org/en/Mochitest

I’m reminded of Selenium a little bit.  I think it’s sort of the same idea.

So how do I write a test for my changes?  Unfortunately, the documentation on how to write a Mochitest is a little thin.  So I guess I’ll be looking at examples, and extrapolating from there.  Let’s see if I can find a similar test written elsewhere.

This is promising:  http://mxr.mozilla.org/mozilla-central/source/testing/mozmill/tests/shared-modules/testAddonsAPI.js

Hm…but this is for Mozmill, and not Mochitest.

Yep, Mozilla uses a lot of different testing frameworks.  It’s a little confusing.  Mozmill is also like Selenium

So who is using testAddonsAPI.js?  Argh.  It looks like nobody.

I’m having a hard time finding tests for any of the stuff in extensions.js.  So I guess it’s time to talk to the expert.  I use hg log to see who the most frequent committer is to extensions.js.  The name Dave Townsend comes up.  http://www.oxymoronical.com/ .  He’s Mossop on IRC, and not online.  So who else is listed in hg log?  Blair McBride.

Had my first Mozilla Messaging weekly meeting.  It’s on a phone.  Interesting how its organized…really not like any phone conversation I’ve been a part of.  People mute themselves…unmute when its time to talk.  Awkward pauses are rampant…pretty cool though.  Coordinating around the world.  Nice!

Ok, back to Blair McBride…after a little hunting around, it turns out his IRC nickname is Unfocused.  I’ve found him in a few of the Mozilla IRC channels, and am waiting to hear back from him.

No word.  So, after some scrubbing, I posted my patch up on Bugzilla, and asked Dave Townsend for a review.  If testing needs to happen, hopefully he’ll let me know and advise me.

Whoop, just got a message.  The tests are here:  http://mxr.mozilla.org/mozilla-central/source/toolkit/mozapps/extensions/test/.  Crap, I should have known.  :/

Ok, lets examine those tests… hold up.  How do I run these?  Trying to run that directory with Mochitest, I only get 1 test to run…wtf?

Success!  TEST_PATH=toolkit/mozapps/extensions/test/ make -C . mochitest-browser-chrome

Sweet, so all those tests run.  Bad news though – a bunch of failing tests.  Going to see if it was my patch.  Ok, looks like a bunch of tests were failing before I even committed anything.  That’s good, I guess.

So, I have to compare them in order to ensure that there aren’t MORE failing tests after my patch goes in.

I now have access to the Intranet.  Sweet – lots of stuff to read here.  “So you’re a new Mozilla Messaging hire…”

Just subscribed to a few internal mailing lists, and submitted by Mozilla feed to be added to Planet Mozilla and Planet Mozilla Messaging.  Woop!

Ok, back to testing, I’ve found this:  http://mxr.mozilla.org/mozilla-central/source/toolkit/mozapps/extensions/test/browser/browser_sorting.js  This looks like paydirt.

The CEO of Mozilla Messaging (David Ascher) just welcomed me to MoMo:

mconley: welcome to the madhouse

Awesome. [WELCOME TO THE JUNGLE]

Sweet!  Tons of stuff for free:  MockProvider, createAddons – I think this’ll be cake tomorrow.

Ooops – Dave Townsend just asked a good question:  what about extensions that are to-be-installed?  to-be-uninstalled?  Where do they go?

I’ll have to check that out tomorrow.

But right now, it’s home time.

Day 1 at Mozilla Messaging: Getting my hands dirty…

So today was my first day working for Mozilla Messaging.

And it was awesome.

Now, I know that I had originally told you that I’d be working on Thunderbird + Unity integration.  That’s still on my TODO list, but today, I put some contributions in with the Firefox team for the upcoming Firefox 4 release.  There are still a few bugs that need to be squashed before Firefox 4 is ready for the prime-time, and we’re all psyched to see it happen, so I lent a hand.

This was the bug I decided to tackle today. Basically, in the new add-ons manager, add-ons are listed alphabetically, regardless of their state (and by state, I mean enabled, disabled, incompatible, blocked, etc).  This leads to kind of a strip-ey look for large numbers of add-ons.

What we’d like instead is to have the add-ons grouped by state, and then sorted alphabetically within that state.  Here’s a mock-up by bug-reporter (and Firefox UX team member) Jennifer Boriss:

Add-ons in list, grouped by status

I got to work around 8:45AM this morning.  Around 10:13AM, I had finished most of my introductions, had my tour, had my accounts set up, found my desk, and started work.

The first thing I did was locate where exactly the code was for listing the add-ons.  Blake pointed out a particularly useful resource called MXR (Mozilla Cross Reference), which lets me search through the source code very quickly.  I had originally been using grep, but this was way better.

My approach to finding the code:  I find a string in the interface that seems unique to the area I’m working in (in this case, it was “Search all add-ons”, which is found in the search text input of the add-ons manager).  I did a search for that string, and it returned the file mozilla-central/toolkit/locales/en-US/chrome/mozapps/extensions/extensions.dtd.

But this only takes us half-way.  Since Firefox is translated, the strings are stored separately from where they’re used.  In extensions.dtd, the string I searched for is given the key “search.placeholder”.

Searching for “search.placeholder” with MXR gives me paydirt:  mozilla-central/toolkit/mozapps/extensions/content/ holds the files extensions.css, extensions.js, extensions.xul and extensions.xml.

So I tool around in extensions.js a bit, reading the developer documentation, and getting to know the code.  The code contains quite a few classes, including something called gViewController.  Digging deeper into it, it looks like gViewController controls (brings up, shuts down) the various “views” available in the add-ons manager.  Cool – so now I just need to find the view that lists the user’s add-ons.

Ah hah!  Found it:  gListView.  gListView grabs all of the installed (or to-be-installed) add-ons, and populates the interface.  So this is where the sorting needs to happen.  I tested to make sure I was in the right place in the code by using dump calls to a terminal.  Bonus:  edits in this portion of the code do not require a full browser restart – instead, I just close and reopen the add-ons manager to see my changes.  Nice!

Initially, my idea was to have 4 “buckets”, one for each of the 4 states (“enabled”, “incompatible”, “disabled”, “blocked”).  I’d divide up the add-ons into those 4 buckets, and then sort each bucket, and then join the 4 buckets for the final result.

Blake came up with a better solution: the Schwartzian Transform (straight out of Space Balls, I know).  The idea is that, instead of having separate buckets like before, we just go through the entire collection of add-ons, and “label” them based on what bucket they belong to.  Then we sort the entire collection, giving the bucket-label a high-order sorting priority.  It sounds complicated, but it actually wasn’t that difficult to code.  It’s a tiny piece of code, its readable, and quite elegant.  So kudos to Blake for that solution.

Anyhow, I wrote my patch.  I just figured out how to run the interface tests for Firefox (they use something called Mochitest).  I set those tests off on one of the fast machines in the office, packed up my gear, and headed home.

It was a solid days work.  Tomorrow, I hope to write some new tests for the Schwartzian transform, and submit my patch to Bugzilla.  Woop!