Tag Archives: mercurial

Things I’ve Learned This Week (May 18 – May 22, 2015)

You might have noticed that I had no “Things I’ve Learned This Week” post last week. Sorry about that – by the end of the week, I looked at my Evernote of “lessons from the week”, and it was empty. I’m certain I’d learned stuff, but I just failed to write it down. So I guess the lesson I learned last week was, always write down what you learn.

How to make your mozilla-central Mercurial clone work faster

I like Mercurial. I also like Git, but recently, I’ve gotten pretty used to Mercurial.

One complaint I hear over and over (and I’m guilty of it myself sometimes), is that “Mercurial is slow”. I’ve even experienced that slowness during some of my Joy of Coding episodes.

This past week, I was helping my awesome new intern get set up to tear into some e10s bugs, and at some point we went through this document to get her .hgrc all set up.

This document did not exist when I first started working with Mercurial – back then, I was using mq or sometimes pbranch, and grumbling about how I missed Git.

But there is some gold in this document.

gps has been doing some killer work documenting best practices with Mercurial, and this document is one of the results of his labour.

The part that’s really made the difference for me is the hgwatchman bit.

watchman is a tool that some folks at Facebook wrote to monitor changes in a folder. hgwatchman is an extension for Mercurial that takes advantage of watchman for a repository, smartly precomputing a bunch of stuff when the folder changes so that when you fire a command, like

hg status

It takes a fraction of the time it’d take without hgwatchman. A fraction.

Here’s how I set hgwatchman up on my MacBook (though you should probably go by the Mercurial for Mozillians doc as the official reference):

  1. Install watchman with brew: 
    brew install watchman
  2. Clone the hgwatchman extension to some folder that you can easily remember and build it: 
    hg clone https://bitbucket.org/facebook/hgwatchman
cd hgwatchman
make local
  3. Add the following lines to my user .hgrc: 
    [extensions]
hgwatchman = cloned-in-dir/hgwatchman/hgwatchman
  4. Make sure the extension is properly installed by running: 
    hg help extensions
  5. hgwatchman should be listed under “enabled extensions”. If it didn’t work, keep in mind that you want to target the hgwatchman directory
  6. And then in my mozilla-central .hg/.hgrc: 
    [watchman]
mode = on
  7. Boom, you’re done!

Congratulations, hg should feel snappier now!

Next step is to try out this chg thing – though I’m having some issues still.

Things I’ve Learned This Week (April 6 – April 10, 2015)

It’s possible to synthesize native Cocoa events and dispatch them to your own app

For example, here is where we synthesize native mouse events for OS X. I think this is mostly used for testing when we want to simulate mouse activity.

Note that if you attempt to replay a queue of synthesized (or cached) native Cocoa events to trackSwipeEventWithOptions, those events might get coalesced and not behave the way you want. mstange and I ran into this while working on this bug to get some basic gesture support working with Nightly+e10s (Specifically, the history swiping gesture on OS X).

We were able to determine that OS X was coalescing the events because we grabbed the section of code that implements trackSwipeEventWithOptions, and used the Hopper Disassembler to decompile the assembly into some pseudocode. After reading it through, we found some logging messages in there referring to coalescing. We noticed that those log messages were only sent when NSDebugSwipeTrackingLogic was set to true, we executed this:

defaults write org.mozilla.nightlydebug NSDebugSwipeTrackingLogic -bool YES

In the console, and then re-ran our swiping test in a debug build of Nightly to see what messages came out. Sure enough, this is what we saw:

2015-04-09 15:11:55.395 firefox[5203:707] ___trackSwipeWithScrollEvent_block_invoke_0 coalescing scrollevents
2015-04-09 15:11:55.395 firefox[5203:707] ___trackSwipeWithScrollEvent_block_invoke_0 cumulativeDelta:-2.000 progress:-0.002
2015-04-09 15:11:55.395 firefox[5203:707] ___trackSwipeWithScrollEvent_block_invoke_0 cumulativeDelta:-2.000 progress:-0.002 adjusted:-0.002
2015-04-09 15:11:55.396 firefox[5203:707] ___trackSwipeWithScrollEvent_block_invoke_0 call trackingHandler(NSEventPhaseChanged, gestureAmount:-0.002)

This coalescing means that trackSwipeEventWithOptions is only getting a subset of the events that we’re sending, which is not what we had intended. It’s still not clear what triggers the coalescing – I suspect it might have to do with how rapidly we flush our native event queue, but mstange suspects it might be more sophisticated than that. Unfortunately, the pseudocode doesn’t make it too clear.

String templates and toSource might run the risk of higher memory use?

I’m not sure I “learned” this so much, but I saw it in passing this week in this bug. Apparently, there was some section of the Marionette testing framework that was doing request / response logging with toSource and some string templates, and this caused a 20MB regression on AWSY. Doing away with those in favour of old-school string concatenation and JSON.stringify seems to have addressed the issue.

When you change the remote attribute on a <xul:browser> you need to re-add the <xul:browser> to the DOM tree

I think I knew this a while back, but I’d forgotten it. I actually re-figured it out during the last episode of The Joy of Coding. When you change the remoteness of a <xul:browser>, you can’t just flip the remote attribute and call it a day. You actually have to remove it from the DOM and re-add it in order for the change to manifest properly.

You also have to re-add any frame scripts you had specially loaded into the previous incarnation of the browser before you flipped the remoteness attribute.1

Using Mercurial, and want to re-land a patch that got backed out? hg graft is your friend!

Suppose you got backed out, and want to reland your patch(es) with some small changes. Try this:

hg update -r tip
hg graft --force BASEREV:ENDREV

This will re-land your changes on top of tip. Note that you need –force, otherwise Mercurial will skip over changes it notices have already landed in the commit ancestry.

These re-landed changes are in the draft stage, so you can update to them, and assuming you are using the evolve extension2, and commit –amend them before pushing. Voila!

Here’s the documentation for hg graft.

  1. We sidestep this with browser tabs by putting those browsers into “groups”, and having any new browsers, remote or otherwise, immediately load a particular set of framescripts. ↩

  2. And if you’re using Mercurial, you probably should be. ↩

Day 2 at Mozilla Messaging: pbranch, testing, teleconferencing, intranet, and more testing

Ok, today was my second day at Mozilla Messaging.  Another good day.  Here are some highlights:

Today, I started off my day by wanting to learn a few things:

  1. How to use pbranch to locally commit my Firefox patch from yesterday
  2. How to write tests for my patch using Mochitest

I started with the first one.

So, for the most part, Mozilla uses Mercurial as its distributed version control system.  I’ve been using Git (arguably Mercurial’s main competitor) since last summer with both MarkUs and Review Board.  Mercurial is something quite different.  Quite different indeed.

pbranch is a tool that lets me have a patch queue.  Basically, organization, and re-organization of any changes I make to the Mozilla code-base is a lot easier using something like pbranch.

So I spent a few minutes going through the pbranch tutorial.  Eventually, I think I got the hang of it – basically, for my extension changes-sets, I create a new branch using hg pnew, and commit to that branch.  I’ll keep committing to that, and when I’m all done, I’ll dump my patch to a Bugzilla attachment.  After I pass code review, someone will merge my patch.  Then I’ll remove my local branch and pull in my changes.  Sweet!

Ok, so at this point, I think I got the workflow.  Next, I needed to figure out how to write a Mochitest.  Thankfully, there’s this tutorial.

Looking through the documentation, I was reminded of Selenium a little bit.  I think it’s sort of the same idea.

So how do I write a test for my changes?  Unfortunately, the documentation on how to write a Mochitest is a little thin.  So I started hunting around, looking for examples to extrapolate from.

At some point, I found myself staring at this code.  Wow!  A full-blown API for manipulating the add-ons manager!  Great!  But it turns out that this is for Mozmill tests, and not Mochitests.  But a quick search through MXR showed that nobody was using that add-on manager API.  Argh.

Was I barking up the wrong tree here?  Where the hell were the add-on manager tests?

I quickly swallowed my pride, and decided to talk to an expert.  I used Mercurial’s log function to determine who had changed extensions.js the most.  The name Dave Townsend came up.  According to his site, he’s “Mossop” on IRC, but he wasn’t online.  The log function also mentioned the name Blair McBride.  On IRC, he’s “Unfocused”.  He was online, but unavailable.

Argh.

It was at this point that Blake told me that the Mozilla Messaging weekly meeting was going to take place.  Apparently this happens every Tuesday at 9:30AM PST.  So we marched over to a conference room, hooked up this super-advanced phone (the phone had a boot-up screen, and then showed the Mozilla logo…whoa!).  A little while later, the meeting began.  The meeting was super fast, and super efficient – especially considering the teams are spread out across the globe.  One person led the meeting, and called the different teams up to give their weekly status.  I also got to introduce myself to the team.  I rambled off something about Thunderbird+Unity and code review, and then stumbled back to my chair.  Cool times.  Anyhow, teleconferencing is going to take some getting used to.

So, with the meeting over, and still no word from Unfocused, I decided to clean up my code a bit, and then posted my patch up on Bugzilla.  I asked Dave Townsend for a code review, and said that if testing needs to happen, hopefully he’d let me know and advise me.

It didn’t take long for a response to come back.  Apparently, there are indeed tests for the add-ons manager, and they’re right here in front of my face.  Crap, I should have known.  :/

So I dove into those tests…wow, there were a lot of them.  And I didn’t have a clue as to how to run any of them.

Following the Mochitest instructions, I eventually tried this:

TEST_PATH=toolkit/mozapps/extensions/test/ make -C $OBJDIR mochitest-plain

(where the TEST_PATH is set to the folder of tests I want to run, and $OBJDIR is an environment variable that points to the objdir compilation folder for Firefox.)

But this only ran a single test out of the bunch, and there were hundreds in there.  So what was the deal?

It turned out that the tests I wanted to run were with higher privileges than your average Mochitest test.  A basic Mochitest test is run using mochitest-plain.  Apparently, I needed to use mochitest-browser-chrome.  Took me a good half-hour to figure that one out.  :/

Anyhow, BAM, I had it – the tests were running.  The bad news:  I had a bunch of failing tests.  The good news:  the same tests failed without any of my changes.  So…great…I guess.

It was at this point in the day that I was given access to the Mozilla Messaging Intranet (the internal wiki).  There was plenty to read there, including something along the lines of “So you’re a new Mozilla Messaging hire”…I gave that a read.  Very interesting.

After that, I subscribed to a few internal mailing lists, and submitted my Mozilla-centric blog feed to be added to Planet Mozilla and Planet Mozilla Messaging.  Woop!

Finally, I got back to testing.  After digging through those add-ons manager tests, I finally found this:  PAYDIRT.

Sweet!  Tons of stuff for free in there:  MockProvider, createAddons… writing tests in there looked like it’d be cake.

But then it was home time.  More tomorrow.

Today, I want to learn a few things:

  1. How to use pbranch to locally commit my Firefox patch from last night
  2. How to write tests for my patch using Mochitest

Lets start with 1:

So Mozilla uses Mercurial as its distributed version control system.  I’ve been using Git since last summer with both MarkUs and Review Board.  Mercurial is something quite different.

Started going through pbranch tutorial.

So for my extension changes, here’s what I’m going to do.  I create a new branch using pnew, and commit to that.  I’ll keep committing to that.  When I’m all done, dump my patch to a Bugzilla attachment.  Someone will merge my patch.  Then I’ll remove my local branch and pull in my changes.  Sweet!

Ok, so I’ve got the workflow (I think).  Next, I need to figure out how to write a Mochitest.  Thankfully, there’s this:  https://developer.mozilla.org/en/Mochitest

I’m reminded of Selenium a little bit.  I think it’s sort of the same idea.

So how do I write a test for my changes?  Unfortunately, the documentation on how to write a Mochitest is a little thin.  So I guess I’ll be looking at examples, and extrapolating from there.  Let’s see if I can find a similar test written elsewhere.

This is promising:  http://mxr.mozilla.org/mozilla-central/source/testing/mozmill/tests/shared-modules/testAddonsAPI.js

Hm…but this is for Mozmill, and not Mochitest.

Yep, Mozilla uses a lot of different testing frameworks.  It’s a little confusing.  Mozmill is also like Selenium

So who is using testAddonsAPI.js?  Argh.  It looks like nobody.

I’m having a hard time finding tests for any of the stuff in extensions.js.  So I guess it’s time to talk to the expert.  I use hg log to see who the most frequent committer is to extensions.js.  The name Dave Townsend comes up.  http://www.oxymoronical.com/ .  He’s Mossop on IRC, and not online.  So who else is listed in hg log?  Blair McBride.

Had my first Mozilla Messaging weekly meeting.  It’s on a phone.  Interesting how its organized…really not like any phone conversation I’ve been a part of.  People mute themselves…unmute when its time to talk.  Awkward pauses are rampant…pretty cool though.  Coordinating around the world.  Nice!

Ok, back to Blair McBride…after a little hunting around, it turns out his IRC nickname is Unfocused.  I’ve found him in a few of the Mozilla IRC channels, and am waiting to hear back from him.

No word.  So, after some scrubbing, I posted my patch up on Bugzilla, and asked Dave Townsend for a review.  If testing needs to happen, hopefully he’ll let me know and advise me.

Whoop, just got a message.  The tests are here:  http://mxr.mozilla.org/mozilla-central/source/toolkit/mozapps/extensions/test/.  Crap, I should have known.  :/

Ok, lets examine those tests… hold up.  How do I run these?  Trying to run that directory with Mochitest, I only get 1 test to run…wtf?

Success!  TEST_PATH=toolkit/mozapps/extensions/test/ make -C . mochitest-browser-chrome

Sweet, so all those tests run.  Bad news though – a bunch of failing tests.  Going to see if it was my patch.  Ok, looks like a bunch of tests were failing before I even committed anything.  That’s good, I guess.

So, I have to compare them in order to ensure that there aren’t MORE failing tests after my patch goes in.

I now have access to the Intranet.  Sweet – lots of stuff to read here.  “So you’re a new Mozilla Messaging hire…”

Just subscribed to a few internal mailing lists, and submitted by Mozilla feed to be added to Planet Mozilla and Planet Mozilla Messaging.  Woop!

Ok, back to testing, I’ve found this:  http://mxr.mozilla.org/mozilla-central/source/toolkit/mozapps/extensions/test/browser/browser_sorting.js  This looks like paydirt.

The CEO of Mozilla Messaging (David Ascher) just welcomed me to MoMo:

mconley: welcome to the madhouse

Awesome. [WELCOME TO THE JUNGLE]

Sweet!  Tons of stuff for free:  MockProvider, createAddons – I think this’ll be cake tomorrow.

Ooops – Dave Townsend just asked a good question:  what about extensions that are to-be-installed?  to-be-uninstalled?  Where do they go?

I’ll have to check that out tomorrow.

But right now, it’s home time.