Photon Engineering Newsletter #14

Just like jaws did last week, I’m taking over for dolske this week to talk about stuff going on with Photon Engineering. So sit back, strap in, and absorb Photon Engineering Newsletter #14!

If you’ve got the release calendar at hand, you’ll note that Nightly 57 merges to Beta on September 20th. Given that there’s usually a soft-freeze before the merge, this means that there are less than 4 weeks remaining for Photon development. That’s right – in less than a month’s time, folks on the Beta channel who might not be following Nightly development are going to get their first Photon experience. That’ll be pretty exciting!

So with the clock winding down, the Photon team has started to shift more towards polish and bug-fixing. At this point, all of the major changes should have landed, and now we need to buff the code to a sparkling sheen.

The first thing you may have noticed is that, after a solid run of dogefox, the icon has shifted again:

The new Nightly icon

We now return you to your regularly scheduled programming

The second big change are our new 60fps1 loading throbbers in the tabs, coming straight to you from the Photon Animations team!

The new loading throbber in Nightly

I think it’s fair to say that Photon Animations are giving Firefox a turbo boost!

Other recent changes

Menus and structure

Animations

  • Did we mention the new tab loading throbber?

Preferences

  • All MVP work is completed! The team is now fixing polish bugs. Outstanding!

Visual redesign

Onboarding

Performance

  1. The screen capturing software I used here is only capturing at 30fps, so it’s really not doing it justice. This tweet might capture it better. ↩

Making tabs close faster in multi-process Firefox

TL;DR: In bug 1336763, I have landed a series of patches that should hopefully make tab closing faster for the majority of cases for users that are using multi-process Firefox.

The rest of this blog post tries to explain why.

The beforeunload event handler

Perhaps you’ve seen this dialog before:

The beforeunload dialog in Firefox

Are you sure?

This dialog shows up when a website that you’ve interacted with (or one of its subframes) has set an event handler for the beforeunload event, and you attempt to close the tab or browse away from the website.

Here’s the documentation for how beforeunload works, but long story short, you can do a thing in the event handler that will cause the browser UI to show that dialog, which means giving the user the opportunity to cancel their request to close or navigate away from the website1.

In any event2, if a page is going to be unloaded, and that page (or one of its subframes) has set one or more beforeunload event handlers, then it is necessary to run those event handlers to see if we’re going to show the dialog, or go ahead and unload the page straight away.

How multi-process Firefox used to handle beforeunload

When closing a tab in multi-process Firefox, what we’ve been doing is sending a message to the content process for that tab to check for (and run any) beforeunload event handlers. The parent sends that message, and then just kinda waits for the content process to respond with whether or not the close should occur. If the content process doesn’t respond within 5 seconds (I know), then we consider it a wash, and just close the tab.

The content process is sometimes doing stuff on the main thread, and sometimes it’s just waiting for messages from the parent. In the latter case, tab closes happen pretty smoothly – the message comes in, beforeunload events are fired (and hopefully those don’t take too long, but you never know), and then hopefully a result goes up to the parent, and it can move on.

That’s the best case scenario – but lots of things can prevent the best case scenario; for one thing, the main thread might be busy doing other stuff when the message is sent from the parent. Perhaps it’s doing a garbage collection, or a cycle collection, or it’s blocked on some busy JavaScript that some silly advertisement company is running in the background of one of your tabs. In that case, the message from the parent won’t be processed until the main thread is ready.

Once the message is received, we’re still not out of the woods – the beforeunload event handlers can run any kind of JavaScript inside them, more or less. For example, one anti-pattern I’ve seen in the wild is to use the beforeunload event as an opportunity to send a sync XMLHttpRequest in order to get some data to a server before the page goes away3. So the script on the page has an opportunity to delay you, even if it’s not going to cause the dialog to appear.

This problem seems to plague all browsers. beforeunload is a real pain, and our current implementation can cause slow tab closing even if the tab doesn’t have beforeunload event handlers set4. The patches in bug 1336763 offer what I think is a decent, simple solution for that common case in Firefox.

Don’t ask, just remember

In bug 1336763, I’ve made it so that for any given tab running in a content process, if a beforeunload event is ever added in that tab (or in any of its subframes), the content process tells the parent process so it can mark that tab as having listeners we need to fire. If the beforeunload events are removed, we unmark the tab. If no beforeunload events are ever added, there’s no mark at all.

The parent process remembers these markings, so that if the user decides to close the tab, the parent can know immediately whether or not it needs to message to tell the content process to run beforeunload event handlers. In the cases where no beforeunload event handlers have been set, we can close immediately without asking for permission from the content process at all.

Details, details

Using some Gecko terminology here, we start by storing a count on something called the TabChild. It might simplify things a bit if you try to imagine the TabChild as the representative of everything in a particular browser tab, and that underneath that TabChild are a bunch of nodes, forming a tree-like structure.

Let’s call these nodes “inner windows”.

The inner windows under the TabChild contain the documents that are loaded in a tab. For simple web pages, that might just be a single document. In that case, we have a TabChild with just a single inner window node under it.

More complicated pages might contain iframes (which themselves contain iframes, etc). In those cases, we have a TabChild with a single inner window node under it, and that node has any number of inner window children (and those children have any number of inner window children, etc)5.

When any of those subframes have a beforeunload event listener added to them via script, the inner window node tells the TabChild to increment its internal count. If a beforeunload event listener is removed via script, the TabChild is told to decrement its internal count.

If the TabChild count ever goes above 0, then we need to tell the parent “Hey, you have at least one beforeunload event listener here”. If that count continues to go up, the TabChild doesn’t need to tell the parent anything – it just needs to record the increase. If the count ever drops back to 0, then the TabChild needs to tell the parent again, “All beforeunload event listeners are clear”.

Pretty straight-forward so far, but there are a few other cases we also have to consider.

Other cases

There are a couple of ways for a set of beforeunload event handlers to go away. We’ve already mentioned one – script on the page might remove them via removeEventListener.

One way is if the inner window gets navigated away from. If we’re on a page, and that page set a beforeunload event handler, and the user clicks on a link, the user might end up navigating away (assuming the dialog wasn’t shown and they didn’t cancel), which essentially replaces the inner window with one for a different page. In that case, script didn’t remove the beforeunload event handlers – the page went away, and so the beforeunload event handlers on the page we’ve unloaded are no longer relevant.

Another way is if an <iframe> which has set a beforeunload event handler is removed from the DOM. Instead of replacing the inner window, we’re snipping the inner window out of the tree structure entirely.

In both of these cases, if there are beforeunload event handlers in the subframe, it’s necessary to tell the TabChild so that the right number can be decremented from the TabChild count.

So what this means is that we need the inner windows to keep a track of how many beforeunload event handlers have been set as well. That way, when they start to tear themselves down, they can tell the TabChild, “Hey, I’m going away now – decrement X number of beforeunload event handlers”.

It might seem redundant to have these two counts – counts in the inner windows, and a total count in the TabChild. It would seem like one can be easily inferred from the other; just sum the beforeunload counts for the inner windows, and you should have your TabChild count.

Having the TabChild keep a count is an optimization that prevents us from having to walk the inner window tree to collect a sum every time the count changes. It’s a classic space / time tradeoff, and I think it’s worth the extra integer member on the TabChild.

Comparison to other browsers

Here’s one way to compare the behaviour across different browsers:

  1. In a browser window with more than one tab open, open the developer tools, and make your way to the JavaScript console.
  2. Drop this tasty little snippet in and press enter: 
    var then = Date.now(); while (Date.now() - then < 15000) {}

This is going to hang the main thread in that tab for 15 seconds, but is otherwise inert.

Now try to close the tab. In Firefox, the tab closes right way. In Safari and Chrome (the two other browsers I have on this machine), the tab hangs out for a while. In Chrome, it appears to wait the full 15 seconds. In Safari, it seems to hit some kind of shorter timeout6.

Wrapping up

This was a neat set of patches to work on, precisely because it had me tour the depths of Gecko (dealing with things like inner / outer windows, the stuff that manages events, etc), which end up resulting in a simple property that the front-end can ultimately access to optimize closing tabs. So it nicely spanned the gap between low-level Gecko and higher-level Firefox, all for an event that was added back in 2004, for better or worse.

If all goes well, this change should ship in Firefox 55, and apply to multi-process tabs.

  1. It’s not always necessary for the beforeunload event handler to show the dialog. The event handler needs to set the returnValue property on the event to a string in order for the dialog to show, but plenty of other stuff can happen in that event handler. ↩

  2. Ooooh pun intended ↩

  3. A better way would be to use navigator.sendBeacon, which allows the browser to send one last XHR in the background for a page even after it’s gone away. ↩

  4. since we have to check to see if any of those event handlers exist. ↩

  5. For my fellow Gecko Hackers – yes, this is not quite right. I’m missing other key structures in my description (specifically, the outer window). Forgive me – this is a very low-resolution mental model to make this post easier to write. 🙂  ↩

  6. Note that it appears that the script running in the console is treated differently from the script running on the page. If you set up a page with that script, I notice that the tabs close immediately on both Chrome and Safari. I might have goofed in my experiment though – it is rather late at night.  ↩

A printing story and a PSA: outparams over the IPC layer might not behave like you’d expect

Here’s a story about a printing bug on OS X, and a lesson about how our IPC layer works.

Last week, :ehsan came up to my desk and said “Mike… printing is broken on OS X with e10s enabled. Did you know this?”

It was sad to hear. Nobody, as far as I knew, had been touching printing code (besides bobowen, but he hadn’t landed his patches yet), which meant that some mystery change had landed and it was my responsibility (as the one who had implemented printing on OS X for e10s) to fix it.

The first step was to confirm it. Yep, it looked like I couldn’t print on my Nightly. Crap1. So we get the bug filed, and then I fired up my local build and lldb, and tried to trace out where the problem was occurring.

Drilling down to the regressing changeset

The problem was that my local build printed just fine. Eyebrow raised, I tried using my default profile with my local build to see if something about my profile was causing printing to break. Printing continued to work with my local build2.

Scratching my head, I used mozregression to bisect the problem down to a single changeset.

Here’s the bug it spat out:

Bug 1209930 – Update Mac clang to match the version in use everywhere else

The hair stood up on the back of my neck. Printing got broken by a compiler upgrade? This had bad news written all over it.

Many try builds

So that, I guess, explained why I couldn’t reproduce the problem locally. I needed to build with a particular compiler in order to experience the bug.

The bad news was that I found out that the version of clang that we’d upgraded to didn’t work on my version of OS X (10.10.5). It was supposed to work on machines running OS X 10.7, but I didn’t have access to any3.

I was able to get debug builds off of try, but I had no luck convincing lldb to let me step through it, despite having the symbols and the right revision checked out4.

So this meant a lot of try builds. The good news is that I was able to add a bunch of logging to try, and just kinda walk away while it built. A few hours later, I’d have my build, and I’d get my results. I wasn’t blocked waiting on it to build, so I could work on other things.

What did the logging reveal?

Our printing code sometimes shows this progress dialog when printing starts. It’s put up so that while we’re laying out the page for printing, the user knows that stuff is happening. We show this progress dialog on Linux and Windows, but not on OS X (I guess OS X users don’t expect such a progress window… this was a decision made way back in the day).

The printing engine in the content process sends up a message to the parent saying, “I’m all set to print, let’s show the progress dialog!”. On OS X, the parent responds, “Uh, nope” (which is a return code of NS_ERROR_NOT_IMPLEMENTED), and the child is supposed to go, “Oh okay, I’ll just print right away”.

But here’s the kicker – along with message to show the progress dialog, the child sends an outparam5. That outparam is sent because for some platforms that show the progress dialog, we want to wait until the dialog closes before starting the actual print. On other platforms, we may just want to start printing right away when laying out the page finishes without waiting for the dialog to close.

That outparam is a bool initialized to a value of false in the print engine, and in the non-e10s case where we return “Sorry, no progress dialog” with NS_ERROR_NOT_IMPLEMENTED, that bool stays false, and when it stays false, it means that we start printing right away.

My logging showed that for some reason, that value was getting flipped to true despite not being touched by (seemingly) anything in the IPC sender / receiver nor the print progress dialog backend (which still just returns NS_ERROR_NOT_IMPLEMENTED when asked to open the printing dialog).

What the hell?

Here’s the PSA

Ehsan helped me dig into what was going on, and we figured it out. Here’s the big lesson / PSA here:

outparams over IPC, when untouched on the receiver side, get filled with values from uninitialized memory when returned to the sender.

I’ll give you an example. Here’s some sorta-pseudocode:

#include <stdio.h>

void someFunc(bool* aOutParam) {
}

int main(int argc, char** argv) {
  bool myBool = false;
  someFunc(&myBool);
  if (myBool) {
    printf("Wait, WHAT?");
    return 1;
  }
  return 0;
}

We should never enter that “Wait, WHAT?” conditional block because myBool remains false throughout. It’s initialized to false, and even though we pass a pointer to it to someFunc, someFunc never touches it, so it stays false, and so we skip the conditional and return 0 as expected.

If, however, you did the same thing, but over IPC… well, now you’re in for a fun treat.

On the receiver side of the IPC message, here’s a chunk of the generated code that calls into RecvShowPrintProgress on the parent side:

            bool notifyOnOpen;
            bool success;
            int32_t id__ = mId;
            if \6, (&(success)))))) {
                mozilla::ipc::ProtocolErrorBreakpoint("Handler for ShowProgress returned error code");
                return MsgProcessingError;
            }

            reply__ = new PPrinting::Reply_ShowProgress(id__);

            Write(notifyOnOpen, reply__);
            Write(success, reply__);
            (reply__)->set_sync();
            (reply__)->set_reply();

That notifyOnOpen bool is the one that is the outparam in the child. Notice that it’s not being initialized to any value! That means its current value could be, well, anything. When we call into RecvShowProgress, we pass a pointer to that anything value. If RecvShowProgress doesn’t do anything with that pointer (like in the OS X case), well… that random value is what gets written to the IPC message that gets sent back to the child.

And somehow a clang upgrade made it far more likely that this value would evaluate to TRUE as a boolean instead of FALSE.

And so the child assumed that it’d have to wait for a dialog to close before starting to print – a dialog that would never open, because we don’t show it on OS X.

So the patch I eventually wrote initializes the outparam to false before calling into the OS X widget code that just returns NS_ERROR_NOT_IMPLEMENTED, and that fixed the bug.

It’s a small patch, but in my mind, it’s a big lesson, at least for me. I had assumed that outparams would work the same over IPC as they do in the same process, and that’s simply not the case.

I’ve filed bug 1220168 to try to make this sort of thing easier to spot in the future.

  1. Our automated testing story for printing is pretty bad. We test some of the built-in UI for printing, but as for actually sending stuff to printer drivers… zero automated tests, which explains why this had gone uncaught for so long. ↩

  2. If you’re wondering, I wasn’t printing out reams of paper to test this. XCode comes with a Printer Simulator, which is like a PDF printer, except that I can get a better sense of the communications being sent to the printer ↩

  3. Okay, there’s one in the office, but apparently trying to build on it is a slow nightmare ↩

  4. Happy to hear and share how to do this if someone will show me ↩

  5. Read this about passing multiple values back from a method call if you’re curious about what outparams are ↩

  6. !(RecvShowProgress(mozilla::Move(browser), mozilla::Move(printProgressDialog), mozilla::Move(isForPrinting), (&(notifyOnOpen ↩

The Joy of Coding – Ep’s 23 – 29

Wow! I’ve been a way from this blog for too long. I also haven’t posted any new episodes for The Joy of Coding. I also haven’t been keeping up with my Things I’ve Learned posts.

Time to get back in the saddle. First thing’s first, here are 6 episodes of The Joy of Coding that have aired. Unfortunately, I haven’t put together summaries for any of them, but I’ve put their agendas near the videos so that might give some clues.

Here we go!

Episode 23

Agenda

Episode 24

Agenda

Episode 25

Agenda

Episode 26

Agenda

Episode 27

Agenda

Episode 28

Agenda

Episode 29

Agenda

The Joy of Coding (Ep. 20): Reviewin’ and Mystery Solvin’

After a two week hiatus, we’re back with Episode 20!

In this episode, I start off by demonstrating my new green screen1, and then dive right into reviewing some code to make the Lightweight Theme web installer work with e10s.

After that, I start investigating a mystery that my intern ran into a few days back, where for some reason, preloaded about:newtab pages were behaving really strangely when they were loaded in the content process. Strangely, as in, the pages wouldn’t do simple things, like reload when the user pressed the Reload button.

Something strange was afoot.

Do we solve the mystery? Do we figure out what’s going on? Do we find a solution? Tune in and find out!

Episode agenda.

References

Bug 653065 – Make the lightweight theme web installer ready for e10s
Bug 1181601 – Unable to receive messages from preloaded, remote newtab page – Notes
@mrrrgn hacks together a WebSocket server implementation in Go. To techno!

  1. Although throughout the video, the lag between the audio and the video gets worse and worse – sorry about that. I’ll see what I can do to fix that for next time. ↩