Customizing the user agent string in Oxide

The Ubuntu webbrowser sets its own default user agent string, and also provides a list of overrides which define different user agent strings for specific sites that do broken UA sniffing. This is a concept that we’ve borrowed from Firefox OS.

With QtWebkit, the overriding is performed via the same mechanism that is used for setting the default user agent string – by setting the WebViewExperimental.userAgent property. However, this has a major shortcoming – it provides no means to override the user agent string for specific frames. Say, for example, that you need to provide an override for www.twitter.com, but not for www.example.org. If a page from www.example.org embeds www.twitter.com in an iframe, there isn’t a mechanism to change the user agent string for the child frame.

When overriding the user agent string for a frame, there are 2 places that this needs to happen:

  • navigator.userAgent
  • The User-Agent header in outgoing HTTP requests.

Fortunately, Chromium provides a mechanism by which we can implement the second one – net::NetworkDelegate::OnBeforeSendHeaders(). Unfortunately, this is called on Chromium’s IO thread which makes it difficult to expose as a signal in QML (a QML engine can only execute code on a single thread). The notification is asynchronous, which means we could dispatch an event to the UI thread, emit a signal in QML and then dispatch an event back to Chromium’s IO thread. However, this has shortcomings of its own:

  • A typical page load can generate more than 100 network requests. For example, loading BBC News generated 165 requests when I tested it here. Assuming we have multiple notifications per request, processing all of these on the UI thread in QML could have a noticeable performance impact.
  • Not all notifications we get from net::NetworkDelegate are asynchronous. For example, cookie access permission requests aren’t.

Enter WebContextDelegateWorker

For handling events that happen on Chromium’s IO thread, we have a new class – WebContextDelegateWorker. This is very similar to QML’s WorkerScript – it runs script on a worker thread and provides sendMessage/onMessage API’s for exchanging data with QML on the UI thread. However, it provides a mechanism for scripts to expose entry points that can be called from Chromium’s IO thread (actually, they’re not called from Chromium’s IO thread – they’re called on the worker thread by blocking the IO thread. But, you don’t need to care about that).

To overridethe User-Agent header for specific requests in Oxide, you can do something like the following:

In QML:

WebView {
  ...
  context: WebContext {
    ...
    networkRequestDelegate: WebContextDelegateWorker {
      source: Qt.resolvedUrl("worker.js")
      onMessage: {
        console.log("We set User-Agent on " + message.url);
      }
    }
  }
}

In worker.js:

// Unfortunately the properties of QUrl don't appear to be available in QML,
// so provide a simple regex to extract the host
var re=/^[^:]+:\/\/([^\/]*)\/.*/;
exports.onBeforeSendHeaders = function(event) {
  if (re.exec(event.url)[1] == "www.twitter.com") {
    event.setHeader("User-Agent", "Foo");
    oxide.sendMessage({url: event.url});
  }
}

This doesn’t affect navigator.userAgent though. In order to provide an override mechanism for this, we had to apply a small patch to Chromium to give embedders the opportunity to provide an override value when accessed. We did consider exposing a signal in QML for overriding this, based on the assumption that this would run much less frequently than network notifications. However, in the end we decided to reuse WebContextDelegateWorker, based on the assumption that developers who want to override navigator.userAgent will probably also want to use the same overrides for the User-Agent header. Doing this means that you only need the override list in one script context.

To override navigator.userAgent, the above code becomes:

In QML:

WebView {
  ...
  context: WebContext {
    ...
    networkRequestDelegate: WebContextDelegateWorker {
      source: Qt.resolvedUrl("worker.js")
      onMessage: {
        console.log("We set User-Agent on " + message.url);
      }
    }
    userAgentOverrideDelegate: networkRequestDelegate
  }
}

In worker.js:

// Unfortunately the properties of QUrl don't appear to be available in QML,
// so provide a simple regex to extract the host
var re=/^[^:]+:\/\/([^\/]*)\/.*/;

exports.onBeforeSendHeaders = function(event) {
  if (re.exec(event.url)[1] == "www.twitter.com") {
    event.setHeader("User-Agent", "Foo");
    oxide.sendMessage({url: event.url});
  }
}

exports.onGetUserAgentOverride = function(data) {
  if (re.exec(data.url)[1] == "www.twitter.com") {
    data.userAgentOverride = "Foo";
  }
}

Setting a default string

Oxide doesn’t have a per-WebView property that’s equivalent to WebViewExperimental.userAgent in QtWebkit, as we decided that this would be redundant with the above override mechanism. Instead, the default user agent string is configured by setting the WebContext.userAgent property.

Other uses for WebContextDelegateWorker

WebContextDelegateWorker is also used for overriding the default storage access policy via WebContext.storageAccessPermissionDelegate (in QML) and exports.onStoragePermissionRequest (in the worker). This is currently only used for cookies, but is due to be expanded to local storage, app cache, indexedDB and WebDB. The default storage access policy is controlled by WebContext.cookiePolicy

Oxide status

It’s been a while since I wrote my last blog post – Introducing Oxide. I’ve been quite busy since, so I thought that I should take this opportunity to provide a quick update on progress.

Our overall aim is to have the webapps container using Oxide in Ubuntu 14.04, and we’re also hoping to be able to transition the Ubuntu browser quite soon as well.

Summary of progress

A lot has changed since my last post:

It….

  • runs independently of X11
  • supports accelerated compositing on both Mir and X11
  • has been running on several devices with Ubuntu Touch – Nexus 4 (mako), Nexus 7 (grouper) and Galaxy Nexus (maguro)
  • has a navigation history API (WebView.navigationHistory)
  • has a WebPreferences API (WebView.preferences)
  • is DPI aware
  • has input method support, and the onscreen keyboard works on Ubuntu Touch
  • has support for touch events and gestures (pinch-to-zoom and fling works)
  • supports third-party cookie blocking
  • has a mechanism for intercepting and modifying outgoing HTTP requests, which can be used for redirecting URL’s or modifying headers (such as User-Agent)
  • has a mechanism for intercepting accesses to navigator.userAgent which allows applications to customize this on a per-frame and per-domain basis (more on this and the above point in another blog post)

In addition to this, I also have the following features in my review queue:

  • Support for JS dialogs (alert, confirm etc)
  • File picker support
  • Cursor change support
  • API for exposing favicons
  • API for delivering console messages to embedders

Oxide also runs WebGL and plays video quite well on both the desktop and the device.

 

Next steps

Here is a list of things we need to tackle quite urgently:

And then, there are some other major tasks which are still important but not complete blockers at the moment:

  • Geolocation support (requires permission request API and an API key for Google’s geolocation service. Eventually, this should use the Ubuntu location service)
  • Device access permission request API (for microphone and webcam)
  • Support window.open() and HTML anchor ‘target=”_blank”‘
  • Add a fullscreen API – the mobile version of Youtube requires this to play videos
  • We need to join Chromium’s release train with the creation of beta and release branches

I also need to clear my review queue :)

For a more complete list of remaining work, please see the list of open bugs.

Testing Oxide

Builds of Oxide are currently uploaded fairly regularly to the Phablet Team PPA. Feel free to download it, play around with the API and report bugs. Sorry, there’s no documentation yet, so you will need to look at the source code or ask me questions about how it works for now. It is quite easy to use though.

The PPA also contains a build of webbrowser-app that uses Oxide instead of QtWebkit. You can try this out, but if you do, you should be aware of the following caveats:

  • It does replace the current webbrowser-app, which uses QtWebkit
  • Whilst it’s ok for basic browsing, testing and bug reporting, there is still major functionality missing that means you probably don’t want to use it for day-to-day browsing just yet.

Getting involved

If you want to help make Oxide (and our web browser) awesome, please take a look at the list of currently open bugs. If you see anything that you’d like to work on, feel free to come and talk to me on IRC (I’m “chrisccoulson” on irc.freenode.net), and I’ll be glad to help out.

Thanks…

For all the hard work from everyone who has contributed or contributing to Oxide so far: Olivier Tilloy, Alexandre Abreu, Jamie Strandboge, Justin McPherson and Maxim Ermilov.

Also, thanks to Bill Filler, David Barth and Pat McGowan for keeping everybody focused.

And to Parameswaran Sivatharman for his tireless efforts on Continuous Integration.

I’m really excited about this :)

Introducing Oxide

I’m terrible at writing blog posts, and I’ve been meaning to write this since UDS at least. Sorry!

If you saw the wrap-up notes for the August UDS, you might have noticed the following statement:

oxide: we sorted out the details of the work that needs to happen for oxide to be [sic] replace qtwebkit, for the touch browser, webapps and SDK, before 14.04

If you’re wondering what on earth is going on, then this post attempts to explain.

A few months ago, I started to look at how we provide post-release support for the various Webkit ports in Ubuntu – in particular, QtWebkit (which is used by the Ubuntu Touch browser and in the Ubuntu SDK).  For an acceptable level of post-release support we need to be able to deliver the following, as a minimum:

  1. Security fixes for the duration of the LTS (5 years).
  2. Updates to add support for new web technologies and to provide web compatibility fixes throughout the support life-cycle.

In addition to this, we also need something that:

  1. Is supportable on the phone, converged device and desktop.
  2. Has good integration with the native toolkit, SDK and accessibility mechanisms.
  3. Allows us to ship a browser that delivers a competitive level of performance.

At the moment, we only really achieve the last 3 – and we do this by relying on other projects to do the work for us.  This is great, but we need a proper post-release story too.  The main difficulty with this is that the upstream projects generally don’t support their releases with updates for the duration that we require in Ubuntu – this means that it becomes the distribution’s responsibility to provide security maintenance for 5 years.  As an example, Ubuntu 13.04 currently includes QtWebkit based on the 5.0 release of Qt – from which, the last maintenance release was 5.0.2 in April 2013.  Since then there have been 4 major releases of Chrome, and in 1 month of commits to the main Webkit SVN repository, I counted no fewer than 7 security fixes.

We’ve been thinking of ways to resolve this for some time now.  I’ve made a list of things that have been discussed:

  1. Backporting individual security fixes from the main Webkit SVN repository for the support life-cycle.
  2. Backporting complete new versions of the supported Webkit ports.
  3. Taking the existing Webkit ports and regularly rebasing them on the latest WebCore and JSC ourselves.
  4. Creating our own Webkit port.
Backporting individual security fixes from the main Webkit SVN repository

This is basically our current strategy, and it is one that we’ve already abandoned for the major browsers that are shipped in the Ubuntu archive (Firefox and Chromium).  There are several problems with this strategy:

  • We don’t have access to Webkit security bugs, so don’t really know what to backport or how to verify what we’re backporting.
  • Backporting patches is actually pretty difficult and error prone, particularly when it’s being done by people who aren’t working on the code.
  • Backporting patches becomes much harder, riskier and more time consuming as the newest Webkit code gradually diverges more from the code that we ship in Ubuntu.

In addition to this, backporting individual security fixes doesn’t help us with our second requirement, which is to ensure that we ship something that is compatible with the latest web technologies.

Backporting complete new versions of Webkit ports

This has been tried recently with WebkitGTK, with limited success.  The problem with this is that new releases tend to depend on newer versions of various other parts of the platform, making it difficult to ship the latest versions on older Ubuntu releases.  As an example, WebkitGTK 2.2.0 depends on Gtk 3.6 (Ubuntu 12.04 includes 3.4), Glib 2.36 (Ubuntu 12.10 includes 2.34) and libsoup 2.42 (Ubuntu 13.04 – the latest stable version, only includes 2.40).  The latest version of WebkitGTK is only buildable on Ubuntu 13.10, which isn’t even released yet.

The current development release of QtWebkit also doesn’t build on Ubuntu 13.10 (which still includes the Qt 5.0 stack).

Regularly rebasing supported Webkit ports on the latest WebCore

This is a non-starter really.  There’s not really much difference between this and forking each of the Webkit ports, and I don’t think that the various upstream developers would be that pleased with us shipping FrankenWebkit versions.

Creating our own Webkit port

This has the advantage that we would be able to control the release model, and regularly rebase our port on the most appropriate revision of WebCore in order to pick up security fixes and web compatibility updates.  We would be able to do this whilst providing guaranteed API stability for the Ubuntu SDK.  Of course, there is the small issue of this requiring a huge amount of work…..

Enter Chromium

Shortly after Google announced that they were going to fork Webkit, we started to explore the possibility of providing a library for our SDK that was based on the Chromium Content API (and hence, V8 + Blink).  I had already started to look at one such effort (Chromium Embedded Framework), although this didn’t really fit our requirements.

Chromium has some advantages over Webkit – it provides more functionality out of the box, so it should be easier to build a library on this rather than create our own port of Webkit.  Webkit no longer powers a major browser on the Linux desktop, so it’s more of a risk for us in the long-term.  Also, Chrome’s popularity and market share makes shipping a browser based on it a good strategic decision.

At our Oakland sprint in April, I took away some initial actions to scope out the amount of work required to provide our own library based on Chromium.  I did that, and have since been working on an interesting little project….

Introducing Oxide

Oxide is a library that allows you to embed a Chromium-powered webview in QML applications.  It currently provides a fairly trivial API that is quite similar to the current QML QtWebkit webview API for simple use cases, although it is nowhere near feature complete yet.

You can grab the code from https://launchpad.net/oxide.

What works currently (not all of these features have full test coverage yet, but I am getting there):

  • Basic navigation (load url / back / forward reload).
  • Incognito mode.
  • Multiple browser contexts so you can have multiple webviews with different permanent storage locations.
  • User scripts, with support for a limited subset of Greasemonkey metadata (exclude, include, match, run-at), an option to inject in to matching subframes, and support for running scripts in different contexts.
  • Message passing API to allow communication between the embedder and user scripts (this is used for the automated tests it has so far).

What next?

The plan is to have the Ubuntu Touch browser and SDK using this by Ubuntu 14.04.  There is a lot of work to do between now and then – some of the things we need are:

  • Accelerated compositing.
  • A proper navigation API with back / forward list.
  • Dialog support (alert / confirm / prompt / file / colour / HTTP auth).
  • Media device access permissions API.

Also see the open bugs in Launchpad.

If you want to help out with this, feel free to grab me on IRC – I’ve got plenty of work I can hand out!  At some point, I plan on writing a guide to help people get started with this.

Why Ubuntu is not using the Firefox ESR

Recently, a thread has appeared on the ubuntu-desktop mailing list asking why Ubuntu (or specifically, Ubuntu 12.04 LTS) is not using the Extended Support Release of Firefox by default. I’ve also been asked this a few times on IRC over the last few weeks (from people inside and outside of Canonical), so I just wanted to clarify the reasoning for this, and why I think that our choice will offer the best experience for Ubuntu users.

Arguments against the ESR

Aside from the fact that offering the Firefox ESR by default to Ubuntu users would make our good friends at Mozilla unhappy (and from my perspective, we have a pretty good relationship with them. I certainly don’t want this to change), here are some reasons why I think that shipping the Firefox ESR by default would be bad for Ubuntu users. Some of these are points from the ESR proposal, and some of them are my own thoughts.

Over time, Firefox ESR will become less secure than Firefox

The ESR proposal states that:

Mozilla will backport security bugs qualified as “Critical” and “High” to the ESR where feasible (there may be cases where a backport cannot be applied with reasonable effort, and those cases are expected to be exceptional)

This means that people running the Firefox ESR may have to wait for the next major release to receive fixes for security bugs with sg:moderate, sg:low or sg:dos severity ratings. What is worse is that ESR users may also miss out on some sg:critical or sg:high rated bugs, where it is just not feasible to backport the fix to the ESR branch.

In addition to this, Firefox ESR users will have to wait longer than regular Firefox users for proactive security improvements such as support for the iframe sandbox attribute, CA pinning, the Mixed Content Blocker or any other new security features / improvements.

Because of this, the Firefox ESR will always become less secure than the regular Firefox releases.

Ubuntu 10.04 LTS users have already had to wait longer than Firefox users on other platforms for new security and privacy features such as Content Security Policy, HTTP Strict Transport Security and Do Not Track. We don’t want Ubuntu to lag behind other platforms in the future.

The risk of introducing bugs is greater with Firefox ESR

There is a common misconception that when a piece of software receives only reactive security fixes, it is the safest option for users and that the risk of breakage is minimal with this approach. In reality, this isn’t exactly true. There is always risk associated with backporting any form of code change from one branch to an older branch, and this risk increases as:

  • The 2 branches diverge further apart, making the backport less straightforward
  • The amount of testing exposure decreases

Clearly, the Firefox ESR will be affected by both of these factors.

The regular Firefox releases pass through the beta channel for 6 weeks before release, where they are exposed to a large community of users who are using the beta as their day-to-day browser. And, whilst the Linux testing community is relatively small (I would like to grow this though), we mustn’t take for granted the positive effects on quality that Ubuntu users get from Firefox beta testers across all platforms. The Firefox ESR will not benefit from this type of large scale pre-release exposure.

In my opinion, bug 667087 is a perfect example of how the ESR approach can lead to the introduction of bugs (this was a regression which only affected the 3.6 branch).

It is only supported for 1-year (well, 54 weeks, to be more exact)

The Firefox ESR is supported for 54 weeks (9 regular Firefox release cycles), with a 12-week (2-cycle) overlap. This means that it would be inevitable that we would have to upgrade users to a new version of Firefox ESR every year, if we provided this by default. Instead of small incremental changes every 6 weeks, users would be faced with much larger and more obvious changes every year. We believe this is generally worse for most users.

We have been following the new Firefox release process since Ubuntu 11.04, and users seem to have adapted to it quite well. Having just upgraded Ubuntu 10.04 LTS users from Firefox 3.6 to Firefox 10, we know that this scale of update is much more painful – for users and for us.

The web is not static

All of the major browser vendors are pushing new technologies on the web, and existing standards are constantly evolving. It is important that we provide Ubuntu users with a browser which keeps up with this, as users coming from competing platforms expect. In the time since Firefox 4 (which isn’t that dissimilar to the time between 2 ESR versions), Mozilla has added support for things such as the <bdi> element, page visibility API, Mozilla’s full screen API, CSS 3D transforms, CSS font-stretch property, cross-domain textures in WebGL, Web Timing, CSS hyphenation in languages other than English, HTML5 context menus and added a bunch of other improvements to IndexedDB, WebSockets, and canvas. Web developers and Firefox users on other platforms already have access to these features.

We don’t want Ubuntu users to regularly be the last people to have access to evolving technologies on the web, and I don’t think it’s great to say to them “if you want access to the latest web technologies like users on other platforms have, you need to upgrade your entire OS in 6 months or use this unsupported PPA instead”. It wouldn’t be good for Ubuntu if the function or appearance of our users favourite websites ends up being degraded by default on our flagship product, as the web evolves faster than the browser that we are shipping.

Until recently, Ubuntu 10.04 LTS users have already been missing out on major technologies such as CSS transitions (which is used in some places in Google+), WebGL and WebM (available on YouTube) whilst we have been shipping Firefox 3.6 to them. In addition to this, Google have already effectively dropped support for Firefox 3.6, as have Flickr, and we have had reports from other people saying that some online banking sites have already bumped their minimum browser requirements beyond Firefox 3.6.  As the web evolves faster, this type of thing may occur more frequently in the future (the alternatives to this are that web developers give themselves a hard time when trying to adopt new technologies by having to support fallbacks for older browsers, or innovation on the web just stagnates as developers are reluctant to adopt new features).

Over time, Firefox ESR will become slower than Firefox

In the same way that Firefox ESR will become less secure than Firefox, it will also become slower and less resource efficient than regular Firefox due to initiatives such as MemShrink, Snappy and continual work on improving performance in the JS engine.

Performance and memory consumption really matter to users, and these things can affect people’s perception of Ubuntu when they compare browser performance with browsers that are shipped on other platforms.

In addition to this, we offer the latest version of Chromium alongside Firefox in the Ubuntu archive. It would be bad for Mozilla for us to offer an outdated Firefox ESR against the very latest version of Chromium, as the difference in performance between the 2 can significantly influence our users perception of the quality of Mozilla’s product. I’m not the only person who thinks this:

I think it would hurt us competitively if Fedora or Ubuntu shipped ESR, because users or journalists would compare ESR with up-to-date Chrome

Arguments in favour of the ESR

Of course, other people have some good points about why the ESR might be a positive thing. I’ll list some of the more frequent points I hear, and explain why I disagree with them.

The Ubuntu LTS is for enterprise users

This isn’t true. Whilst it is true that enterprise users tend to stick with the LTS release for the longer period of support and less frequent upgrades between OS versions, the LTS is targeted and used by all types of users.

Users who stick with the LTS want stability. Users who want the latest-and-greatest should upgrade between the regular 6-month releases

There are several things wrong with this argument:

  • It assumes that because a user doesn’t want to upgrade their entire OS every 6 months and because they want 3-5 years of support, that they don’t want the latest applications.  I don’t want to upgrade my cell phone more than once every 2 years because it is a pain to adapt to a new device, but I certainly do want to be offered the latest apps on it for the time that it is supported.
  • It assumes that LTS users choose to stay on the LTS.  In fact, when somebody installs the LTS, we will only offer LTS – LTS upgrades for them unless they change a setting in the Updates tab of the Software Sources settings.
  • It assumes that the Firefox ESR provides more stability than the regular Firefox release, and that we won’t get stability from shipping regular Firefox releases.  I’ve already explained above why I don’t think this is the case.
  • It assumes that stability is all that is required to satisfy LTS users.  The reality is a lot more complicated than this.

LTS users actually do seek out the latest software.  As the maintainer of the Firefox Beta PPA and the (now retired) Firefox Stable PPA, I have some interesting download statistics for these PPA’s:

  • Ubuntu 10.04 LTS users are consistently the second highest consumer of the Firefox Beta PPA.  In fact, the number of downloads from Ubuntu 10.04 LTS users is around the same as (and sometimes exceeds) the number of downloads from Ubuntu 10.10 and Ubuntu 11.04 combined.  Note that the highest consumer is always the most recent supported release.
  • The last upload to the Firefox Stable PPA (9.0.1, which was also uploaded to lucid-proposed) was downloaded by 3-times as many users on Ubuntu 10.04 LTS as it was from users on Ubuntu 10.10.

Also, I accidentally introduced a packaging bug in to our daily builds last week which temporarily broke upgrades for daily build users on Ubuntu 10.04 LTS and Ubuntu 11.04.  To my surprise, we got a bug report from a 10.04 user within minutes of the broken packages being published. We then got a fairly steady stream of bug reports from 10.04 users until the packages were fixed.  In total, we had 7 bug reports from Ubuntu 10.04 LTS users, and 1 bug report from an Ubuntu 11.04 user.  Prior to this, I had always made the assumption that Ubuntu 10.04 LTS users would be the smallest consumer of Firefox daily builds, but I may have to reevaluate this view now.

Ok, it’s difficult to read too much in to this relatively small amount of data and I’m not sure how much it really proves.  In any case, I think you’ll find that the LTS users aren’t quite as conservative as some people make them out to be.

The LTS should be stable, secure, supported, predictable

The regular Firefox releases are more secure than the ESR, will be just as stable (with the significantly larger audience of beta testers) and are better supported. The 6 weekly releases are also predictable.

Of course our flagship product needs to be stable, secure and supported. But, it needs to be much more than this too.

Addons break between releases

Whilst this was problematic in the early stages of the rapid release process, this isn’t as much of a problem now. Starting in Firefox 10, most addons are compatible by default (the exceptions are themes and addons with binary components). Prior to this, addon compatibility has been regularly exceeding 95% before each new Firefox release (for the top 95% of addons which were compatible with the previous Firefox version).

Soooooo…..

I hope this answers some of your questions about why Ubuntu is not shipping the Firefox ESR by default. Of course, I’m more than happy to listen to peoples concerns.

It is entirely possible that we might provide a Firefox ESR build for people who are managing large deployments of Ubuntu, although the details of this aren’t decided yet. However, this isn’t going to be the default browser for our LTS. If it existed, it would be shipped in a PPA (much like we have been doing for the Firefox Stable PPA), and we would have to be clear to users that it wouldn’t receive the same level of support we give to the regular Firefox versions.

Thank you for reading :-)

Retiring the Firefox Stable PPA

With the announcement that users of Ubuntu 10.04 LTS and Ubuntu 10.10 will be upgraded to the latest version of Firefox, we are going to be retiring the Firefox Stable PPA.

Since the release of Firefox 4 last year, we have been providing this PPA so that Ubuntu 10.04 LTS and Ubuntu 10.10 users didn’t have to miss out on the new features, performance improvements, security enhancements and improved compatibility with newer web technologies provided by the latest releases of Firefox.

The next version of Firefox (Firefox 10, to be released on Tuesday of this week) will be delivered as a security update to all Ubuntu releases – hopefully by the end of the week. This means that users will receive the update without having to do anything.

I have no plans to upload the next release to the Firefox Stable PPA.

If you’re aware of any online resource (eg, on the Ubuntu Wiki, Ubuntu Forums, Ask Ubuntu or any other site providing help to Ubuntu users) which recommends to use this PPA to receive the latest version of Firefox, I would appreciate you taking the time to either correct this information (or removing it, if more appropriate).

The woes of air travel

As I return from a week at the Canonical Platform Rally, I’m reminded of the many things I dislike about air travel. Note that I dislike** air travel in general – I find it stressful and unpleasant so this post is not going to be very positive. I’m fairly sure I’m not the only person to feel like this though.

Anyhow, here’s the things that annoy me the most about flying:

  • Unsuitably sized cabin baggage
    I thought that airlines had strict guidelines on what size baggage you can carry in to the cabin, but it seems to me that most of them don’t enforce this and most passengers don’t really care. I consider my laptop and its case to be reasonably sized, and I probably wouldn’t carry on anything much larger than this. I consider something the size of my suitcase (which I check in) to be oversized, yet it’s fairly normal to see cabin baggage of this size.

    I travelled on a Boeing 737 from Budapest yesterday. On this aircraft, there is one overhead luggage compartment for every 2 rows of 3 seats (that’s 6 passengers). Imagine my surprise when I got to my seat yesterday to find that the overhead compartment had been completely filled by 2 fairly large suitcases. Yep, that’s right – 2 people filled the luggage compartment, leaving the other 4 passengers to either cram things underneath the seat in front, find another luggage compartment or stow their luggage somewhere at the back of the aircraft. Of course, the second option was not really viable as most overhead compartments were already filled with similarly sized suitcases rammed in to them. I stowed my laptop and coat underneath the seat in front of me, leaving myself with even less leg room than I would normally get. Not happy…

  • Passengers boarding late
    Delaying a flight can have consequences for your fellow passengers – especially those with short connecting flights. The departure area in the airport is full of information displays, and there are regular PA announcements. It is not usually all that difficult to arrive at your gate for boarding on-time.

    My outgoing flight from Birmingham to Budapest involved a short connection in Amsterdam. After most passengers had boarded the aircraft, we sat around waiting for 2 more passengers to board. After around 10 minutes, the cabin crew announced that we would be delayed whilst they removed the missing passengers luggage from the aircraft, at which point they finally boarded! However, by the time we got in the air we were more than 10 minutes behind schedule, making my connection in Amsterdam even more stressful.

    Of course, some passengers might miss PA announcements due to the next point….

  • Talking over PA announcements
    Some groups of people feel that PA announcements at the airport are their cue to increase the volume of their conversation. However, this isn’t the case. Most of your fellow air travelers don’t want to listen to how much you drank last night, how many girls you slept with, or whatever else large groups of young men like to talk about in public. Most of them would actually like to hear the announcement instead.

    My connecting flight from Amsterdam to Birmingham was delayed yesterday due to a technical fault with the aircraft, which meant that we had to move to another gate where we could board a different aircraft. However, I didn’t hear the new gate number in the announcement due to the group of people next to me who would not STFU and insisted on talking loudly over every announcement being made.

    Fortunately, it was fairly easy to just follow everybody else to the correct gate.

  • Not being prepared at security
    Ok, first time air travelers may not be fully aware of procedures at the security check in the airport, so I’ll let them off. I’m sure that most of the people I see at airports are not first time travelers though, so why, when the queue for security is already large and slow moving, do people not take measures to prepare themselves whilst waiting in the queue? This can save some time and make the whole process work a little more efficiently. For me, this means placing the contents of my trouser pockets in to my hand luggage, removing my laptop from its case and removing my jacket before I reach the security check. I don’t carry any liquids and nor do I wear a belt.

    Listening to other peoples music
    This isn’t really specific to air travel, and seems to affect most forms of public transport that I’ve been on. Earphones are designed to restrict sound to the listener using them. So, why do some passengers feel it necessary to increase the volume to such a level that nearby passengers have to listen to their music? Of course, not only are they annoying their fellow passengers, they are also damaging their hearing.

    Anyway, those are some of the things that I dislike about air travel. I felt like I needed to get that off my chest. I’m not actually this miserable by the way :)

    ** “dislike” is a very kind way of putting it

    New addition coming to the family

    So, it seems that we’re getting a new addition to our family, and Ruby is going to have a little brother or sister:

    I actually found this out whilst I was at UDS in Orlando (perfect timing!), and have been waiting to tell people ever since.

    He / she is due to arrive during the first week of July – I can’t wait!

    Ubuntu, Firefox updates, beta and why we need you

    Everyone reading this is probably already well aware of Mozilla’s (now not-so) new 6-week release cycle for Firefox, and that we (Ubuntu) have been delivering these new releases to Ubuntu users since 11.04.  We’re going to continue to do this for the next LTS, Ubuntu 12.04, and with the big drive for increased quality for this cycle, we need to ensure that this quality is maintained for the 5 years that the LTS is supported.  The Firefox release process obviously brings some additional challenges in this area that we don’t have for other applications in Ubuntu.

    Stable release updates (SRUs)

    Traditionally for other applications, all individual changes landing in to a stable release are reviewed for quality and suitability.  All changes go through the “proposed” pocket where they can be tested by a limited audience for 7 days before being released to the general user base.  All individual changes generally have a test case that users of the proposed pocket can follow.  This process is documented here

    Firefox updates

    It isn’t realistic for us to review every individual change in each new Firefox release, nor is it realistic for us to write a test case for each individual change and ask a small number of users to verify them.  It is also not normally appropriate to delay security updates for 7 days whilst we wait for feedback from proposed testers.  Because of this, we haven’t used the traditional SRU method to deliver Firefox updates in the past.

    So, what’s the problem now?

    With the new 6-week releases introducing more than just security and stability fixes, we need to rethink how we test browser updates.  The SRU process doesn’t scale well to browser updates for a few reasons:

    • We can’t wait 7 days after the official Mozilla release to ship a security update
    • The SRU process is great for testing self-contained, specific and isolated fixes with a well defined test-case, but I would question the value of dropping a browser update in to the proposed pocket every 6 weeks, asking users to run it to “see if it works ok” and then waiting for one user to wave their hand on a bug report to say that they started it and it seems to run (eg, this comment on Firefox 5 tracking bug)
    • A few days testing every 6 weeks isn’t long enough. In fact, 7 days is nowhere near long enough in my opinion.

    Well, how do Mozilla handle this?

    Each Firefox release has 6 weeks of development, followed by 12 weeks of stabilization whereby the release is tested by a community of enthusiastic testers.  This process can be visualized by the following graphic, borrowed from here.

    You can see that there are 4 channels running in parallel (these are called nightly, aurora, beta and release), and each Firefox version spends 6 weeks in each channel before moving on to the next one in the release train.  Features land on the nightly channel, whereas aurora and beta are (meant to be) focused solely on stabilizing these new features.  Aurora is less stable than beta, and is intended for users who don’t mind sacrificing a little bit of stability for the newest innovations.  The beta channel is intended for people who want to try out new features in the next version of Firefox, but don’t want to sacrifice the stability that they are used to.

    What pops out at the end of the 6 week beta cycle is the next Firefox release.

    Ok, so what’s the point in this blog post?

    With each Firefox release spending 6 weeks in the beta channel, there is no point in wasting all of this time by using only 7 days of it for testing in Ubuntu.  To make the best use of this 6 weeks we provide our own beta PPA for Ubuntu users, which tracks the upstream beta channel.  Builds are provided for all Ubuntu releases.  This PPA is the Firefox equivalent of the proposed pocket for other Ubuntu packages, with the following extra advantages:

    • It provides 6 weeks of testing as opposed to 1 week
    • Users can opt in to it and use it continuously as their default browser.  After 6 weeks, they will just get the next beta and the whole cycle starts again.

    Some additional information about this PPA:

    • It is still possible to send crash reports directly to upstream, just like you can with the version in the official archive.
    • It is possible to report bugs to Launchpad in the usual way (using Apport).
    • We provide all associated supporting packages, to make the user experience as smooth as possible.
    • It’s supported by the Ubuntu developers who provide Firefox for you.

    Don’t be afraid of the “beta” name – these builds are much closer to RC quality than beta, and experience so far has shown this to be the case even at the start of each 6 week beta cycle.

    So, what can I do to help?

    Well, first of all, start running the Firefox beta if you aren’t already.  If you’re on the development release of Ubuntu, then you are already running it.  If you aren’t, then just run:

    sudo add-apt-repository ppa:mozillateam/firefox-next && sudo apt-get update && sudo apt-get upgrade

    That’s all there is to it! Now you can start enjoying the next Firefox release :-)

    You can also help by blogging and tweeting about this, and telling your friends and family about it.

    B…b…b…but, I prefer to stick to official packages rather than use PPA’s, and I can still report bugs and help you if I stick to the release build

    Yes, of course you can. But it’s way too late to be finding bugs and regressions in the official release build – by this point, we’ve already shipped it to our users as a security update. It’s much more helpful for people to find and report regressions before it ends up in the archive as the next security update.

    I generally believe that anybody involved with Ubuntu Development in any capacity who runs Firefox should at least be using the beta releases all of the time.