Why Ubuntu is not using the Firefox ESR

Recently, a thread has appeared on the ubuntu-desktop mailing list asking why Ubuntu (or specifically, Ubuntu 12.04 LTS) is not using the Extended Support Release of Firefox by default. I’ve also been asked this a few times on IRC over the last few weeks (from people inside and outside of Canonical), so I just wanted to clarify the reasoning for this, and why I think that our choice will offer the best experience for Ubuntu users.

Arguments against the ESR

Aside from the fact that offering the Firefox ESR by default to Ubuntu users would make our good friends at Mozilla unhappy (and from my perspective, we have a pretty good relationship with them. I certainly don’t want this to change), here are some reasons why I think that shipping the Firefox ESR by default would be bad for Ubuntu users. Some of these are points from the ESR proposal, and some of them are my own thoughts.

Over time, Firefox ESR will become less secure than Firefox

The ESR proposal states that:

Mozilla will backport security bugs qualified as “Critical” and “High” to the ESR where feasible (there may be cases where a backport cannot be applied with reasonable effort, and those cases are expected to be exceptional)

This means that people running the Firefox ESR may have to wait for the next major release to receive fixes for security bugs with sg:moderate, sg:low or sg:dos severity ratings. What is worse is that ESR users may also miss out on some sg:critical or sg:high rated bugs, where it is just not feasible to backport the fix to the ESR branch.

In addition to this, Firefox ESR users will have to wait longer than regular Firefox users for proactive security improvements such as support for the iframe sandbox attribute, CA pinning, the Mixed Content Blocker or any other new security features / improvements.

Because of this, the Firefox ESR will always become less secure than the regular Firefox releases.

Ubuntu 10.04 LTS users have already had to wait longer than Firefox users on other platforms for new security and privacy features such as Content Security Policy, HTTP Strict Transport Security and Do Not Track. We don’t want Ubuntu to lag behind other platforms in the future.

The risk of introducing bugs is greater with Firefox ESR

There is a common misconception that when a piece of software receives only reactive security fixes, it is the safest option for users and that the risk of breakage is minimal with this approach. In reality, this isn’t exactly true. There is always risk associated with backporting any form of code change from one branch to an older branch, and this risk increases as:

  • The 2 branches diverge further apart, making the backport less straightforward
  • The amount of testing exposure decreases

Clearly, the Firefox ESR will be affected by both of these factors.

The regular Firefox releases pass through the beta channel for 6 weeks before release, where they are exposed to a large community of users who are using the beta as their day-to-day browser. And, whilst the Linux testing community is relatively small (I would like to grow this though), we mustn’t take for granted the positive effects on quality that Ubuntu users get from Firefox beta testers across all platforms. The Firefox ESR will not benefit from this type of large scale pre-release exposure.

In my opinion, bug 667087 is a perfect example of how the ESR approach can lead to the introduction of bugs (this was a regression which only affected the 3.6 branch).

It is only supported for 1-year (well, 54 weeks, to be more exact)

The Firefox ESR is supported for 54 weeks (9 regular Firefox release cycles), with a 12-week (2-cycle) overlap. This means that it would be inevitable that we would have to upgrade users to a new version of Firefox ESR every year, if we provided this by default. Instead of small incremental changes every 6 weeks, users would be faced with much larger and more obvious changes every year. We believe this is generally worse for most users.

We have been following the new Firefox release process since Ubuntu 11.04, and users seem to have adapted to it quite well. Having just upgraded Ubuntu 10.04 LTS users from Firefox 3.6 to Firefox 10, we know that this scale of update is much more painful – for users and for us.

The web is not static

All of the major browser vendors are pushing new technologies on the web, and existing standards are constantly evolving. It is important that we provide Ubuntu users with a browser which keeps up with this, as users coming from competing platforms expect. In the time since Firefox 4 (which isn’t that dissimilar to the time between 2 ESR versions), Mozilla has added support for things such as the <bdi> element, page visibility API, Mozilla’s full screen API, CSS 3D transforms, CSS font-stretch property, cross-domain textures in WebGL, Web Timing, CSS hyphenation in languages other than English, HTML5 context menus and added a bunch of other improvements to IndexedDB, WebSockets, and canvas. Web developers and Firefox users on other platforms already have access to these features.

We don’t want Ubuntu users to regularly be the last people to have access to evolving technologies on the web, and I don’t think it’s great to say to them “if you want access to the latest web technologies like users on other platforms have, you need to upgrade your entire OS in 6 months or use this unsupported PPA instead”. It wouldn’t be good for Ubuntu if the function or appearance of our users favourite websites ends up being degraded by default on our flagship product, as the web evolves faster than the browser that we are shipping.

Until recently, Ubuntu 10.04 LTS users have already been missing out on major technologies such as CSS transitions (which is used in some places in Google+), WebGL and WebM (available on YouTube) whilst we have been shipping Firefox 3.6 to them. In addition to this, Google have already effectively dropped support for Firefox 3.6, as have Flickr, and we have had reports from other people saying that some online banking sites have already bumped their minimum browser requirements beyond Firefox 3.6.  As the web evolves faster, this type of thing may occur more frequently in the future (the alternatives to this are that web developers give themselves a hard time when trying to adopt new technologies by having to support fallbacks for older browsers, or innovation on the web just stagnates as developers are reluctant to adopt new features).

Over time, Firefox ESR will become slower than Firefox

In the same way that Firefox ESR will become less secure than Firefox, it will also become slower and less resource efficient than regular Firefox due to initiatives such as MemShrink, Snappy and continual work on improving performance in the JS engine.

Performance and memory consumption really matter to users, and these things can affect people’s perception of Ubuntu when they compare browser performance with browsers that are shipped on other platforms.

In addition to this, we offer the latest version of Chromium alongside Firefox in the Ubuntu archive. It would be bad for Mozilla for us to offer an outdated Firefox ESR against the very latest version of Chromium, as the difference in performance between the 2 can significantly influence our users perception of the quality of Mozilla’s product. I’m not the only person who thinks this:

I think it would hurt us competitively if Fedora or Ubuntu shipped ESR, because users or journalists would compare ESR with up-to-date Chrome

Arguments in favour of the ESR

Of course, other people have some good points about why the ESR might be a positive thing. I’ll list some of the more frequent points I hear, and explain why I disagree with them.

The Ubuntu LTS is for enterprise users

This isn’t true. Whilst it is true that enterprise users tend to stick with the LTS release for the longer period of support and less frequent upgrades between OS versions, the LTS is targeted and used by all types of users.

Users who stick with the LTS want stability. Users who want the latest-and-greatest should upgrade between the regular 6-month releases

There are several things wrong with this argument:

  • It assumes that because a user doesn’t want to upgrade their entire OS every 6 months and because they want 3-5 years of support, that they don’t want the latest applications.  I don’t want to upgrade my cell phone more than once every 2 years because it is a pain to adapt to a new device, but I certainly do want to be offered the latest apps on it for the time that it is supported.
  • It assumes that LTS users choose to stay on the LTS.  In fact, when somebody installs the LTS, we will only offer LTS – LTS upgrades for them unless they change a setting in the Updates tab of the Software Sources settings.
  • It assumes that the Firefox ESR provides more stability than the regular Firefox release, and that we won’t get stability from shipping regular Firefox releases.  I’ve already explained above why I don’t think this is the case.
  • It assumes that stability is all that is required to satisfy LTS users.  The reality is a lot more complicated than this.

LTS users actually do seek out the latest software.  As the maintainer of the Firefox Beta PPA and the (now retired) Firefox Stable PPA, I have some interesting download statistics for these PPA’s:

  • Ubuntu 10.04 LTS users are consistently the second highest consumer of the Firefox Beta PPA.  In fact, the number of downloads from Ubuntu 10.04 LTS users is around the same as (and sometimes exceeds) the number of downloads from Ubuntu 10.10 and Ubuntu 11.04 combined.  Note that the highest consumer is always the most recent supported release.
  • The last upload to the Firefox Stable PPA (9.0.1, which was also uploaded to lucid-proposed) was downloaded by 3-times as many users on Ubuntu 10.04 LTS as it was from users on Ubuntu 10.10.

Also, I accidentally introduced a packaging bug in to our daily builds last week which temporarily broke upgrades for daily build users on Ubuntu 10.04 LTS and Ubuntu 11.04.  To my surprise, we got a bug report from a 10.04 user within minutes of the broken packages being published. We then got a fairly steady stream of bug reports from 10.04 users until the packages were fixed.  In total, we had 7 bug reports from Ubuntu 10.04 LTS users, and 1 bug report from an Ubuntu 11.04 user.  Prior to this, I had always made the assumption that Ubuntu 10.04 LTS users would be the smallest consumer of Firefox daily builds, but I may have to reevaluate this view now.

Ok, it’s difficult to read too much in to this relatively small amount of data and I’m not sure how much it really proves.  In any case, I think you’ll find that the LTS users aren’t quite as conservative as some people make them out to be.

The LTS should be stable, secure, supported, predictable

The regular Firefox releases are more secure than the ESR, will be just as stable (with the significantly larger audience of beta testers) and are better supported. The 6 weekly releases are also predictable.

Of course our flagship product needs to be stable, secure and supported. But, it needs to be much more than this too.

Addons break between releases

Whilst this was problematic in the early stages of the rapid release process, this isn’t as much of a problem now. Starting in Firefox 10, most addons are compatible by default (the exceptions are themes and addons with binary components). Prior to this, addon compatibility has been regularly exceeding 95% before each new Firefox release (for the top 95% of addons which were compatible with the previous Firefox version).

Soooooo…..

I hope this answers some of your questions about why Ubuntu is not shipping the Firefox ESR by default. Of course, I’m more than happy to listen to peoples concerns.

It is entirely possible that we might provide a Firefox ESR build for people who are managing large deployments of Ubuntu, although the details of this aren’t decided yet. However, this isn’t going to be the default browser for our LTS. If it existed, it would be shipped in a PPA (much like we have been doing for the Firefox Stable PPA), and we would have to be clear to users that it wouldn’t receive the same level of support we give to the regular Firefox versions.

Thank you for reading :-)

  • Jonas

    Your “The web is not static” argument doesn’t hold water, I think: People in the real world need to target IE9, or worse, 8, which is a far lower common denominator than any change that happened between firefox 3.6 and 10. Ff 3.6 is still a more modern browser than IE9. So there’s NO WAY web sites will depend on the newest features or that Ubuntu users would be “the last people to have access to evolving technologies” — because unfortunately we are not living in a universe with no Internet Explorer.

    There are many other reasons to not use the ESR which are good reasons, so this doesn’t change anything, but for correctness, I felt I should point this out.

    • Chris Coulson

      You make a good argument with IE9, and I have to admit that I didn’t consider IE that much when I was writing this (probably because I haven’t used it in years). In any case, I don’t think that’s a good enough excuse for us to offer a sub-par experience too

  • Malizor

    So ESR:
    - Are less secure and reliable than regular releases
    - Are bad for Mozilla image

    So, what’s the point of ESR?

    Are you sure you are not caricaturing a bit?

    • Chris Coulson

      No, I am not.

      The point about it gradually becoming less secure is valid and is even listed as a risk on the final ESR proposal: https://wiki.mozilla.org/Enterprise/Firefox/ExtendedSupport:Proposal

      Where did I say that the ESR is bad for Mozilla’s image? I said that it would be bad if we offered it by default alongside the latest version of one of their competitors products, which is very different. Clearly there are some valid use-cases for the ESR, but I’m just stating why I don’t think these apply to ordinary Ubuntu users.

    • http://jeremy.bicha.net/ Jeremy Bicha

      The point of ESR is that IT leaders at enough large-scale Firefox deployments complained about Firefox’s development cycle so something had to be done. It doesn’t necessarily mean that ESR was actually a good idea, or that ESR is better for every large-scale Firefox deployment.

      • Alan Milnes

        Actually Rapid Release is the crazy idea, we just tried to inject a bit of sanity into the discussions :-)

  • Martin Pitt

    Thanks for the nice summary! Indeed I think that what many users want is a stable OS/platform with up-to-date applications. Now, we do not generally update application packages in LTSes to also suit users which actually do want stabily (also in terms of UI and workflows).

    However, the browser stands out here because it is not really an application by itself (when you are offline, it’s really quite useless), but its much more like a “window into the web”, and as such we better make sure that users always get the most appropriate “window”, which the current Firefox version is much more than an one or two year old version.

  • http://twitter.com/jspaleta Jef Spaleta

    I think the discussion around this issue has more to do with the connotative dissonance in the perception of what the LTS release is meant to provide as to what is can achievably provide. This might be a good time to take stock of how users currently view the LTS deliverable and perhaps update the LTS strategic messaging to correct some misconceptions.

    -jef

  • Guest

    Chris, you mentioned the “Firefox Beta PPA and the (now retired) Firefox Stable PPA”. The funny thing is that the stable ppa is still active whereas the beta ppa has a warning “Publishing has been disabled for this archive.” A mistake?

    • http://www.chriscoulson.me.uk/ Chris Coulson

      No, that’s not a mistake. We turn off publishing before we do new builds so that your translations don’t break if one architecture finishes building before the other :)

      I haven’t disabled the stable PPA yet because Launchpad keeps timing out.

      • http://www.chriscoulson.me.uk/ Chris Coulson

        Oh, disabling the stable PPA works now, so I’ve just done that. I’ve also turned on publishing now for the new beta as well.

  • Anonymous

    Having read both sides of the argument and understanding the perils of ESR, for us it is a better option.

    My users do not need the latest and greatest. They are working, not playing games online (or at least I hope not !). If they want the latest & greatest, go home and get it. I don’t want my users asking me every 6 weeks where this, that, or the other feature or menu has gone. Had enough of it already.

    Ironically, we disabled the update channel and went back to 3.x shortly after they started rapid release as I was fed up with answering user questions about the new versions. It was taking up a lot of my time. Since then, my users have not once come back and told me they have had any problems on web sites.

    I haven’t found out where Ubuntu are on Thunderbird ESR, but I guess the same arguments will apply. This is even more important to us, and therefore just adds another nail to the Ubuntu coffin.

    Suffice to say that as I have been given no CHOICE on the matter by Ubuntu ( I have read the thread as to why they will not maintain two versions), I have voted with my feet and after a number of years using it, I have left and will not be going back.

    The Unity debacle just compounded the whole problem.

    Why is it that the more successful a company, the less they listen ? Seems inversely proportional to me. Linux is great because of the choice, but Ubuntu seems to be forgetting that and becoming more dictatorial every day.

    What do they say ? Power corrupts, and absolute power corrupts absolutely ???

  • Jonas

    I get your arguments, even so I don’t share all of them. If Ubuntu doesn’t want to take ESR into official repos it’s okay with me. But, please, could you just make it an easy-to-maintain option for those who want to use it? I would be more than happy with an ESR PPA.

    • http://www.chriscoulson.me.uk/ Chris Coulson

      There aren’t any plans to do this at the moment, as there just isn’t enough demand for it to justify the additional work of maintaining yet another Firefox branch

      • Jonas

        no demand…? honestly, the net is crawling with dire questions for exact that PPA. I’m working in a small business office. If I can’t maintain our Ubuntu laptops with an easy automatic update process we’re stuck again at where we were before ESR.

        Though Thunderbird is our more pressing concern. We make excessive use of PGP and last year we lost Enigmail at least one time due to that current Mozilla Update Madness. I really hope someone’s putting those Firefox/Thunderbird ESR PPA’s on – we really need it.

  • Prabhjot

    So where does that leave Firefox-Stable. What has been planned for users who do like to have atleast some testing been done before upgrading?

  • Guest

    I like how at the end you break out large deployments vs small deployments. We currently have about 4,000 LTS clients deployed, and the rate of change and volume of data associated with change matters as much as the actual change does, so I think your assessment of why (and which) people change is spot-on, but I also believe there may be some of us who make alternate choices based on mitigating some rate or volume of change over time, especially when we interact with content vendors who don’t innovate with any perceivable frequency. ;)

  • Alan Milnes

    I’m still not convinced, it’s a pity ESR is only 54 weeks but surely LTS is an obvious candidate for this version. What your starts tell me is there are lots of people on LTS who shouldn’t be there ….

  • Bleck

    One main argument for ESR is “extensions”. I don’t only want my thunderbird/firefox cores to be fucntionnaly stable I also *need* the extensions I’m using. I can’t put at risk the functionnalities they provide, every 6 weeks ! Mozilla people can say what they want about supposed extensions ascending compatibility since #10 version, this is not the reality. Some critical (to me) extensions are yet maintained on an ESR basis. That sounds logical since extensions devoloppers don’t all want nor can publish a new release every 6 weeks.

    What do FF and TB worth without extensions, compared to competitors ? Not much. Brillant functionnalities are worthless if you destroy your ecosystem. That’s what Mozilla acknowledged introducing the ESR. But it’s true they promise they won’t make it seriously, in terms of security. Saying “you have to choose between security and stability” is just crazy !

    Such a crazy policy from Mozilla team transfers the choice to the distributions managers and/or the users. Whatever choice they make they are sure it will be a bad choice. Once aknowledge, it’s difficult to say which is the best bad choice…

  • Luciana

    Yeah, this is madness. Firefox and every browser today is going crazy. And they are behaving like an antivirus: if you are not update, you are in danger. Bullshit. A good well designed browser don’t need to be updated so often. It’s all marketing and to make easy to spy and track people in a non-sense way (adding every day more “features” that ‘phones home’, to Mozilla o Google, o whatever, all in the sake of being updated). I’m tired of see how the internet is “FORCING” people to update browsers, breaking websites “on purpose”. What I mean is that many webmasters are using new browser technology TO FORCE this update (even Firefox 3.6 is now outdated, c’mon!!). They all should be thinking in old browsers too (and old machines with old OS), and how to maintain compatibility instead (like old good days). So again, this is insane. And this is the true no matter that they say. Period.

  • Stanislav German-Evtushenko

    Points are very clear but, as many times already mentioned, addons compatibility is a problem. Firefox 18 has just come out and been automatically updated on my Work PC (Ubuntu 10.04) and brief addon just stopped to work. Okay, this is just an RSS reader plugin but imagine a company with 1000 Ubuntu PCs and one day they have got a problem with an business critical addon on all of their PCs.

    I would suggest to ship Ubuntu with Firefox ESR by default and at the same time allow to install the latest version of Firefox from the same repository, for example:
    apt-get install firefox-esr
    apt-get install firefox-latest-stable

    • Jonas

      I second! Why don’t ship Ubuntu X.04 (LTS) with ESR by default and Ubuntu X.10 with latest-stable by default, with having the possibility to change. This shouldn’t be to difficult and would match the supposed needs of theirs respective users.

  • guest

    So don’t use Ubuntu LTS instead go use Fedora or something else that makes sure I HAVE to update so that the changes won’t be so drastic. Damn, I was really hoping to use Ubuntu LTS but as Chris said
    ESR a long term implimentation of software will become less secure than the regular updates. I used to use 10.04 LTS and it did seem like programs really weren’t upgraded at all even though I heard about security threats.
    Apparently, something long term will introduce bugs even more than using something that is rapid release. 12.04 LTS is going to be supported for 5 years! Imagine the changes people will have to face by then. New email client ? New media player ? New web browser ? Default search engine ? Boot loader, dev manager, etc…After five years of being used to something?! Better to change it every six or so months so that people don’t get too used to something. Not that I can notice any dang visible changes from the last fifteen versions of Firefox but I do notice many differences in Ubuntu so I guess this applies for Ubuntu even more so.
    I remember in 10.04 LTS you wouldn’t see new tech just security updates and whatnot so I guess that I wouldn’t get new improved technologies on the web and off that is constantly evolving. It’s cute to be in bed with Google, nay ? I can’t get the HTML5 videos to work on any damn computer without Flash. On Firefox at least. Windows or Linux distros alike. Only way I can do that is to jump through hoops of creating a website and embedding the videos into it then viewing that way. Because if I go to YouTube I’m told I need flash. Yes, I have tried joining the HTML5 trial doesn’t do anything for me.
    Banks…A friend’s bank. I won’t mention which one anyone hoping to check it for security holes and steal some money can just go look at all banks around the world. In this bank they refuse to use anything but IE old and dead versions at that…I sure as hell wouldn’t want to bank there but still that’s your only way of accessing that bank. What does one do there install Wine ? Also I see plenty of banks and other sites still have support for 3.X versions not because they have to and that it’s the users fault for wanting to stay on a lower version but because the developers don’t want to deal with this constant change so that stuff much actually be a bit warped on newer versions. Or newer browsers. Sort of like when one switched from IE to Firefox in the 2.0 era and sites were rendered terribly because no one wanted to fix their code. And like that one bank I mentioned I’m sure they aren’t easily hackable. I guess I could tell my friends to stop using it because it’s unsecure but try telling that to most end users…

    Will Ubuntu also become slower as new versions come out and one stays in LTS ?
    I keep seeing benchmarks saying that Ubuntu keeps getting slower and slower and that Windows actually performs better on similar hardware. So, maybe this is one point for staying LTS than some other distros ?

    I was going to argue that these points are all moot because LTS is for enterprise users but it’s not so forget that. It was also meant for end users and it’s not a grand idea for them to use these unsecure, buggy, neglected releases.

    As you point out:
    The LTS should be stable, secure, supported, predictable

    The regular Firefox releases are more secure than the ESR, will be just as
    stable (with the significantly larger audience of beta testers) and are better
    supported. The 6 weekly releases are also predictable.
    Replace Firefox with Ubuntu and ESR with LTS and weekly to monthly ?

    My add-ons have still broken a large amount of the time since FF 10
    I guess the bottom five percent don’t matter. Thank you for that. :)
    Yes, that did answer some of my questions as to why not to use Ubuntu.

    Oh and to boot LTS won’t get the same level of support as rapid release, cool.

  • lamawithonel

    Why not do a “firefox-esr” package that conflicts with “firefox”?

    Some users want stability. Some users want features. All users deserve choice.