Customizing the user agent string in Oxide

The Ubuntu webbrowser sets its own default user agent string, and also provides a list of overrides which define different user agent strings for specific sites that do broken UA sniffing. This is a concept that we’ve borrowed from Firefox OS.

With QtWebkit, the overriding is performed via the same mechanism that is used for setting the default user agent string – by setting the WebViewExperimental.userAgent property. However, this has a major shortcoming – it provides no means to override the user agent string for specific frames. Say, for example, that you need to provide an override for www.twitter.com, but not for www.example.org. If a page from www.example.org embeds www.twitter.com in an iframe, there isn’t a mechanism to change the user agent string for the child frame.

When overriding the user agent string for a frame, there are 2 places that this needs to happen:

  • navigator.userAgent
  • The User-Agent header in outgoing HTTP requests.

Fortunately, Chromium provides a mechanism by which we can implement the second one – net::NetworkDelegate::OnBeforeSendHeaders(). Unfortunately, this is called on Chromium’s IO thread which makes it difficult to expose as a signal in QML (a QML engine can only execute code on a single thread). The notification is asynchronous, which means we could dispatch an event to the UI thread, emit a signal in QML and then dispatch an event back to Chromium’s IO thread. However, this has shortcomings of its own:

  • A typical page load can generate more than 100 network requests. For example, loading BBC News generated 165 requests when I tested it here. Assuming we have multiple notifications per request, processing all of these on the UI thread in QML could have a noticeable performance impact.
  • Not all notifications we get from net::NetworkDelegate are asynchronous. For example, cookie access permission requests aren’t.

Enter WebContextDelegateWorker

For handling events that happen on Chromium’s IO thread, we have a new class – WebContextDelegateWorker. This is very similar to QML’s WorkerScript – it runs script on a worker thread and provides sendMessage/onMessage API’s for exchanging data with QML on the UI thread. However, it provides a mechanism for scripts to expose entry points that can be called from Chromium’s IO thread (actually, they’re not called from Chromium’s IO thread – they’re called on the worker thread by blocking the IO thread. But, you don’t need to care about that).

To overridethe User-Agent header for specific requests in Oxide, you can do something like the following:

In QML:

WebView {
  ...
  context: WebContext {
    ...
    networkRequestDelegate: WebContextDelegateWorker {
      source: Qt.resolvedUrl("worker.js")
      onMessage: {
        console.log("We set User-Agent on " + message.url);
      }
    }
  }
}

In worker.js:

// Unfortunately the properties of QUrl don't appear to be available in QML,
// so provide a simple regex to extract the host
var re=/^[^:]+:\/\/([^\/]*)\/.*/;
exports.onBeforeSendHeaders = function(event) {
  if (re.exec(event.url)[1] == "www.twitter.com") {
    event.setHeader("User-Agent", "Foo");
    oxide.sendMessage({url: event.url});
  }
}

This doesn’t affect navigator.userAgent though. In order to provide an override mechanism for this, we had to apply a small patch to Chromium to give embedders the opportunity to provide an override value when accessed. We did consider exposing a signal in QML for overriding this, based on the assumption that this would run much less frequently than network notifications. However, in the end we decided to reuse WebContextDelegateWorker, based on the assumption that developers who want to override navigator.userAgent will probably also want to use the same overrides for the User-Agent header. Doing this means that you only need the override list in one script context.

To override navigator.userAgent, the above code becomes:

In QML:

WebView {
  ...
  context: WebContext {
    ...
    networkRequestDelegate: WebContextDelegateWorker {
      source: Qt.resolvedUrl("worker.js")
      onMessage: {
        console.log("We set User-Agent on " + message.url);
      }
    }
    userAgentOverrideDelegate: networkRequestDelegate
  }
}

In worker.js:

// Unfortunately the properties of QUrl don't appear to be available in QML,
// so provide a simple regex to extract the host
var re=/^[^:]+:\/\/([^\/]*)\/.*/;

exports.onBeforeSendHeaders = function(event) {
  if (re.exec(event.url)[1] == "www.twitter.com") {
    event.setHeader("User-Agent", "Foo");
    oxide.sendMessage({url: event.url});
  }
}

exports.onGetUserAgentOverride = function(data) {
  if (re.exec(data.url)[1] == "www.twitter.com") {
    data.userAgentOverride = "Foo";
  }
}

Setting a default string

Oxide doesn’t have a per-WebView property that’s equivalent to WebViewExperimental.userAgent in QtWebkit, as we decided that this would be redundant with the above override mechanism. Instead, the default user agent string is configured by setting the WebContext.userAgent property.

Other uses for WebContextDelegateWorker

WebContextDelegateWorker is also used for overriding the default storage access policy via WebContext.storageAccessPermissionDelegate (in QML) and exports.onStoragePermissionRequest (in the worker). This is currently only used for cookies, but is due to be expanded to local storage, app cache, indexedDB and WebDB. The default storage access policy is controlled by WebContext.cookiePolicy

Leave a Reply

Your email address will not be published. Required fields are marked *