Programs that respect your privacy

The “X-Requested-With” Header

As mentioned is several places on this website and in the app, there are some negative implications of using Android’s WebView to render web pages in Privacy Browser. The purpose of this post is to describe one of these downsides, explain the mitigations that are currently being taken, and lay out thoughts for a more permanent solution down the road.

The Background

When any app uses Android’s WebView to request a web page, WebView attaches an extra header, named “X-Requested-With”, with the app ID. “X-Requested-With” was designed to be used as a flag to mark AJAX (Asynchronous JavaScript and XML) requests. In that sense, WebView’s use of the field for a different purpose is an abuse of the standard and can cause problems for some web pages.

The Problem

One of the web tracking technologies that Privacy Browser is designed to mitigate is browser fingerprinting. Any piece of information the browser sends a web server that makes it stick out from the crowd increases the ability of the web server to uniquely fingerprint the browser. Including the app ID in the header, especially as long as Privacy Browser has a small market share, increases the chance that the total information sent to the server is unique.

For the Standard version of Privacy Browser the app ID is “com.stoutner.privacybrowser.standard”; the Free version is “com.stoutner.privacybrowser.standard”. The following is a log from that shows what information Privacy Browser <= 1.8 transmits to a web server (note that the User Agent has been changed from the default to “PrivacyBrowser/1.0”).

GET / HTTP/1.1||Connection:keep-alive|Cache-Control:max-age=0|Upgrade-Insecure-Requests:1|User-Agent:PrivacyBrowser/1.0|Accept:text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8|Accept-Encoding:gzip, deflate|Accept-Language:en-US|X-Requested-With:com.stoutner.privacybrowser.standard
The Workaround

Google doesn’t want to make it easy to get rid of the “X-Requested-With” header. However, there is a mechanism for replacing header information. This doesn’t allow a program to stop sending the “X-Requested-With” header, but it does allow a program to replace the app ID with a null value. Beginning with Privacy Browser 1.9, the following information will be sent in the headers:

GET / HTTP/1.1||Connection:keep-alive|Upgrade-Insecure-Requests:1|User-Agent:PrivacyBrowser/1.0|x-requested-with:|Accept:text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8|Accept-Encoding:gzip, deflate|Accept-Language:en-US
The Problems with the Workaround

The first problem with this workaround is that sending “X-Requested-With:” is not the same as not sending it at all. There are only going to be a few browsers that send “X-Requested-With:” as a header, so for fingerprinting purposes this workaround is only slightly better than including the app ID.

The second problem is that the technique for overwriting WebView’s headers only works on the initial request to a website. Any dependent requests for resources (like images, CSS files, JavaScript files, etc.) use the default headers. So even in Privacy Browser >= 1.9, the log of a request for a resource will be as follows:

GET /wp-content/themes/twentysixteen/style.css?ver=4.5.3 HTTP/1.1||Connection:keep-alive|Cache-Control:max-age=0|User-Agent:PrivacyBrowser/1.0|Accept:text/css,*/*;q=0.1|Referer:https%3a//|Accept-Encoding:gzip, deflate|Accept-Language:en-US|X-Requested-With:com.stoutner.privacybrowser.standard|If-None-Match:"10d4b-535f10e3fb580-gzip"|If-Modified-Since:Thu, 23 Jun 2016 12%3a18%3a46 GMT

This is a problem for every browser that uses Android’s WebView. For example, Lightning 4.3.3 behaves the same way as Privacy Browser >= 1.9: initial web requests include “X-Requested-With:” and resource requests include “X-Requested-With:acr.browser.lightning”.

The Long-Term Solution

There is a provision in WebView to bypass the default web loading functions by manually acquiring the information using HttpURLConnection and feeding it into WebView. This would allow the complete removal of the “X-Requested-With” header. However, making sure that all possible types of HTTP communication are handled correctly would be complex and the initial implementation would likely result in buggy behavior on some websites.

Another solution is to stop using Android’s WebView and embed a customized WebView in Privacy Browser. This would also allow for the complete removal of the “X-Requested-With” header, but would be a time consuming commitment to maintaining the custom WebView (both from a security and bug perspective).

Long term, it is likely that using a custom WebView will be the best solution for Privacy Browser. The plan is to implement all the privacy functions that are compatible with Android’s WebView (pick the low hanging fruit) and then do a detailed analysis of the best way to move beyond that point.

7 responses to “The “X-Requested-With” Header”

  1. […] Privacy Browser now handles “mailto:” links on websites.  A couple of bugs were also fixed that caused the browser to crash when working with bookmarks with no favorite icon and the navigation drawer to sometimes stop working after viewing a full-screen video.  Progress was made regarding the “X-Requested-With” header, which was covered in more detail in a previous post. […]

  2. […] of implementation imposed by Android’s WebView that were described in the post about the “X-Requested-With” header. Specifically, the DNT header is only sent with the initial URL request, and not with resource […]

  3. Why not just comment out the relevant section of the source, build it under a different package name, and only use the system default as a last-ditch fallback? I’m also not entirely sure why the LD_PRELOAD trick with function overloading like tools like torsocks and proxychains use wouldn’t work here. Am I missing something?

    • The plan is to do exactly that with the creation of Privacy WebView in the 4.x series. But it is a non-trivial endeavor to create a rolling fork of WebView. For example, it is unlikely that F-Droid will be able to easily build a custom version of WebView because it does not use the build system that is commonly used by most Android apps, meaning that it will take effort and likely contributions to the F-Droid source code to be able to distribute a custom WebView through their platform. However, if you are aware of an easy way to do it you are welcome to submit a patch to the code.

    • A user could use a local proxy if they desired, but there are significant shortcomings to doing so.

      1. A local proxy cannot modify HTTPS traffic (the default) unless it unwinds the SSL connection, modifies the header, and reinitializes the HTTPS connection on the other side. That is, the proxy would have to perform a MITM (Man In The Middle) attack. Doing so would require installing a wildcard SSL certificate on the device for all domains on the internet, trusting that certificate, and having the proxy use that certificate for all internet traffic. If you look at examples on the internet of systems that have done things like that it doesn’t often end well from a security perspective.
      2. The two ways to run a local proxy are either to use a local port or to use Android’s VPN capabilities. Using a local port would likely not play well with Orbot, which would break Tor for those wanting to use it. Running as a VPN would eliminate the ability to run another VPN, which many users are already doing.
      3. Running a local proxy as a MITM would mean that Privacy Browser could not see the original certificate presented by the website, which would break SSL Certificate pinning.

      Instead, I plan to fix this the correct way as described at in the 4.x series with Privacy WebView.

Leave a Reply

Your email address will not be published.