Download Problems on Android 9 Pie

There appears to be a bug in Android’s Download Manager on Android 9 Pie that causes it to periodically crash and refuse to download files. New downloads will show up as queued, but will fail to download. I’m assuming that at some stage Google will release an update that fixes this, but in the meantime users can force close the Download Manager process, which will temporarily resolve the issue. Go to Apps & notifications, click the option to See all apps, tap the options menu in the upper-right corner, select Show system, select Download Manager from the list, then tap on Force stop. Note that it may also be necessary to clear Download Manager’s cache or storage in some cases.

Privacy Browser 2.13

Privacy Browser 2.13 has been released. There is now an activity for exporting and importing settings and bookmarks. Both Privacy Browser and Privacy Browser Free use the same format, so backed up settings can be used to migrate between the two. There is a page that has more detailed information on the database format. Note that in future releases it will be possible to encrypt the exports and automate the export and import process, which will allow for syncing of settings across devices.

This release adds the Read Storage permission, which allows Privacy Browser to import files from public directories. On Android Marshmallow (6.0) and newer, this permission is only available if the user grants it. If it is denied, app directories can still be used. This is the last permission I am currently planning on adding to Privacy Browser.

This release switches the default URL scheme from HTTP to HTTPS. For URLs that are entered into the URL bar at the top of the screen, in the past, if no protocol was specified, http:// would be added to the beginning of the URL. Now, https:// will be added. This is feasible because most websites now are offered over HTTPS. Users can still visit an HTTP website by specifying http:// in the URL.

The blocklist menu items now display the number of blocked items for each list, which are updated live.

The Refresh menu item now becomes a Stop button when a website is loading. If additional app bar buttons are displayed, it is available as an X on the app bar.

User agent and night mode controls are now available from the options layout menu.

A bug was fixed, introduced in Privacy Browser 2.12, that caused the system bars to disappear after viewing a full-screen video. The Clear Data options menu item was fixed to be ghosted when all submenu items are ghosted (previously broken on Oreo [Android 8] and newer). The size problems with the Waiting for Orbot message were partially fixed. Note that in the future I plan to redesign the Waiting for Orbot message to use a dialog.

With this release the target API was bumped to 28 (Android 9 Pie). An updated Italian translation was provided by Francesco Buratti and an updated Spanish translation was provided by Jose A. León. The Russian translation was also updated.

The next major release of Privacy Browser will have the option to encrypt exported settings.

 

New Default Homepage and Search Engine

With the release of Privacy Browser 2.12, the default homepage and search engine has been switched to Searx.me. This only applies to new installs of Privacy Browser. Existing users who upgrade will keep whatever their current settings are until they manually change them.

The default Tor homepage and search engine has been changed to http://ulrn6sryqaifefld.onion/, which is a Searx instance operated by the same organization that runs Search.me.

There are several reasons this change was made. I will list them beginning with the most significant.

  1. DuckDuckGo has a tracker on the home page.
  2. DuckDuckGo tracks the ads you click on before redirecting you. You can see this in the screenshot below.
  3. DuckDuckGo’s .onion site doesn’t work with JavaScript disabled and they seem disinterested in fixing it.
  4. DuckDuckGo requires workarounds to function with both JavaScript disabled or enabled.

In looking for replacements I settled on Searx for the following reasons.

  1. Searx doesn’t load any trackers.
  2. Searx doesn’t track any of the links you click on.
  3. The entire system that runs Searx is open source software released under the AGPLv3+ license.

You can host a Searx instance yourself or use one of the many public instances. I chose to go with Searx.me for the default in Privacy Browser because it is the most commonly used instance and has a .onion site. Searx.me is managed by Adam Tauber, who is the principal developer of Searx. There is no way to independently verify that the code running on his server matches the code in the Searx repository, but if it does then the system truly does not track you. Even with that limitation, there is no other search engine I have found that comes as close to the ideals of Privacy Browser.

Note that the .onion site does not offer HTTPS. Proponents of Tor will tell you that they don’t need HTTPS because the encryption is handled by the Tor system. But given that every indication is that Tor has been compromised by the NSA, I would prefer not to relay on the encryption of the Tor protocol, but rather run HTTPS across Tor even for .onion sites.

A final though about default search engines and homepages in Privacy Browser. Most major browsers get kickbacks from search engines for making them their default.  Mozilla’s revenue totals hundreds of millions of dollars per years in such kickbacks. This alters their behavior, such that they select a search engine based on how much they will get paid, not on what is best for their users. They also don’t do some things that would improve the privacy of their users because they would make their search engines overlords unhappy. It is very important to me that Privacy Browser never has a financial relationship with any search engine. That way, I can change the default search engine at any time based on the best interests of my users.

Privacy Browser 2.12

Privacy Browser 2.12 has been released. The default homepage and search engine has been changed from DuckDuckGo to Searx. This is a significant enough change that I have written a separate post explaining why the decision was made. This change only effects new installs. Existing installs will maintain their current settings unless updated by the user.

EasyPrivacy has a policy of not including entries that they consider consistently problematic. This results in some requests being allowed that should actually be blocked.  After considering the situation I decided to create a supplement to EasyPrivacy called UltraPrivacy, which is enabled by default.

There is also an option to block all third-party requests. This is good for user privacy, but it breaks about half the websites out there, so it is disabled by default.

Blocklist controls have been added to the Options menu.

There is a new Guide tab explaining how the Requests activity works.

The “Waiting for Orbot” message wasn’t displaying under certain circumstances, which has now been fixed.

Two bugs were fixed in the blocklist processing which were incorrectly blocking some resource requests. Two problems were fixed with the layout of full screen videos.

An updated Italian translation was provided by Francesco Buratti. An updated Spanish translation was provided by Jose A. León. The Russian translation was also updated.

The next version of Privacy Browser will have the ability to import and export settings.

Problems with Orbot

There is a bug with the most current version of Orbot (16.0.2-RC-1) that causes HTTP proxying to fail (HTTPS proxying works just fine). That means that webpages that begin with https:// will load just fine but webpages that begin with http:// will not.

I have filed a bug report with the developers of Orbot.

There are two workaround you can use until they release an update that fixes the problem.

  1. Downgrade to version 16.0.0-RC-2 of Orbot, which works fine.
  2. Enable Orbot’s VPN mode and disable Privacy Browser’s Orbot proxy setting.  Privacy Browser’s URL bar background will not be blue, but all traffic will be routed through Tor because of the OS level VPN. The potential downside to this workaround is that all the device’s traffic is being forced through Tor, which may not be desired.

Note that there was a separate bug that was fixed in Privacy Browser 2.11 that related to using Orbot.

Privacy Browser 2.11

Privacy Browser 2.11 has been released. The major new feature is a Requests activity that shows how many requests were made and how many were blocked.

Tapping on an individual request displays further details.

The Requests entry in the navigation menu displays the number of blocked requests.

I have written some information about how the blocklists work. The next release will include a Guide tab that explains each of the items in the request details. Note that in the future it will be possible to create custom user blocklists and load any blocklist that use the AdBlock syntax.

A bug, introduced by a change in a recent update of WebView that prevented proxying through Orbot, was fixed. This bug caused proxying to silently fail. The URL bar background would turn blue, Orbot would launch, but unless Orbot was functioning in VPN mode, WebView would send all requests directly to the internet.

Screenshots, video recording, and viewing on non-secure displays are now disabled by default. For those who need it, this functionality can be enabled in settings. Note that because of limitations in Android, some information, including menus and the keyboard, can be captured by screen recordings even when this setting is disabled.

Swipe to refresh is now available in domain and on-the-fly settings.  Additionally, if “display additional app bar icons” is enabled in settings, the refresh button is now displayed in the app bar.

Beginning in Android Oreo (API 26), form data support has been removed from WebView. It has been replaced by the Android OS autofill functionality. As such, the form data controls no longer appear in Privacy Browser when running on Android Oreo or newer. They will continue to function on older versions of Android.

A crash was fixed that was caused by viewing or loading domain settings for an empty URL.

The major feature of the next release will be the ability to block all third-party requests.

Privacy Browser 2.10

Privacy Browser 2.10 has been released. Uploading of files is now enabled for Lollipop and newer (API >= 21). Initially I thought this would require the Read Storage permission, but it turns out that beginning in Android Lollipop (API 21) Google added a system file chooser API to WebKit. This allows the browser to request the OS to display a file chooser, which has the Read Storage permission. The file chooser hands the file back to WebKit for upload. This is different than granting Read Storage permission to Privacy Browser because the user must explicitly select a file from the list; it does not allow Privacy Browser to access files in the background without user interaction.

Note that there are other planned features in the 2.x series that will probably require Read Storage permissions, like the import and export of settings and the import of bookmarks from other browsers. But I am not going to add it until it is needed. Also, note that many interfaces report the Privacy Browser has the Read Storage permission when it doesn’t. I am not certain why that happens, but it may be because the Read Storage and Write Storage permissions are linked under the Storage dangerous permission. Such that if a user grants the storage permission for Write Storage and later the app adds the Read Storage permission to the manifest it will be granted without further user interaction or notification.

As a personal milestone, the feature request to add file uploads was the first issue entered into Redmine. When I setup Redmine on 2 March 2016 Privacy Browser 1.0 had just been released. I created 15 feature requests that day to track items I knew I wanted to add. They weren’t entered in any particular order, but it turns out that the uploading of files was number 1. The oldest issue still open is number 5, fine-grained cookie controls, which, because of limitations in WebView, will have to wait until the 4.x series to be implemented.

This update changes the way user agents are stored and updated. In the past, when a user agent was set to mimic a different browser, it was for a specific version, like Firefox 56 on Windows 10. When the sample user agents were updated in a later release of Privacy Browser, the selection would remain Firefox 56 on Windows 10. This sort of defeated the purpose of making Privacy Browser mimic another browser because repeated user intervention was required to keep it updated.

With the new design, the user will select a generic setting, like Firefox on Windows. A separate list is maintained with the current user agent that matches this selection. When the list is updated with a new release, it will automatically be applied. For users who have already selected an older style user agent, it will stay that way until they select one from the new list.

There is now a Download URL entry in the context menu. This makes it possible to download files that Privacy Browser would otherwise display, like HTML or text files.

For those in Europe using Privacy Browser Free, a new ad consent dialog will display on first launch to comply with the GDPR. The dialog is also accessible from the options menu. There is an accompanying update to the privacy policy.

The GDPR has forced Google to create several privacy controls for their ad network that didn’t previously exist. I have used these controls to disable personalized ads and to disable tracking and remarketing for all users (by specifying that the user is under the age of consent, because maybe you are and it is better safe than sorry ).

A bug introduced in version 2.9 that prevented bookmarks from being loaded from the Bookmarks activity (but not from the bookmarks drawer) has been fixed. And a bug was fixed that caused some changes in domain settings to not be applied until after a reload. The workflow was also improved when adding or editing domain settings from the options menu.

Google’s Firebase library, used to display ads in Privacy Browser Free, keeps adding extra permissions at build time. The latest addition, READ_PHONE_STATE, is particularly annoying because it grants access to the phone number of the device, the current cellular network information, the status of any ongoing calls, and a list of any PhoneAccounts registered on the device. This is a dangerous permission, which requires explicit user interaction on Android Marshmallow and newer (API >= 23). In my testing I have not seen advertisements attempt to request or use this permission (I have seen them attempt to use the GPS permission, which is one of the reasons I am inclined to never add that one to Privacy Browser). I have considered getting rid of the Free version entirely, but I feel that it is a good way for many users to try out Privacy Browser before deciding if they want to commit to the paid version or learn how to use F-Droid.

Firebase has also added the com.google.android.finsky.permission.BIND_GET_INSTALL_REFERRER_SERVICE. There isn’t much information online about what this permission does, but it appears to be used to tell from which source an app is installed. So, an advertisement would be able to tell if Privacy Browser Free was installed from Google Play, XDA Labs, Amazon, or directly from Stoutner.com. There is some indication that this might be removed in the next version of Firebase (it has been removed in Google Play Services 15.0.2), but we will have to wait and see.

As usual, Francesco Buratti has provided an updated Italian translation and Jose A. León has provided an updated Spanish translation. There is also an anonymously updated Russian translation. This translation work takes a significant amount of effort and those who speak these languages should be grateful for their work.

The Block List activity was planned for the 2.10 release, but it was pushed off due to the need to release changes in time for the GDPR deadline. It will be the major feature in the next release.

Privacy Browser 2.9

Privacy Browser 2.9 has been released. The major change is that the write storage permission has been added as was previously announced in the roadmap. This allows downloads to be stored in the public download directory, and will also allow for a number of other planned features, like the import and export of settings.

It is now possible to control the block lists in domain settings. This allows a block list to be disabled if it there is a false positive on a particular domain or if the user wants to financially support the domain by viewing ads.

Custom URLs are now referred to a chooser to open in other apps. This allows, for example, market:// URLs to open an app store or oauth2redirect:// URLs to complete the Mastodon signup process.

A bookmarks tab has been added to the Guide. Some users, understandably, have difficulty finding the bookmarks. Hopefully, this will point them in the right direction.

Privacy Browser now has an adaptive icon. This is something I initially resisted doing, but it is the way everything is going on newer devices. It also allows me to replace the bitmap launcher icons with vector ones, which are smaller and allow for perfect layout on all devices.

There is now an explicit warning for users of Incognito Mode that forward and back do not work when it is enabled. Previously it wasn’t clear to many users that if the history was deleted forward and back would not work.

The favorite icon is now preserved when returning from the settings or domains activities. Cookies are now no longer erroneously deleted in Incognito Mode. And the webpage is no longer reloaded when restarting Privacy Browser from the launcher.

Privacy Browser 2.9 contains the first full Russian translation. Francesco Buratti provided an updated Italian translation and Jose A. León provided an updated Spanish translation. Stefan Erhardt provided a partially updated German translation. I am grateful for all their time and effort.

The next release of Privacy Browser will add the read storage permission which will allow for the uploading of files to websites. It will also have a block list activity that shows details about every request that is blocked. This will be useful for determining if a resource is incorrectly blocked, as well as for ascertaining what websites are doing to track users.

WebRTC

I receive questions about WebRTC (Web Real-Time Communication) frequently enough that I thought it would be worthwhile to write a post about it.

First, a little bit of background about WebRTC for those who might not be familiar with it. It is a web standard for enabling video and audio chat in the browser. WebRTC connections are typically brokered by a server, but to enable efficient communication of audio and video, the clients exchange their IP information so they can communicate directly. Depending on the configuration of the software involved, if a user is trying to mask their IP address, WebRTC can be used by a server to discover their true IP addresses (both private and public IPv4 addresses as well as the IPv6 address).

WebRTC requires JavaScript to function. By default, JavaScript is disabled in Privacy Browser. So, those who use Privacy Browser with the default settings do not need to be worried about WebRTC leaking their IP addresses.

Some users have Tor Orbot proxy enabled or are otherwise using a VPN to mask their IP address but also need to have JavaScript enabled for some websites. By default, Orbot runs in proxy mode. The proxy controls in Android’s WebView allow proxying of general HTTP and HTTPS data but not WebRTC. In this configuration Privacy Browser will leak IP addresses through WebRTC when connected to Orbot. However, Orbot can also run in VPN mode, which will force all traffic from the device over Orbot.

When Orbot is in VPN mode, Privacy Browser does not leak any IP address information via WebRTC as demonstrated by this screenshot from Browser Leaks. The key in the status bar at the top of the screen indicates that VPN mode is enabled. The local IP address listed is not the local IP address of the device, but rather the local IP address assigned by Tor, which in this case is 10.10.10.1.

Other VPN services will also mask the WebRTC IP address if they are configured correctly. Of course, the downside to using a VPN service is that then they can spy on everything you do and sell the information to the highest bidders.

There are some browsers that have support for disabling WebRTC completely, so that it can’t leak IP address information under any circumstances. Orfox and Fire.onion are two examples. Both of these browsers are modified versions of Firefox, and, as such, are able to disable WebRTC in the Gecko rendering engine. Privacy Browser currently uses Android’s built-in WebView as the rendering engine. Google does not provide a mechanism to disable WebRTC in WebView. However, in the 4.x series, I intend to create a rolling fork of WebView called Privacy WebView that will allow WebRTC to be completely disabled even when JavaScript is enabled.

To round out this conversation, I think it is important to point out that masking one’s IP address does not provide as much privacy as many people assume. The large technology companies have spent a lot of money building massive profiles of users, even those who are not their customers. These profiles contain much more detailed location information than an IP address discloses. And these systems are specifically designed to track users across IP addresses, because they want to track people from their homes, to their work, and across cell phone networks. Many of the technologies that companies use to track users are dependent on client devices running JavaScript. Blocklists like EasyPrivacy, which are included in Privacy Browser, are able to block some of this tracking, but they are not perfect. For users that need real privacy, the best defense is to browse the internet with JavaScript disabled.