Is the Website Blocked? Verifying Internet Censorship with OONI Explorer

A brief article on verifying blocking of websites using OONI Explorer

Introduction

OONI Explorer is the world's largest open dataset on internet censorship, consisting of millions of measurements collected from more than 200 countries since 2012. The measurements are collected through the OONI Probe, available as mobile and desktop apps.

When a user tests for possible censorship with the OONI app, it would give one of the following possible results:

OK: meaning that the website works fine and there is no censorship;
Anomaly: meaning that there are some issues in accessing the website, although not necessarily censorship and could also indicate problems only on the user’s end;
Confirmed: meaning that there is a confirmed censorship on the website; or
Failure: meaning that the test had somehow failed and did not complete

The chart depicts measurements collected using OONI for the past month, showing the four types of results obtained.

Confirmed blockings

A likely blocked website would show up as a Confirmed Blocked or an Anomaly on the OONI Explorer.

Example of a Confirmed Blocked measurement on the OONI Explorer

Example of an Anomaly measurement on the OONI Explorer

A common inquiry is how does OONI classify a test result as being confirmed? Confirmed blockings are additional verification from results that are classified as anomalies (or sometimes even failures).

OONI confirms censorship in a measurement when there are block pages like these:

There are also instances where there is no block page shown, but the address the website redirects to a link that traces to a government authority. For example, censored websites in Malaysia are redirected to the IP address 175.139.142.25. This address is linked to MCMC, Malaysian Communications and Multimedia Commission, which was confirmed through a MCMC’s letter published by Malay Mail in 2018.

An example of a blocked website (www.bersih.org) redirecting to ip address 175.139.142.25, MCMC’s block page

OONI identifies blockings like these as blocking fingerprints through a GitHub repository. This database contains blockings at a few levels including:

national level blockpage
ISP level blockpage
text pattern related to a middlebox product
text pattern related to a voluntary institution blockpage (school, office)
vague blocking word

According to OONI, confirmed blockings on the Explorer are limited to only national level and ISP level.

False positives

Anomalies which are not confirmed are likely to be false positives, indicating a problem such as the user’s internet connection, the website itself is down, or a blocking or filter is implemented in the user’s office or school network.

Separating between false positives and confirmed blockings can be a manual and tedious task by looking at each OONI measurement. While some of these false positives can be obvious even for someone without any technical knowledge, some measurements would need to be checked further e.g. for signs of blockings in the TLS certificate.

An example of an ‘obvious’ false positive measurement in India (https://explorer.ooni.org/measurement/20220418T011856Z_webconnectivity_IN_55824_n1_TIx5TMqHoNP2jOXH?input=http%3A%2F%2F8thstreetlatinas.com%2F) where the network administrator has blocked the website.

There are also instances where it is vague, and censorship can only be determined when there is a sufficient number of anomalies recorded over a period of time.

A number of independent news websites in Cambodia were found blocked in February. While many of the measurements show redirects to the local host address 127.0.0.1, it cannot be confirmed as blocking until there are sufficient measurements as shown in the chart.

Contributing to OONI’s Blocking Fingerprints

Recording of these blocking fingerprints and false positives is important for research using OONI data, especially since it makes the data ‘cleaner’ and more accurate, thus making the case stronger when advocating for human rights related to internet censorship.

Based on Sinar Project’s research for the iMAP project, an internet censorship project covering 9 countries in the South and Southeast Asia region, 59 new fingerprints are being added to OONI’s database. This has identified new confirmed blockings, including countries such as the Philippines, Vietnam and Cambodia where there were no confirmed classifications before

Here is how you can contribute to further improve OONI’s measurements and results:

Contribute to the OONI blocking fingerprints: If there are measurements you found where this is a block page but identified as anomaly on OONI, you can contribute here. Recording of false positives may also be done here.
Testing: You may test on various platforms, both on Mobile (iOS and Android) and Desktop, including on the CLI on Linux platforms. The domains you test can be either randomly selected from the Citizenlab Test Lists or custom test lists specific to your needs.
Contribute to the test lists: You can contribute to the test lists on GitHub or on OONI.
Translate the OONI Probe to your local language here.
Participate in community discussions on the OONI’s Slack channel.

About iMAP

Internet Monitoring Action Project (iMAP) aims to establish regional and in-country networks that monitor network interference and restrictions to the freedom of expression online in 9 countries: Myanmar, Cambodia, Hong Kong, India, Indonesia, Malaysia, Philippines, Thailand, and Vietnam. Sinar Project is currently working with national digital rights partners in these 9 countries. The project is done via Open Observatory Network Interference (OONI) detection and reporting systems, involving the maintenance of test lists and measurements.

More information available at: imap.sinarproject.org. Any enquiries and suggestions about this report can be directed to [email protected].