[ooni-talk] OONI Monthly Report: March 2022

[Maria Xynou]

Hello,

OONI's March 2022 status report is shared below.

*# OONI Monthly Report: March 2022*

Throughout March 2022, the OONI team worked on the following sprints:

* Sprint 60 (1st - 13th March 2022)
* Sprint 61 (14th - 31st March 2022)

Our work can be tracked through the various OONI GitHub repositories:
https://github.com/ooni

Highlights are shared in this report below.

*## Published report on new blocks in Russia amid war in Ukraine*

We published a research report on new censorship events in Russia amid the
war in Ukraine, based on OONI data analysis.

You can read the report here:
https://ooni.org/post/2022-russia-blocks-amid-ru-ua-conflict/

Below we share some key findings:

* Media censorship. Russia recently started blocking access to independent
Russian news media websites (such as Dozhd and New Times) and foreign news
media websites (such as BBC, Deutsche Welle, Russian version of Voice of
America, and RFE/RL service websites).

* Blocking of a website about captured and killed Russian soldiers. Russia
blocked access to 200rf.com, which was created by representatives of the
Ministry of Internal Affairs of Ukraine to share information about captured
and killed Russian soldiers in Ukraine.

* Centralized throttling of Twitter. OONI data suggests that access to
twitter.com was throttled in Russia between 26th February 2022 to 4th March
2022. As the throttling of twitter.com seems to have stopped across all
ISPs in Russia at the same time (~08:00 on 4th March 2022 UTC), it appears
to have been centralized. Moreover, interference to twitter.com appears to
have changed from throttling to blocking (through the injection of a RST
packet) on 4th March 2022.

* Blocking of Facebook and Twitter. OONI data shows that access to
facebook.com and twitter.com was blocked in Russia by 4th March 2022
(primarily through the injection of RST packets).

* Decentralized censorship. Every Internet Service Provider (ISP) in Russia
is responsible for implementing government-mandated blocks independently.
As a result, we observe variance in how internet censorship is implemented
across Russia, as blocks are not implemented on all networks in the
country, and different ISPs adopt different censorship techniques. Some
ISPs implement blocks through the use of multiple techniques at the same
time, making circumvention harder.

* Different censorship techniques. To block websites, OONI data shows that
Russian ISPs adopt the following censorship techniques (beyond throttling):

* DNS manipulation, redirecting in some cases to blockpages
* HTTP man-in-the-middle, serving blockpages
* TLS man-in-the-middle
* Injection of a RST packet after the ClientHello during the TLS handshake
(most prevalent)
* Timing out the session after the ClientHello during the TLS handshake
* Closing the connection after the ClientHello during the TLS handshake

These findings received press coverage from Wired (
https://www.wired.com/story/russia-splinternet-censorship/), Business
Insider (
https://www.businessinsider.com/splinternet-in-pictures-what-russian-internet-looks-like-right-now-2022-3),
VPNCompare (https://www.vpncompare.co.uk/ooni-russia-censorship-propaganda/),
and Protocol (https://www.protocol.com/russian-internet-crackdown).

This research report also resulted in a donation to OONI from Pantheon, who
(along with several other open web leaders) published a statement in
support of OONI’s mission:
https://pantheon.io/blog/pantheon-rises-support-open-web

*## OONI Probe Mobile*

We released OONI Probe Android 3.6.0:
https://github.com/ooni/probe-android/releases/tag/v3.6.0

This release includes the following improvements:
* New Tor Snowflake experiment
* New DNS Check experiment
* RiseupVPN test temporarily disabled
* Links to OONI blog posts and research reports
* User-initiated tests start off minimized
* Improved progress bar and test runtime counter
* Measurement engine synced with OONI Probe CLI 3.14
* Bug fixes and improvements

We followed-up with an OONI Probe Android 3.6.1 release (
https://github.com/ooni/probe-android/releases/tag/v3.6.1) with a bug fix.

On OONI Probe Android, we also worked on:
* Enabling users to disable the VPN warning (
https://github.com/ooni/probe/issues/2015);
* Exploring how to make the VPN check compatible with Netguard firewall (
https://github.com/ooni/probe/issues/1697);
* Fixing a bug affecting the re-run button (
https://github.com/ooni/probe/issues/2043);
* Adding support for in-app language selection (
https://github.com/ooni/probe/issues/1933).

On OONI Probe iOS, we worked on:
* Marking measurements coming from a VPN with a special annotation (
https://github.com/ooni/probe/issues/1901);
* Ensuring that tests start minimized (
https://github.com/ooni/probe/issues/1976);
* Changing the styling of the proxy indicator (
https://github.com/ooni/probe/issues/1919);
* Ensuring that the VPN modal is only displayed before tests start running (
https://github.com/ooni/probe/issues/1982);
* Addressing a bug that resulted in the minimized running test indicator
from missing in the measurement list view (
https://github.com/ooni/probe/issues/1981);
* Ensuring that VPN warnings are not displayed when the VPN warning option
is disabled (https://github.com/ooni/probe/issues/2015,
https://github.com/ooni/probe-ios/pull/478);
* Fixing the background color in the experimental test group animation (
https://github.com/ooni/probe/issues/2038);
* Improving the progress bar of tests (
https://github.com/ooni/probe/issues/2021,
https://github.com/ooni/probe/issues/1936).

*## OONI Probe Desktop*

We released OONI Probe Desktop 3.7.0:

Leading up to this release, we made several release candidates for
extensive testing and quality assurance (
https://github.com/ooni/probe-desktop/releases).

Notably, OONI Probe Desktop 3.7.0 includes the following improvements:
* New Tor integration (https://github.com/ooni/ooni.org/issues/761);
* New Tor Snowflake experiment (
https://github.com/ooni/probe-desktop/pull/271);
* New auto-scrolling container for logs when running tests (
https://github.com/ooni/probe-desktop/pull/288);
* RiseupVPN test temporarily disabled (
https://github.com/ooni/probe/issues/2002);
* Bug fix affecting the custom website editor (
https://github.com/ooni/probe-desktop/pull/273);
* Bug fix ensuring fallback to English for unsupported languages (
https://github.com/ooni/probe-desktop/pull/274);
* Electron upgraded to v12 (https://github.com/ooni/probe-desktop/pull/278).


*## Expanding censorship measurement methodologies*

We wrote the third prototype of Websteps (codenamed “winter 2022”), our new
experiment for measuring the blocking of websites. Rather than modifying
the existing implementation, we chose to fork relevant packages of the
probe-cli repository into a separate repository (
https://github.com/bassosimone/websteps-illustrated/), which allowed us to
iterate faster and perform more changes without the risk of breaking the
existing codebase or splitting each change into easy-to-review, small
patches.

We achieved quite a lot of progress in this fork, ranging from improvements
to the underlying libraries to significant progress in terms of the final
version of Websteps. The repository itself contains extensive documentation
about what changed. Here we only provide some highlights:

* We wrote a complete specification of Websteps that not only describes the
measurement methodology, but also the analysis algorithms;

* We tested this Webstep implementation on a few networks in China, Iran,
and Italy;

* We improved the conceptual model of Websteps, which now allows us to
easily express what we are measuring, with more clarify with regards to the
measurement algorithm;

* We developed a new integration testing framework for Websteps that
(rather than relying on simulating censorship using Linux networking
capabilities as we currently do) is based on re-running actual measurements
collected on censored networks;

* We significantly improved the DNS implementation by adding support for
PTR, CNAME, and NS records, fixing bugs and adding performance improvements;

* We implemented a DNS Ping extension for Websteps that only runs after
specific DNS measurements to collect extra information (e.g. multiple DNS
replies) that is rather useful to collect evidence of DNS-based censorship;

* We wrote a Websteps measurement library in Python that allows us to
process and visualize measurements and manage integration test cases.

As a next step, we are engaging external researchers for an additional
round of review of the experiment. Following this review (and any further
changes based on feedback), we will start the process of integrating the
changes into the OONI Probe CLI repository.

*## Launch of Circumvention Tool Reachability Dashboard*

We launched a new dashboard (https://explorer.ooni.org/chart/circumvention)
that presents findings from all censorship circumvention tool test results
(including the new Snowflake test) on OONI Explorer.

This new Circumvention Tool Reachability Dashboard presents aggregate views
of real-time OONI data collected from the reachability testing of
circumvention tools around the world. Through this interactive dashboard,
users can check whether circumvention tools work and monitor their
potential blocking around the world based on real-time OONI data. As the
dashboard presents charts based on aggregate views of OONI data, these
charts can potentially support relevant outreach and advocacy efforts.

*## OONI Measurement Aggregation Toolkit (MAT)*

Throughout March 2022, we made significant progress on the Measurement
Aggregation Toolkit (MAT) in preparation for the public launch in April
2022. Based on community feedback and extensive testing, we worked on
making several improvements (
https://github.com/ooni/explorer/issues?q=is%3Aissue+label%3AMAT+is%3Aopen).


Notably, we added support (to both OONI Explorer Search and the MAT) for
filtering Web Connectivity measurements based on website categories (
https://github.com/ooni/explorer/issues/288). This enables OONI Explorer
and MAT users to filter measurements based on website categories (such as
news media) and to easily discover thematic website censorship, such as the
blocking of news media websites, human rights websites, and many other
categories of websites (we use the Citizen Lab test list category codes).

To ensure compatibility between the MAT and the OONI Explorer Search Tool,
we added support for domain-based filtering (
https://github.com/ooni/explorer/issues/664) in the MAT (as opposed to
filtering by URL). This enables users to get aggregate views of all
measurements pertaining to a domain, rather than having to explore
measurements separately for each URL corresponding to a domain.

We made a series of improvements to the tables and charts generated through
the MAT, and we improved the labels and copy used in various features (such
as buttons and drop-down menus) of the platform. We also worked on adding
code-level documentation to highlight critical parts of the codebase.
Following all the improvements, we created a preview link for further
testing leading up to the launch.

*## OONI backend*

During March 2022, OONI backend activities mainly involved the reprocessing
of legacy OONI measurements to improve data quality. As part of this, we
are working on improving the measurement re-processor (
https://github.com/ooni/backend/issues/569). To monitor the progress, we
developed new dashboards on Jupyter.

As part of the migration to Clickhouse, we implemented fetching
measurements from the spool directory served by Nginx and developed speed
improvements for the API's get_measurement. We benchmarked ways to upload
database tables from Clickhouse to S3. This could be used in the future to
automatically publish database tables to external researchers.

We made improvements to our URL prioritization algorithm (
https://github.com/ooni/backend/issues/562) to ensure that popular social
media URLs (which are frequently blocked around the world) are always
prioritized for testing globally. As part of our work to move probes to the
new test helper, we created a monitoring dashboard, we added performance
metrics to the API, and we forwarded some of the production traffic to the
new hosts.

*## Improving OONI data analysis capabilities*

As part of our efforts to consolidate block page fingerprints that we find
with those collected by the Citizen Lab (
https://github.com/ooni/backend/issues/516), we opened a pull request on
the Citizen Lab repository for filtering annotations with DNS signatures
from Russia (https://github.com/citizenlab/filtering-annotations/pull/1).
These were extracted as part of our OONI data analysis efforts when
investigating the emergence of new blocks in Russia following the invasion
of Ukraine.

*## Creating an online OONI Training Course for Advocacy Assembly*

During March 2022, we worked on completing all the materials for the online
OONI training course that we are creating for Small Media’s Advocacy
Assembly project (https://www.advocacyassembly.org/).

In particular, we:
* Created 11 screencasts for various OONI tools;
* Wrote copy for 10 training slides (which are included in several chapters
of the course);
* Created quizzes for 3 training chapters;
* Coordinated with partners on creating case study videos.

*## Test list updates*

In response to censorship events, we contributed updates to the following
Citizen Lab test lists:

* Russia: https://github.com/citizenlab/test-lists/pull/929,
https://github.com/citizenlab/test-lists/pull/930,
https://github.com/citizenlab/test-lists/pull/934
* Ukraine: https://github.com/citizenlab/test-lists/pull/935
* Egypt: https://github.com/citizenlab/test-lists/pull/939
* Global: https://github.com/citizenlab/test-lists/pull/938,
https://github.com/citizenlab/test-lists/pull/947,
https://github.com/citizenlab/test-lists/pull/948

We also reviewed and merged several test list pull requests contributed by
community members.

*## Notable community use of OONI Probe and OONI data*

*### ISOC blog posts on blocks in Russia*

As part of their “Internet Perspectives: Ukraine and Russia” blog series (
https://pulse.internetsociety.org/blog/internet-perspectives-ukraine-and-russia),
Internet Society (ISOC) published the following 2 blog posts based on OONI
data:

* OONI Data: Looking For Anomalies and Blocks:
https://pulse.internetsociety.org/blog/internet-perspectives-ukraine-and-russia#OONI
* Top Categories for New Russian Internet blocks (OONI Anomalies):
https://pulse.internetsociety.org/blog/internet-perspectives-ukraine-and-russia#OoniAnomalies

*### Article on network congestion and blocks in Iran*

Digiato (an Iranian technology news portal) published an article discussing
network congestion in Iran, where Iranian experts discuss intermittent
blocks in the country, citing OONI data (
https://digiato.com/article/2022/02/23/why-iran-internet-is-congested). The
article also features a screenshot of a chart generated from OONI’s
Measurement Aggregation Toolkit (to demonstrate the intermittent blocks).



*## Community activities### OONI presentation at Spacemesh Meetup*

Spacemesh is a cryptocurrency organization that also has a social causes
program (https://spacemesh.io/blog/spacemesh-social-causes-program/). As
part of this program, they focused on “Fighting Censorship” and engaged
their global community with using OONI Probe (
https://spacemesh.io/blog/winning_together-raise-the-flag/#challenge-overview).
In particular, they encouraged their community to install and run OONI
Probe, and to practice several OONI Probe hands-on exercises (“challenges”)
that we shared with them. We were also invited to give an OONI presentation
for Spacemesh.

On 8th March 2022, OONI’s Maria presented OONI at a Spacemesh meetup. The
presentation can be viewed here: https://vimeo.com/688867593

Spacemesh subsequently wrote about OONI in their weekly newsletter:
https://smesher.substack.com/p/weekly-inspiration-for-humans-in

*### OONI Community Meeting*

On 29th March 2022, we hosted the monthly OONI Community Meeting on our
Slack channel (https://slack.ooni.org/), during which we discussed the
following topics:

1) Updates from the OONI team (new research and OONI Probe releases)

2) Cambodia’s National Internet Gateway

3) Addressing false positives in OONI measurements

4) Simplifying the OONI FAQ

*## Userbase*

In March 2022, 40,833,632 OONI Probe measurements were collected from 3,155
AS networks in 167 countries around the world.

This information can also be found through our measurement stats on OONI
Explorer (see chart on “monthly coverage worldwide”):
https://explorer.ooni.org/

~ OONI team.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.torproject.org/pipermail/ooni-talk/attachments/20220418/677a00e5/attachment-0001.htm>