[ooni-talk] OONI Monthly Report: December 2021

[Maria Xynou]

Hello,

OONI's December 2021 status report is shared below.

*# OONI Monthly Report: December 2021*

Throughout December 2021, the OONI team worked on the following sprints:

* Sprint 53 (1st-5th December 2021)
* Sprint 54 (6th-19th December 2021)

Our work can be tracked through the various OONI GitHub repositories:
https://github.com/ooni

Highlights are shared in this report below.

*## Published report on Tor blocking in Russia*

We published a report documenting the blocking of Tor in Russia based on
OONI data.

Read the report here: https://ooni.org/post/2021-russia-blocks-tor/

Our report shares OONI data and analysis on the blocking of the Tor network
and Tor Project website on some AS networks in Russia.

Wired also published a story on Tor blocking in Russia, sharing OONI data
and findings: https://www.wired.co.uk/article/russia-block-tor-censorship

*## Year in Review: OONI in 2021*

In December 2021, we published our annual end-of-year post, "Year in
Review: OONI in 2021", which shares:

* OONI highlights from 2021

* Exciting OONI activities by community members in 2021

* Upcoming OONI projects in 2022

You can read this post here: https://ooni.org/post/ooni-in-2021/

Warm thanks to our amazing community for making our work possible!

We thank every OONI Probe user out there, and we're excited for 2022!

*## Published blog post about OONI Probe integration into iThena*

 iThena (https://root.ithena.net/) is a distributed computation and
measurement project based on the Berkeley Open Infrastructure for Network
Computing (BOINC) platform.

In December 2021, we published a blog post where we discussed how iThena
helped boost OONI censorship measurement coverage worldwide by integrating
OONI Probe into their platform.

This blog post is available here:
https://ooni.org/post/2021-ithena-boosts-ooni-measurement-coverage/

*## Published guest post by The Bachchao Project*

In October 2021, we participated in the Bachchao Project’s 2-day event
(“India, Let’s Build the List”) where we helped facilitate sessions for
communities in India to update the Citizen Lab’s test list for India.

In December 2021, we published the Bachchao Project’s guest blog post about
the event on the OONI website:
https://ooni.org/post/2021-event-india-test-list/

*## OONI Probe Mobile*

We released OONI Probe Mobile 3.5.0 for Android (
https://github.com/ooni/probe-android/releases/tag/v3.5.0) and iOS (
https://github.com/ooni/probe-ios/releases/tag/v3.5.0).

This release includes the following improvements:
* Automated testing for instant messaging app tests (WhatsApp, Facebook
Messenger, Signal, Telegram)
* Automated testing for circumvention tool tests (Tor, Psiphon, RiseupVPN)
* Automated testing requires at least 20% of battery level
* OONI Probe CLI updated to 3.13.0
* Bug fixes

Following this release, OONI Probe mobile app users can run all OONI Probe
tests (excluding the bandwidth-intensive performance tests) automatically
on a regular basis. We expect that this will not only help boost OONI
measurement coverage further, but it will also be helpful in ensuring that
social media apps and circumvention tools (both of which are frequently
blocked around the world) are more regularly tested.

We started making improvements to how we do testing of the OONI Probe apps
by doing integration testing of all components inside of a mono-repo (
https://github.com/ooni/probe/issues/1926). We also worked towards
addressing OONI Probe technical debt (
https://github.com/ooni/probe/issues/1808).

*## OONI Probe Web*

We now have a working prototype of OONI Probe Web (
https://github.com/ooni/probe-web) that implements the full functionality.

Features of OONI Probe Web currently include:
* Informed consent onboarding process (
https://github.com/ooni/probe-web/pull/1);
* Communication with the check-in API for receiving test-list targets;
* Submission of measurements to the testing collector (
https://github.com/ooni/probe/issues/1494).

Unlike the OONI Probe apps, OONI Probe Web is limited to website testing.
The prototype includes a feature that enables users to determine if they
would like to limit their testing to a specific number of URLs.

*## Integrating tor into OONI Probe*

We completed the integration of tor into OONI Probe Mobile and OONI Probe
Desktop (https://github.com/ooni/probe/issues/1867). It is now possible to
use go-libtor when it's available. We switch between using go-libtor or
searching for the tor binary in the path by using build constraints. The
branch that integrates this work is the following:
https://github.com/ooni/probe-cli/pull/614

We also integrated tor into OONI Probe Desktop (
https://github.com/ooni/probe-desktop/pull/261) and into OONI Probe CLI (
https://github.com/ooni/probe/issues/1872,
https://github.com/ooni/probe/issues/1866).

*## Expanding censorship measurement methodologies*

We released OONI Probe CLI 3.13 (
https://github.com/ooni/probe-cli/releases/tag/v3.13.0) through which users
can now run our new DNScheck experiment (
https://github.com/ooni/spec/blob/master/nettests/ts-028-dnscheck.md). This
release (along with our 3.13-beta and 3.14-alpha releases made in December
2021) includes a series of other improvements to our experiments and
measurement methodologies (https://github.com/ooni/probe-cli/releases), as
well as updated dependencies.

*## OONI Measurement Aggregation Toolkit (MAT)*

We continued to make improvements to the OONI Measurement Aggregation
Toolkit (MAT).

In particular, we improved the usability of the one-dimensional charts, we
fixed a tooltip rendering issue, we resized the panels for the charts and
table, we fixed issues with non-measurement-start-day based charts, we
fixed the sorting order of the table, and we worked on other UI
improvements (https://github.com/ooni/explorer/issues/624). We also added
support for displaying the various blocking types (DNS, TCP/IP,
HTTP-failure, HTTP-diff) as an axis value in the MAT charts. Improvements
to the MAT can be viewed through commits made in the following pull
request: https://github.com/ooni/explorer/pull/622

*## OONI backend*

In December 2021, we continued to make progress on migrating our backend
infrastructure to Clickhouse (https://github.com/ooni/pipeline/pull/371).

To this end, we worked on:
* Switching more tables over to Clickhouse and improving test coverage;
* Implementing Clickhouse in our fastpath data processing pipeline;
* Implementing the Clickhouse feeder;
* Updating JSONL table in Clickhouse from feeder;
* Updating CI to test Clickhouse;
* Splitting the platform in tiers;
* Adding a deletion flag on the uploader;
* Re-importing the JSONL table and rebuilding the fastpath table with the
domain column.

We also worked on the following backend activities:
* Implemented an integrated API/fastpath multi-host dashboard;
* Added backend support for Tor Snowflake experiment feature extraction (
https://github.com/ooni/pipeline/pull/372);
* Added backend support for extracting Tor Snowflake runtime metrics and
exposing them via a new API endpoint (https://github.com/ooni/api/pull/274);

* Handled upcoming VM reboots;
* Continued to monitor OONI measurement coverage from unattended runs on
OONI Probe Mobile and Desktop.

*## Established new partnership with Zaina Foundation*

In December 2021, we established a new partnership with Tanzania's Zaina
Foundation (https://zainafoundationtz.org/).

Over the past year, we had already been collaborating with Zaina
Foundation, who facilitated OONI training for women human rights defenders
in Tanzania, conducted usability studies for OONI Probe and OONI Explorer,
and translated OONI Probe to Swahili. We are therefore thrilled to have now
formalized our partnership with Zaina Foundation, and we’re excited for our
ongoing collaboration over the next few years.

To highlight Zaina Foundation’s important work, we published a new page on
the OONI Partners page: https://ooni.org/partners/zaina-foundation/



*## OONI website### New Donate page*

In December 2021, we published a new Donate page on the OONI website:
https://ooni.org/donate/

Through this page, we aim to encourage donations in support of OONI’s work.

*### Edited various OONI webpages*

We edited and updated the copy of several web pages on the OONI site. In
particular, we edited the About page (
https://github.com/ooni/ooni.org/pull/1041), we updated the Frequently
Asked Questions (FAQ) section (https://github.com/ooni/ooni.org/pull/1042),
and we restructured and edited the Nettests section (
https://github.com/ooni/ooni.org/pull/1043). We also

*## Hosting an OTF Information Controls Fellow*

As of December 2021, we started to serve as the host organization for Open
Technology Fund (OTF) Information Controls research fellow, Gurshabad
Grover (https://twitter.com/OpenObservatory/status/1471454369866719237).

During his 1-year research fellowship with us, Gurshabad aims to uncover
how ISPs, through their technical decisions, can exacerbate or minimize the
effects of state-ordered censorship.



*## Community activities### PiProbe*

Our Venezuelan partners, Venezuela Inteligente, created a new OONI Probe
distribution for Raspberry Pis, called “PiProbe” (
https://github.com/VEinteligente/PiProbe). On 15th December 2021, they
presented PiProbe to collect feedback from the OONI team and broader
community.

*## Userbase*

In December 2021, 39,609,318 OONI Probe measurements were collected from
2,785 AS networks in 165 countries around the world.

This information can also be found through our measurement stats on OONI
Explorer (see chart on “monthly coverage worldwide”):
https://explorer.ooni.org/

~ OONI team.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.torproject.org/pipermail/ooni-talk/attachments/20220114/42d4e661/attachment.htm>