[tor-project] OONI Monthly Report: July 2022

Wed Oct 19 17:38:28 UTC 2022

Hello,

These days we're primarily supported by the DRL, which is why we do
quarterly reporting. That said, we'd like to share monthly updates from the
OONI team with the community, hence our July 2022 report shared below. I'll
also follow-up to share our August and September 2022 reports.

*# OONI Monthly Report: July 2022*

Throughout July 2022, the OONI team worked on the following sprints:

* Sprint 69 (1st-17th July 2022)
* Sprint 70 (18th-31st July 2022)

Our work can be tracked through the various OONI GitHub repositories:
https://github.com/ooni

Highlights are shared in this report below.

*## Launched new Test Lists Editor*

In July 2022, OONI launched a new Test Lists Editor:
https://test-lists.ooni.org/

The Test Lists Editor enables the public to review and contribute to the
Citizen Lab test lists (
https://github.com/citizenlab/test-lists/tree/master/lists), without a
GitHub account. Their contributions automatically end up as pull requests
on the Citizen Lab test list repository, which means they'd still be
peer-reviewed.

We also published a blog post which provides a brief user guide and shares
information about the Test Lists Editor:
https://ooni.org/post/2022-test-lists-editor/

*## Updated the OONI Data Policy*

In preparation for the launch of the new Test Lists Editor (
https://test-lists.ooni.org/), we updated the OONI Data Policy to include a
new section ("OONI web services") that mentions that we do not store email
addresses (used to send login links for OONI accounts), and which shares
the providers (and their data policies) that we use for deploying web
services (https://github.com/ooni/ooni.org/pull/1198).

The updated OONI Data Policy is available here:
https://ooni.org/about/data-policy

*## OONI Probe Mobile*

In July 2022, we released OONI Probe Mobile 3.7.0 for:
* Android: https://github.com/ooni/probe-android/releases/tag/v3.7.0
* iOS: https://github.com/ooni/probe-ios/releases/tag/v3.7.0

This release includes:
* New Vanila Tor experiment;
* Automated testing of experimental tests;
* In-app language selection;
* Support for sharing a link to the OONI Probe app;
* Measurement engine synced with OONI Probe CLI 3.15.2;
* Bug fixes and improvements.

While working towards the OONI Probe Mobile 3.7.0 release, we also:
* Fixed a bug that prevented the VPN warning from appearing when running an
OONI Run link (https://github.com/ooni/probe-ios/pull/496);
* Worked towards improving the app performance when loading and showing
measurements (https://github.com/ooni/probe/issues/2150);
* Fixed a bug that affected storage usage (
https://github.com/ooni/probe/issues/2149);
* Worked towards improving the pagination of test results (
https://github.com/ooni/probe/issues/1430);
* Updated the measurement-log retention policy (
https://github.com/ooni/probe/issues/2162);
* Worked on a prototype of the OONI Probe app using flutter, which will
allow us to have a shared codebase between the OONI Probe mobile and
desktop apps.

*## OONI Run*

In an attempt to meet community needs (particularly those of researchers
performing custom testing), we designed and implemented a minimal version
of the next generation version of OONI Run (“OONI Run v2”) that enables
community members to run OONI Run links without backend support (
https://github.com/ooni/probe-cli/pull/844). We made this functionality
available to miniooni users with the goal of enabling community members to
start experimenting with this new OONI Run version (
https://github.com/ooni/probe/issues/2184) and share feedback which can
support further development and design decisions.

Based on community requests, we also introduced a command line flag which
enables users to repeat a measurement every given number of seconds (until
OONI Run v2 has support for repeating the measurement with a schedule):
https://github.com/ooni/probe-cli/pull/819

As part of our plan for incrementally releasing a new version of OONI Run (
https://run.ooni.io/), we started updating the specification for the new
version of OONI Run based on our experimental miniooni-based deployment (
https://github.com/ooni/spec/pull/249).

*## Expanding censorship measurement methodologies*

We worked on data quality improvements to OONI measurements. Specifically,
we improved the way in which the Web Connectivity test helper works, by
using a single, independent  HTTP Client per test helper request, to avoid
hitting the limit of maximum connections per HTTP Client (
https://github.com/ooni/probe/issues/2182).

We supported our Google Summer of Code (GSoC) student in developing a TLS
middlebox experiment (https://github.com/ooni/probe-cli/pull/817) and
experimented around improving how we perform DNS resolution to enable the
opportunistic measurement of DoH endpoints (
https://github.com/ooni/probe-cli/pull/846). We also reviewed the work of
an OTF Information Controls Research Fellow that we’re currently hosting (
https://github.com/ooni/minivpn/pull/10,
https://github.com/ooni/minivpn/pull/12).

*## OONI Explorer*

As part of our work on improving OONI Explorer, we worked on:
* Adding support for localization (https://github.com/ooni/explorer/pull/705
);
* Implementing domain-centric pages (
https://github.com/ooni/explorer/pull/762);
* Maintenance, package upgrades, and fixing security vulnerabilities;
* Improving the SEO of the Measurement Aggregation Toolkit (MAT).

*## OONI backend*

In July 2022, we continued to work on the migration to the Clickhouse
database. This involved a series of backend tasks including the testing and
switching to more API entry points, numerous configuration changes, and
end-to-end testing of the processing of measurements. We fixed various bugs
in the fastpath, including the handling of the measurement start time.

We worked on the reprocessing of legacy measurements, which involved the
deletion of old legacy measurements from the fastpath pipeline table,
fixing a bug and adding metrics for the fetching of measurements across API
hosts, and improving upon JSONL tables, among many other backend tasks (
https://github.com/ooni/api/pull/297). We improved the API performance by
enabling gzip compression of measurements.

We created internal dashboards to monitor and test the new ASN-based URL
prioritization system (https://github.com/ooni/api/pull/293).  We also
worked on improvements to the new test helper (
https://github.com/ooni/pipeline/pull/392) and performed security updates
to the infrastructure.

*## Interviewing Community Coordinator candidates*

In July 2022, we carried out and completed the second (and final) round of
interviews for the OONI Community Coordinator position (
https://ooni.org/post/2022-job-opening-ooni-community-coordinator/).

We were thrilled to have received applications from (and to have the
opportunity to meet) many strong candidates. Based on both rounds of
interviews, we decided to hire Elizaveta Yachmeneva for the OONI Community
Coordinator position (who joined the OONI team in August 2022).

*## Test list updates*

In July 2022, we updated the test list for the Philippines to include
websites that were reportedly blocked (
https://github.com/citizenlab/test-lists/pull/1022).

*## New blocks emerged in Indonesia*

On 29th July 2022, new blocks emerged in Indonesia. We shared relevant OONI
data (and findings) with the #KeepItOn advocacy community.

Specifically, the blocked services included:

* PayPal:
https://explorer.ooni.org/chart/mat?probe_cc=ID&test_name=web_connectivity&domain=www.paypal.com&since=2022-04-02&until=2022-08-02&axis_x=measurement_start_day
* Yahoo:
https://explorer.ooni.org/chart/mat?probe_cc=ID&test_name=web_connectivity&domain=yahoo.com&since=2022-04-02&until=2022-08-02&axis_x=measurement_start_day
* Steam:
https://explorer.ooni.org/chart/mat?probe_cc=ID&test_name=web_connectivity&domain=store.steampowered.com&since=2022-04-02&until=2022-08-02&axis_x=measurement_start_day
* Origin:
https://explorer.ooni.org/chart/mat?probe_cc=ID&test_name=web_connectivity&domain=origin.com&since=2022-04-02&until=2022-08-02&axis_x=measurement_start_day
* Epic Games:
https://explorer.ooni.org/chart/mat?probe_cc=ID&test_name=web_connectivity&domain=epicgames.com&since=2022-04-02&until=2022-08-02&axis_x=measurement_start_day
* Dota2:
https://explorer.ooni.org/chart/mat?probe_cc=ID&test_name=web_connectivity&domain=www.dota2.com&since=2022-04-02&until=2022-08-02&axis_x=measurement_start_day

In many of the above cases, we were able to automatically confirm the
blocks based on the IP returned from DNS resolution. In other cases, we
were able to confirm the blocks because they were implemented through an
HTTP middlebox serving a blockpage (
https://explorer.ooni.org/measurement/20220731T055038Z_webconnectivity_ID_45727_n1_37dd9ijGb4P2ILK6?input=http%3A%2F%2Fwww.yahoo.com%2F
).

To encourage further testing of the blocked services, we shared an OONI Run
link with the #KeepItOn community.

*## OONI citations### Digital Defenders Partnership publication*

Digital Defenders Partnership published a guide (in Spanish) describing how
to document internet blockages and circumvent internet censorship. This
publication, which cites OONI and describes OONI tools, is available here:
https://www.digitaldefenders.org/wp-content/uploads/2022/09/2207-ISG_final-es.pdf

*### OPTIMA report on the impact of internet shutdowns on women in Uganda*

Sandra Aceng published a research report (as part of Internews’ OPTIMA
project) which documents the impact of internet shutdowns on women in
Uganda. This study makes use of OONI data collected from Uganda around the
country’s 2021 elections.

The report is available here:
https://preparepreventresist.org/wp-content/uploads/2022/07/Impact-of-Shutdowns-on-Women-in-Uganda.pdf

*## Community activities### OONI training session for journalists in Kenya*

On 28th July 2022, OONI’s Maria facilitated an OONI training session for
journalists in Kenya. The workshop involved introducing participants to
OONI tools and data for investigating internet censorship in Kenya and
around the world.

*### OONI Community Meeting*

On 26th July 2022, we hosted the monthly OONI Community Meeting on our
Slack channel (https://slack.ooni.org/), during which we discussed the
following topics:

1) Using OONI Explorer to group measurements by region

2) Request for community feedback: Do users want to have the option to
delete specific measurements manually or automatically?

3) Request for community feedback on write/erase cycle limits

*## Userbase*

In July 2022, 47,752,189 OONI Probe measurements were collected from 2,892
ASNs in 167 countries around the world.

This information can also be found through our measurement stats on OONI
Explorer (see chart on “monthly coverage worldwide”):
https://explorer.ooni.org/

~ OONI team.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.torproject.org/pipermail/tor-project/attachments/20221019/56158776/attachment-0001.htm>