[tor-project] OONI Monthly Report: April 2021

[Maria Xynou]

Hello,

Hope you're all doing well.

These days we're primarily supported by the DRL, which is why we do
quarterly reporting. That said, we'd like to share monthly updates from the
OONI team with the community, hence our April 2021 report shared below.
I'll also follow-up to share our May and June 2021 reports.

*# OONI Monthly Report: April 2021*

Throughout April 2021, the OONI team worked on the following sprints:

* Sprint 36 - Rhodophyta (1-11 April, 2021)
* Sprint 37 - Vampire squid (12-25 April, 2021)
* Sprint 38 - Umbrella octopus (26-30 April, 2021)

Our work can be tracked through the various OONI GitHub repositories:
https://github.com/ooni

Highlights are shared in this report below.

*## Released new OONI Probe test for Signal Private Messenger App*

The OONI Probe apps now feature a brand new test for the Signal Private
Messenger app! You can test the Signal app on your network and check if
it's blocked.

The specification of the OONI Probe Signal test is available here:
https://github.com/ooni/spec/blob/master/nettests/ts-029-signal.md

We also wrote a Signal test description (for a more general audience),
which we published on our website: https://ooni.org/nettest/signal

Through the OONI Probe Signal test, users can check if it’s possible to
establish a connection to the various backend services used by the Signal
app or if Signal is being interfered with on their network.

Following the implementation of our Signal test (
https://github.com/ooni/probe-cli/pull/230), we created a UI for displaying
the test results inside of the OONI Probe apps. We then worked on
integrating the Signal test inside of the OONI Probe mobile and desktop
apps in preparation for the new app releases.

Our new Signal test was shipped as part of OONI Probe Desktop 3.4.0 (
https://github.com/ooni/probe-desktop/pull/202) and OONI Probe Mobile
2.11.0 (https://github.com/ooni/probe-android/pull/422,
https://github.com/ooni/probe/issues/1415).

All Signal test results from around the world are openly published in
near real-time on OONI Explorer:
https://explorer.ooni.org/search?since=2021-03-15&test_name=signal

*## Updated the user guides for OONI Probe Mobile and Desktop Apps*

In April 2021, we published updated versions of our user guides for the
OONI Probe apps:

1) User Guide: OONI Probe Mobile: https://ooni.org/support/ooni-probe-mobile

2) User Guide: OONI Probe Desktop:
https://ooni.org/support/ooni-probe-desktop

Following relevant OONI Probe mobile and desktop releases, we updated the
guides to include new features, settings, step-by-step instructions, and
screenshots (https://github.com/ooni/ooni.org/issues/844).

*## Beta for automated testing on OONI Probe Android*

In April 2021, we released a beta version for automated testing on OONI
Probe Android! This means that users can have OONI Probe test websites
automatically multiple times per day (without having to remember to run
OONI Probe or do anything).

We reached out to community members, requesting that they help test the
beta and report any issues they encounter. Users could help test the beta
version by:

1) Joining the beta program:
https://play.google.com/apps/testing/org.openobservatory.ooniprobe

2) Updating (through the Play Store) to OONI Probe 3.0.0-beta.1 (Android)

3) Through the OONI Probe 3.0.0-beta.1 app, tapping Settings -> Automated
testing and enabling the "Run tests automatically" setting.

The beta included settings for having automated tests run only when users
are connected to WiFi and while their phone is charging (though they can
configure this in the "Automated testing" settings).

To avoid cluttering the UI of the OONI Probe Test Results screen, the test
results from automated testing will only be made available on OONI Explorer
(published automatically in real-time): https://explorer.ooni.org/

*## Improvements to the testing of OONI Probe Android*

We onboarded an external contractor (Bloco) to assist us with the
development of integration and unit tests for OONI Probe Android. This has
led to improving the overall code quality of the app, as well as ensuring
that it’s better tested so that we can have higher confidence in what we
ship to end users.

This work is documented through the following pull request:
https://github.com/ooni/probe-android/pull/426

*## Research on implementing automated testing on iOS*

We carried out research to explore the feasibility of adding support to the
OONI Probe iOS app to perform background tasks (required for automated
testing). We also created a prototype for OONI Probe iOS background tasks:
https://github.com/ooni/probe/issues/1459

*## OONI Probe backend proxy support*

We worked on adding backend proxy support to the OONI Probe apps in order
to make them more resilient to intentional or accidental blocking. More
specifically, we added support that will enable OONI Probe users to use
Psiphon or a custom proxy to communicate with OONI backend services.

In April 2021, we released an alpha version with Psiphon proxy support:
https://github.com/ooni/probe-cli/releases/tag/v3.10.0-alpha. We also
created a UI for the new OONI backend proxy settings in the OONI Probe apps.

All of this work is documented through the following ticket:
https://github.com/ooni/probe/issues/985

*## Preview release of OONI Probe Desktop with RTL support*

In April 2021, we made a preview release of OONI Probe Desktop with RTL
support: https://github.com/ooni/probe-desktop/releases/tag/v3.5.0-rtl.1

This experimental release includes support for languages that use
Right-to-Left (RTL) scripts, such as Arabic and Farsi. We shared this
preview release with community members to get their feedback on how to
improve it before the public release. Based on their feedback, we made a
series of improvements.

*## OONI Probe Command Line Interface (CLI)*

We released OONI Probe CLI 3.9.0 (https://github.com/ooni/probe/issues/1369)
which includes a series of improvements and end-to-end testing. We
continued to work on redesigning the probeservices package (
https://github.com/ooni/probe/issues/1405) to add circumvention support,
making OONI Probe more resilient to censorship. We also completed work
related to implementing the check-in API in OONI Probe CLI (
https://github.com/ooni/probe/issues/1299).

*## Changes to the OONI Probe Android TLS fingerprint*

We noticed that OONI Probe Android was not working properly on some
networks in Iran. We therefore investigated the issue and discovered that
the block had to do with the fact that a specific ordering of the TLS
cipher suites was leading to fingerprinting and blocking on this network.

As a result of this, we patched the golang TLS code to maintain the same
cipher suite ordering as it does on desktop, which resulted in us managing
to unblock the app in Iran. More details about this issue are available
through the following ticket: https://github.com/ooni/probe/issues/1444
(and our relevant blog post:
https://ooni.org/post/making-ooni-probe-android-more-resilient/#changing-our-android-tls-fingerprint).


*## Research on implementing a new Snowflake experiment in OONI Probe*

We carried out research (and testing) necessary for evaluating the
feasibility of implementing a new Snowflake experiment in the OONI Probe
apps and concluded that it is feasible. Our work on this research prototype
is documented through the following ticket:
https://github.com/ooni/probe-engine/issues/283

*## Data quality improvements*

We improved upon the quality of OONI Probe RiseupVPN measurements by fixing
a bug to reduce the ratio of false positives. This work is documented
through the following ticket: https://github.com/ooni/probe/issues/1354

Based on the RiseupVPN test, RiseupVPN was flagged as “likely blocked” even
if a single gateway was blocked, without taking into account the built-in
fallback mechanisms. Moreover, the gateways were indicated as “likely
blocked” if they were malfunctioning for other reasons (such as
misconfiguration, DDoS, reboot on security updates etc.). As a result, this
produced a lot of false positives. To address this, we made a series of
improvements in order to improve the accuracy of the RiseupVPN test and to
reduce the ratio of false positives.

*## Improvements to OONI Explorer*

We added support for grouping test names by the relative test group under
the “Test Name” filter of the OONI Explorer Search page:
https://github.com/ooni/explorer/pull/573

We fixed an issue related to how we were displaying the timestamps in the
summary text of the OONI Explorer measurement pages:
https://github.com/ooni/explorer/issues/541

Thanks to a community contribution, we now have support in the measurement
pages for expanding all the measurement data JSON keys:
https://github.com/ooni/explorer/pull/567

*## Creating a real-time incident response dashboard*

We made considerable progress on the Measurement Aggregation Toolkit, which
allows users to plot aggregate charts of measurements in response to
ongoing censorship events. In particular, we added support for plotting
charts on multiple axes, allowing us to display anomaly counts of
measurements over time per URL.

This work is documented through the following pull request:
https://github.com/ooni/explorer/pull/579

*## Proof of concept for new URL submission web platform*

To enable community members to contribute to the Citizen Lab test lists
(which include URLs measured by OONI Probe), we are creating a web platform
that will enable users to review and contribute to the test lists without
using GitHub (contributions from this new web platform will still end up as
pull requests on the Citizen Lab test list repository for review).

In April 2021, we created a proof of concept backend implementing the full
URL submission functionality. We Integrated the URL submission/editing
backend into the OONI API: https://github.com/ooni/backend/issues/490. This
allows users to contribute URLs to the Citizen Lab test lists without
needing to create a GitHub account or knowing how to format a pull request.
This work is documented through the following ticket:
https://github.com/ooni/backend/issues/487

*## OONI backend*

Throughout April 2021, we worked on the following backend activities:

* Added support to the OONI API for the new OONI Probe Signal test
* Investigated a check-in test list bug (which wasn’t returning entries),
added metrics, fixed the bug, and added fallback options:
https://github.com/ooni/probe/issues/1328
* Monitored the growth of incoming OONI measurements from unattended runs
(to ensure that our infrastructure can handle the increased load and to
check if measurement coverage increases as expected)
* We started evaluating DuckDB as an alternative database solution by
building a test DB and performing a quick benchmark
* Server infrastructure improvements related to VPS that we use for running
OONI Probe CLI experiments
* Worked on switching ooniprobe and backend Debian packages to S3:
https://github.com/ooni/probe/issues/1359
* Made progress on the OONI API registration and login, deployed it on our
testing infrastructure and performed testing with probes:
https://github.com/ooni/backend/issues/491
* Integrated the URL submission/editing backend into the OONI API:
https://github.com/ooni/backend/issues/490
* Discussed backend/db requirements for storing new dimensions around OONI
Web Connectivity measurements (server IP address, multiple DNS resolvers,
blocking taxonomy): https://github.com/ooni/backend/issues/492

*## Dealt with the discontinuation of Bintray*

We used to rely on Bintray for hosting the Debian package repositories for
probe-cli, as well as several internal tooling used in our infrastructure.
Bintray has announced that they are going to discontinue the service (
https://jfrog.com/blog/into-the-sunset-bintray-jcenter-gocenter-and-chartcenter/)
and as a result of this, we have had to migrate our infrastructure away
from Bintray and replace it with an s3 based solution. This work is
documented through the following ticket:
https://github.com/ooni/probe/issues/1437

We also moved the debian package generation over to github actions so that
it’s following the same pattern used in the rest of our repositories.
Moreover we added support for signing of packages and added support for the
detection of duplicate packages.

*## Updated OONI Data Policy*

On 13th April 2021, we published an updated version of the OONI Data
Policy: https://ooni.org/about/data-policy

This updated documentation reflects the changes we made in terms of our use
of analytics in the OONI Probe apps (
https://github.com/ooni/ooni.org/pull/838).

Following a careful audit of various analytics tools (as reported in March
2021), we removed Firebase analytics and app usage metrics entirely from
the OONI Probe mobile app, while ensuring that no analytics are used on
OONI Probe F-Droid. Rather, we limited the OONI Probe mobile app to the use
of Sentry for crash reports (when users opt-in) and to the use of Countly
for sending push notifications on Android. On OONI Probe desktop, we
removed Matomo app usage metrics, and limited the app to the use of Sentry
for crash reports (when users opt-in). We made these changes as part of our
data minimization efforts.

*## Academic paper investigating internet censorship in Myanmar*

We collaborated with CAIDA, Censored Planet, and other researchers on an
academic research paper investigating the censorship events in Myanmar
following the military coup (on 1st February 2021). This paper builds upon
our previous study on censorship events in Myanmar (
https://ooni.org/post/2021-myanmar-internet-blocks-and-outages/), offering
additional data analysis and new insights.

In April 2021, we worked on (and completed) OONI data analysis required for
this academic paper (https://github.com/ooni/ooni.org/issues/855).
Specifically, we analyzed OONI measurements collected from Myanmar with the
aim of examining whether and how censorship events changed in the country
between January 2021 to April 2021, how censorship varied across AS
networks, and we investigated potential cases of collateral damage (as a
result of IP blocks).

Based on our analysis, we produced multiple charts (communicating key
findings) and started working on writing the paper.

*## Data analysis for IPYS Venezuela*

We provided data analysis support to our Venezuelan partner, IPYS Venezuela
(in support of their annual digital rights report). This involved a
year-long analysis of OONI measurements collected from Venezuela, primarily
aimed at identifying the blocking of websites (and how that varies across
AS networks). As a result of this analysis, we produced a relevant chart
and CSV file (https://github.com/ooni/ooni.org/issues/823), which we shared
with IPYS Venezuela in support of their research.

*## Data analysis for Azerbaijan Internet Watch*

In support of our partner, Azerbaijan Internet Watch, we analyzed OONI
measurements collected from Azerbaijan between 1st January 2021 to 30th
April 2021. As part of this analysis, we documented the (ongoing) blocking
of independent news media and circumvention tool websites (based on OONI
data).

Based on our analysis, we produced relevant charts (that summarize key
findings) and a report:
https://www.az-netwatch.org/wp-content/uploads/2021/04/2021-04-Azerbaijan-Internet-Watch-Updates.pdf

Azerbaijan Internet Watch also published a summary of our report findings
on their site:
https://www.az-netwatch.org/news/azerbaijan-keeps-blocking-access-to-key-independent-news-platforms-ooni-april-report/

*## Data analysis for the Tor Project*

We collaborate with the Tor Project’s anti-censorship team on monitoring
Tor blocking around the world. As part of these efforts, we analyzed OONI
measurements collected from Belarus over the last year to examine
torproject.org reachability.

Based on this analysis, we produced and shared relevant CSV files and
charts with the Tor Project’s anti-censorship team.

*## Test list updates*

In April 2021, we updated the Citizen Lab test list for Togo (
https://github.com/citizenlab/test-lists/pull/766) by adding media websites
shared by Reporters Without Borders.

*## Planning the OONI Partner Training 2021*

In April 2021, we started planning the logistics surrounding the
organization of an OONI Partner Training. We were initially hoping to host
an in-person event for all our partners, but in light of the COVID-19
pandemic (and associated travel restrictions), we decided to facilitate the
event entirely online.

Given that our partners are located around the world (from Latin America to
Asia), we needed to consider training formats that would accommodate
different time zones and different community needs. As such, we started
organizational work related to hosting online OONI Partner Trainings in
April 2021 (the events were scheduled for late June 2021 and early July
2021), and created a Concept Note.

*## Ford Foundation Financial Innovation & Resilience Training*

As part of our participation in the Ford Foundation’s Financial Innovation
and Resilience training program, we attended a relevant workshop on 14th
April 2021, and a relevant webinar on 22nd April 2021.

We were offered this great opportunity because we are a grantee of the Ford
Foundation, who support OONI’s community-related work. The knowledge and
skills gained through this training program have helped us rethink and
improve OONI’s fundraising strategies.



*## Community activities### 87th Nexa Lunch Seminar*

On 28th April 2021, OONI’s Simone was a speaker at the Nexa Center for
Internet and Society’s (online) 87th Lunch Seminar (titled “The risks of
digital sovereignty”). Information about the event is available here:
https://nexa.polito.it/lunch-87

*### Research on Twitter throttling in Russia*

Censored Planet produced a great research report examining the throttling
of Twitter in Russia: https://censoredplanet.org/throttling. OONI’s Simone
collaborated with the Censored Planet team and contributed to this research.

*### OONI Community Meeting*

On 27th April 2021, we hosted the monthly OONI Community Meeting on our
Slack channel (https://slack.ooni.org/), during which we discussed the
following topics:

1. Updates from the OONI team: (1) New Signal test, (2) Automated testing
on OONI Probe Android 3.0.0-beta.1, (3) Switching from legacy ooniprobe to
OONI Probe CLI.

2. Updates on internet censorship in Uganda (social media and VPN blocking)
and Tanzania (Twitter blocking).

3. Updates on OONI’s new URL submission web platform.

4. The research benefit of potentially including dead/expired URLs in test
lists.

*## Userbase*

In April 2021, 8,539,001 OONI Probe measurements were collected from 5,141
AS networks in 200 countries around the world.

This information can also be found through our measurement stats on OONI
Explorer (see chart on “monthly coverage worldwide”):
https://explorer.ooni.org/

~ OONI team.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.torproject.org/pipermail/tor-project/attachments/20210715/9e5660a9/attachment-0001.htm>