[tor-project] OONI Monthly Report: July 2021

[Maria Xynou]

Hello,

Hope you're all doing well.

These days we're primarily supported by the DRL, which is why we do
quarterly reporting. That said, we'd like to share monthly updates from the
OONI team with the community, hence our July 2021 report shared below. I'll
also follow-up to share our August and September 2021 reports.

*# OONI Monthly Report: July 2021*

Throughout July 2021, the OONI team worked on the following sprints:

* Sprint 43 - Confused Kraken (5th - 18th July 2021)
* Sprint 44 - Dancing Sirens (19th - 31st July 2021)

Our work can be tracked through the various OONI GitHub repositories:
https://github.com/ooni

Highlights are shared in this report below.

*## Published research report on media censorship in Azerbaijan*

In collaboration with Azerbaijan Internet Watch, we co-published a new
research report titled: "Media censorship in Azerbaijan through the lens of
network measurement".

Our report is available through the following sites:

* OONI: https://ooni.org/post/2021-azerbaijan/

* Azerbaijan Internet Watch:
https://www.az-netwatch.org/news/media-censorship-in-azerbaijan-through-the-lens-of-network-measurement-july-2021-report/

As part of our analysis of OONI measurements collected from Azerbaijan
between January 2020 to May 2021, we found:

** Blocking of independent news media and circumvention tool websites.
*Throughout
the testing period, several independent news media and circumvention tool
sites presented HTTP failures caused by connection timeouts. This suggests
the potential use of Deep Packet Inspection (DPI) by ISPs in Azerbaijan.

** Attempts to block Tor and Psiphon.* ISPs in Azerbaijan attempted to
block Tor and Psiphon amid the 2020 Nagorno-Karabakh war. However, both
attempts appear to have been quite ineffective, as these tools have
built-in circumvention fallback options. Tor Metrics also show a spike in
usage during this period.

** Temporary blocking of social media amid 2020 Nagorno-Karabakh war.* Between
September 2020 to November 2020, several social media websites presented
the same HTTP failures (as news media and circumvention tool sites), while
the testing of WhatsApp and Telegram presented signs of TLS level
interference.

** Variance of censorship across networks.* ISPs in Azerbaijan appear to be
adopting similar censorship techniques. However, censorship varies from
network to network, as different ISPs block different websites and apps at
different moments in time.

*## Hosted a 3-day online OONI Partner Training*

Between 5th-7th July 2021, we organized and hosted a 3-day online OONI
Partner Training for our partners from 9 countries in Asia and the Middle
East (https://ooni.org/partners). The event also included participants from
international organizations, Reporters Without Borders and Human Rights
Watch.

Overall, 45 participants from 23 digital rights organizations attended this
3-day OONI Partner Training. To ensure a safe, friendly, and pleasant
experience for all during the training, we asked all participants to adhere
to OONI’s Code of Conduct (
https://github.com/ooni/ooni.org/blob/master/CODE_OF_CONDUCT.md) and to
follow the Chatham House Rule on Privacy (
https://www.chathamhouse.org/about-us/chatham-house-rule).

The goal of the OONI Partner Training was to share knowledge and skills on
OONI’s censorship measurement tools, dataset, and methodologies to enable
their use as part of research and advocacy efforts. We also aimed to
collect feedback on how to improve OONI tools to better serve community
needs.

The event was facilitated entirely online (over Zoom), and entailed two 1.5
hour sessions per day (over 3 days). We facilitated the following 6
sessions as part of the 3-day OONI Partner Training:

1) Information Controls Around the World

2) Introduction to Internet Censorship

3) Using the OONI Probe Apps

4) Using OONI Run to coordinate censorship testing

5) Using OONI Explorer to find censorship measurement data

6) Interpreting OONI data

We published a blog post which shares further information about the
training sessions and overall event:
https://ooni.org/post/ooni-partner-training-2021/

At the end of the 3-day training, we shared a post-training survey (
https://forms.gle/n62ktvcfD1u7DgPMA) with all participants to collect their
feedback on how to improve future OONI training events.

To support ongoing learning, we shared a Resources document (
https://ooni.org/documents/2021-ooni-partner-training-resources/ooni-resources.pdf)
with participants that includes OONI-related resources, as well as other
resources (on censorship circumvention, digital security, monitoring
internet connectivity shutdowns, and other censorship measurement
platforms) that participants expressed interest in.

We also shared OONI training slides (
https://docs.google.com/presentation/d/1mpoB_ObjMH0EWQhyZRpKWnwl5PUHf-Sa5PKaVclDEH8/edit?usp=sharing)
that they can use (and customize) to facilitate OONI workshops for their
communities.

In July 2021, we issued OONI certificates for participants who attended our
2 OONI Partner Training events (
https://ooni.org/post/ooni-partner-training-2021/).

*## Blocking of social media in Cuba amid protests*

On 12th July 2021, Cuba started blocking access to online social media
services amid anti-government protests.

On the same day, we reported on the blocking of the WhatsApp, Telegram and
Signal apps based on real-time data collected from Cuba:
https://twitter.com/OpenObservatory/status/1414622433156476930

OONI data on the blocking of WhatsApp, Telegram, and Signal in Cuba is
available through the following links:

* Signal:
https://explorer.ooni.org/search?until=2021-07-13&since=2021-06-12&probe_cc=CU&test_name=signal

* Telegram:
https://explorer.ooni.org/search?until=2021-07-13&since=2021-06-12&probe_cc=CU&test_name=telegram

* WhatsApp:
https://explorer.ooni.org/search?until=2021-07-13&since=2021-06-12&probe_cc=CU&test_name=whatsapp


The blocking of the Signal Private Messenger App in Cuba was detected based
on OONI’s new Signal test (https://ooni.org/nettest/signal), which was
shipped in April 2021.

Our new Signal test also contributes ongoing measurements on the blocking
of the Signal app in Iran (
https://explorer.ooni.org/search?until=2021-07-30&since=2021-06-30&probe_cc=IR&test_name=signal)
and China (
https://explorer.ooni.org/search?until=2021-07-30&since=2021-06-30&probe_cc=CN&test_name=signal),
among other countries.

We recently added support to our backend for the adjustment of URL
priorities (as part of URL prioritization work). This allows us to specify
in which countries certain URLs should have higher priority for testing and
empowers us to dynamically adjust these in response to censorship events.

Following the social media blockade in Cuba, we bumped up the priority of
the GRP (Social Media) category code (from the Citizen Lab test lists:
https://github.com/citizenlab/test-lists/blob/master/lists/00-LEGEND-new_category_codes.csv)
on 14th July 2021 for Cuba. This enabled us to detect (and collect
measurements on) the temporary blocking of facebook.com in Cuba (
https://explorer.ooni.org/search?until=2021-07-30&since=2021-06-30&probe_cc=CU&domain=www.facebook.com&only=anomalies).
It also enabled us to detect (and collect measurements on) the blocking of
tiktok.com in Cuba (
https://explorer.ooni.org/search?until=2021-07-30&since=2021-06-30&probe_cc=CU&domain=www.tiktok.com).
Following the prioritization of URLs from this category, we were able to
collect a total of 960 measurements for the GRP category code from Cuba.

OONI’s response to the recent social media censorship events in Cuba has
received the following press coverage:

https://www.nbcnews.com/tech/tech-news/cubans-protest-government-cracks-internet-access-messaging-apps-rcna1400

https://www.wsj.com/articles/internet-powered-mass-protests-in-cuba-then-the-government-pulled-the-plug-11626358893?page=1

https://cpj.org/2021/07/journalists-detained-and-harassed-internet-disrupted-amid-cuban-protests/

Cuban organization, YucaByte, published a report documenting the censorship
events in Cuba amid the protests (citing OONI data):
https://www.yucabyte.org/2021/07/12/11j-internet-cuba/

*## OONI Probe Mobile*

We released OONI Probe Android 3.2.0 (
https://github.com/ooni/probe-android/releases/tag/v3.2.0) and OONI Probe
iOS 3.2.0 (https://github.com/ooni/probe-ios/releases/tag/v3.2.0).

This release includes:

* A reminder encouraging users to enable automated testing
* A badge to indicate that a proxy is being used
* Support for minimizing a running test
* Support for validating URLs shared via OONI Run (
https://github.com/ooni/probe-android/pull/443)
* Font size fix on iPhone SE (
https://github.com/ooni/probe-ios/commit/8f3fcf90b286e7e9ffee18c4b8ea223f2f3470e4
)

These features were added to encourage users to enable automated testing
and contribute measurements regularly. In response to community requests,
we also added a badge to indicate that a proxy is being used, and we added
support for minimizing a running test (enabling users to make use of other
features of the OONI Probe app while a test is running).

*## OONI Probe Desktop*

We released OONI Probe Desktop 3.5.2:
https://github.com/ooni/probe-desktop/releases/tag/v3.5.2

This release fixed a critical bug in autoupdate which prevented downloading
and installing updates.

In July 2021, we provided mentorship to Google Summer of Code (GSoC)
student Shivam Shekhar, who worked on implementing end-to-end tests for
OONI Probe Desktop. In particular, he implemented tests for the code
responsible for running the OONI Probe command line tool (
https://github.com/ooni/probe-desktop/pull/231). Moreover, tests were
implemented for several UI components as part of the following pull
request: https://github.com/ooni/probe-desktop/pull/229

*## Progress on browser-based OONI Probe*

We made some progress on the browser based implementation of OONI Probe.
Specifically, we integrated React into the project so that we can reuse the
existing components we developed for web based projects (see:
https://github.com/ooni/probe-web/commit/a9da79b9384f7f496bb3278a4a00cacc39973e11).
We also did some research into how to tweak the fetch API calls so that we
minimise the amount of sensitive data transmitted to the measured web hosts
(see:
https://github.com/ooni/probe-web/commit/5bf0ba41a9e2464da070ba21f4463ddce23a3be5
).

*## Expanding OONI censorship measurement methodologies*

We looked into alternative TLS stacks that we could use for performing
measurements, but potentially also for communicating with our backend
services. This was prompted by the fact that we have noticed that some
countries (such as Iran) will implement blocking of all TLS connections
that present a specific fingerprint.

In particular, we looked into https://github.com/refraction-networking/utls and
researched how we could go about integrating it into the OONI codebase, as
well as what work would be necessary to update it to the latest version of
the golang TLS library. This work is documented through the following
ticket: https://github.com/ooni/probe/issues/1702

In July 2021, we provided mentorship to Google Summer of Code (GSoC)
student Kathrin Elmenhorst, with whom we collaborated on making several
improvements to how errors are handled inside of our measurement engine.
Specifically, we ensured that it’s possible to accurately map more classes
of errors to OONI error strings, as documented through the following
ticket: https://github.com/ooni/probe/issues/1505

We also made some progress on comparing the new golang-written Web
Connectivity test helper to the old python-based one. The goal of this
comparison is to determine how well-suited the new test helper is at
replacing the legacy one and if, in doing the switch from one to the other,
we are introducing any inaccuracies in the resulting measurements.

*## Created mockups for new OONI Probe Snowflake test results*

We implemented mockups for displaying the test results of our new Snowflake
pluggable transport test inside of OONI Probe. This has been tracked as
part of the following issue: https://github.com/ooni/ooni.org/issues/759

*## Research on whether to ship PTs as a separate binary*

We carried out research to explore whether to ship pluggable transports as
separate binaries or link them with the ooniprobe binary. Based on this
research, we wrote an internal document which explains our design choices,
PT spec considerations, and conclusions. In the end, we chose to integrate
our new Snowflake experiment into ooniprobe as a Go library leveraging
version 2 of the pluggable-transports specification:
https://github.com/ooni/ooni.org/issues/765

*## OONI backend*

In July 2021, we worked on the following OONI backend activities:

* Fixed certbot nginx configuration bug (server/port overlap)
* Made improvements to the API rate limiting system and developed new
integration tests to ensure it’s working as expected:
https://github.com/ooni/api/pull/256
* Added support for the global prioritized testing of popular social media
URLs: https://github.com/ooni/backend/issues/507
* Added support for the prioritized testing of the GRP Citizen Lab category
code in Cuba
* Worked on the test helper rotation tool which automatically deploys on
cloud infrastructure machines for running backend infrastructure. These are
rotated automatically on a periodic basis:
https://github.com/ooni/backend/issues/496
* As part of the development of the rotating test helpers, we also did a
major rework to how TLS certificates are generated. Specifically we
integrated it with DNS and certbot and deployed it to production.
* Test with probes on test API.
* API: Deployed test.ooni domains in CORS:
https://github.com/ooni/api/pull/259
* API: Switched URL submission to account_id:
https://github.com/ooni/api/pull/260
* Reprocessed legacy measurements and created JSONL files
* Monitored measurement coverage by unattended runs
* Updated internal ASN coverage charts
* Developed charts that make predictions on the expected measurement volume
based on the submission API. We have these charts implemented in our
internal backend dashboards and have configured alerts to notify us if the
expected measurement volume falls below a certain threshold.
* Implemented support for running end-to-end integration tests of the API
and fastpath on our continuous integration platform. Through this setup we
can ensure that a large part of the backend infrastructure is working as
expected and that the API and fastpath are run on real OONI measurement
data: https://github.com/ooni/api/pull/258.
* Refactored scripts to run tests on different environments

*## Building a web platform for test list updates*

We deployed the private beta of our new web platform for test list updates
to production (https://test-lists.ooni.org/login). We worked on fixing
issues reported as part of internal team testing of the platform (
https://github.com/ooni/test-lists-ui/pull/2). We also shared the private
beta with a few community members to collect their feedback, based on which
we worked on further improvements, which are documented as issues here:
https://github.com/ooni/test-lists-ui/issues

Once the private beta is further polished, we will share it more
extensively to collect further community feedback before launching the
platform.

*## Collaboration with Netalitica*

Netalitica researchers continued to do excellent work in reviewing and
updating more Citizen Lab test lists. In July 2021, we reviewed their
updates to the test lists for India and Burundi and shared feedback for
improvements.

*## Test list updates*

In July 2021, we contributed updates to the following Citizen Lab test
lists:

* Russia: https://github.com/citizenlab/test-lists/pull/805
* Ethiopia: https://github.com/citizenlab/test-lists/pull/804
* Armenia: https://github.com/citizenlab/test-lists/pull/803

*## New partnership with Internet Society (ISOC)*

In July 2021, we established a new partnership with Internet Society (ISOC).

As part of this partnership, OONI is now a data partner (
https://pulse.internetsociety.org/partners) for ISOC’s Pulse project on
Internet shutdowns: https://pulse.internetsociety.org/shutdowns

As a data partner, we will provide ISOC with relevant OONI data in support
of their Pulse project which maps internet shutdowns around the world.



*## Community activities### Zaina Foundation Internet Shutdown Training*

On 12th and 13th July 2021, Zaina Foundation organized and facilitated an
Internet Shutdown Training for human rights defenders in Tanzania. One of
the core objectives of this training was to train coalition members on OONI
censorship measurement methodologies and to explore OONI data.

As part of this training, Zaina Foundation facilitated the following 2 OONI
workshops:

1) Introduction to OONI Probe and OONI Explorer
2) OONI practical session

Following the training, Zaina Foundation shared community feedback with us
and coordinated with local communities on translating OONI Probe to Swahili.

*### OONI presentation at the Nexa Center for Internet and Society*

On 28th July 2021, OONI’s Arturo presented OONI at the Nexa Center for
Internet and Society of the Polytechnic University of Turin. As part of his
presentation, Arturo provided OONI updates on the state of internet
censorship worldwide in 2020.

Information about the event is available here:
https://nexa.polito.it/lunch-90

The presentation can be viewed through the following link:
https://www.youtube.com/watch?v=fO53FLTu4a8

*### Access Now webinar on internet shutdowns*

On 30th July 2021, OONI’s Maria presented OONI as part of Access Now’s
webinar titled: “#KeepItOn during an internet shutdown: How to measure,
document, and circumvent network interference”. In particular, she
discussed how OONI’s tools and data can be used for measuring partial
internet shutdowns (involving the blocking of websites and apps).

This webinar was live-streamed on YouTube: https://youtu.be/aFIBrRXKm2E

*### OONI documentation in Farsi*

Iranian community members created detailed and comprehensive documentation
which explains OONI tools and methodologies in Farsi.

This documentation is available here:
https://wikicensorship.github.io/fa/docs/measure-internet-censorship/OONI/

The Farsi documentation covers the following areas:

* Investigate Internet censorship through the OONI probe
* How Websites Test Works
* Directly enter custom addresses in the application
* Create a custom test list
* Analysis of results
* Simple analysis of results
* Advanced results analysis
* Diagnosis of sanctions
* Internal breaker setting only for backend connection
* Data mining in OONI results data

*### OONI Probe video tutorial in Spanish*

Our Bolivian partners, Fundación Internet Bolivia (
https://ooni.org/partners/fundacion-internet-bolivia.org/), created a video
tutorial in Spanish where they present OONI and explain OONI Probe,
providing a live demo of using the tool.

This is available here:
https://peertube.tv/videos/watch/e6991978-a3d7-4718-b688-482a9a4de525

*### OONI Community Meeting*

On 27th July 2021, we hosted the monthly OONI Community Meeting on our
Slack channel (https://slack.ooni.org/), during which we discussed the
following topics:

1) New web platform for updating the Citizen Lab test lists: Request for
community feedback

2) Adding Signal under instant messaging tests for Debian/Ubuntu OONI Probe
users

3) Backend proxy for OONI Probe Desktop and OONI Probe CLI

*## Userbase*

In July 2021, 13,522,345 OONI Probe measurements were collected from 4,969
AS networks in 195 countries around the world.

This information can also be found through our measurement stats on OONI
Explorer (see chart on “monthly coverage worldwide”):
https://explorer.ooni.org/

~ OONI team.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.torproject.org/pipermail/tor-project/attachments/20211012/47553284/attachment-0001.htm>