[tor-project] OONI Monthly Report: July 2023

[Maria Xynou]

Hello,

This email shares OONI's monthly report for July 2023.

*# OONI Monthly Report: July 2023*

Throughout July 2023, the OONI team worked on the following sprints:

* Sprint 95 (3rd - 16th July 2023)
* Sprint 96 (17th - 30th July 2023)

Our work can be tracked through the various OONI GitHub repositories:
https://github.com/ooni

Highlights are shared in this report below.

*## New partnership*

We are excited to have formed a new partnership with Digital Rights Lab
Sudan (https://twitter.com/DRLab_Sudan)!

The goal of the partnership is to collaborate on the study of internet
censorship in Sudan. We published a dedicated partner page for Digital
Rights Lab Sudan, highlighting their important work:
https://ooni.org/partners/drlab/

*## Published report on China blocking OONI*

In July 2023, China started blocking access to our website (ooni.org) and
censorship measurement app (OONI Probe).

We published a report documenting the blocking of OONI in China with OONI
data: https://ooni.org/post/2023-china-blocks-ooni/

The block appears to be implemented by means of DNS injection and TLS
interference. This is the first time that we're observing a large and
persistent drop in OONI measurement coverage as a result of OONI Probe
blocking. The report also includes advice for circumventing the blocking of
OONI Probe in China. However, running OONI Probe in China is now probably
riskier.

*## OONI Probe Mobile*

Throughout July 2023, we worked towards the release of OONI Probe Mobile
3.8.3. As part of this, we worked on extensive QA testing for both Android (
https://github.com/ooni/probe/issues/2504) and iOS (
https://github.com/ooni/probe/issues/2509).

Based on community feedback, we worked on adding functionality to OONI
Probe Android to automatically test new networks (
https://github.com/ooni/probe/issues/900).

We also worked on improvements to a new Android app (“News Media Scan”)
that we have been developing in collaboration with Deutsche Welle over the
past months.

*## OONI Probe Desktop*

In July 2023, we released OONI Probe Desktop 3.9.1 (
https://github.com/ooni/probe-desktop/releases/tag/v3.9.1) with the latest
version of OONI Probe CLI and updated dependencies.



*## OONI Run### Building OONI Run v.2*

In July 2023, we added experimental support to the OONI Probe Engine for
invoking the fetch API allowing a client to obtain OONI Run v2 descriptors (
https://github.com/ooni/probe-cli/pull/1176). This functionality allows the
OONI Probe mobile app to start implementing OONI Run v2 (
https://github.com/ooni/probe-android/pull/582). We then worked towards
adding support for running OONI Run v2 links in the OONI Probe mobile app (
https://github.com/ooni/probe/issues/2474). The general idea for this first
iteration is to implement minimal functionality into the Engine and let the
app do most of the work.

Notably, we reached a first milestone where we have an experimental version
of the OONI Probe Android app where it is possible to:
* Inspect and install an OONI Run v2 link into the OONI Probe Android app
dashboard;
* Run such a link directly or together with other nettests;
* See the results it produced reusing the view used for experimental
nettests.

Subsequent development milestones will focus on implementing the missing
functionality (e.g., updating OONI Run v2 links) and on optimizing the
implementation.

We continued working on the UI improvements for the OONI Run v2 web editor (
https://github.com/ooni/run/pull/131). We also added backend support for
separating the versioning of the copy (included in translations) from the
nettest data (https://github.com/ooni/backend/issues/694), and we added
localization support to OONI Run v.2 (
https://github.com/ooni/backend/pull/699). To enable the development of the
OONI Run platform, we allowed login redirection for more domains on the API
(https://github.com/ooni/backend/pull/697).

*### Launch of OONI Run supporting localization*

In July 2023, we launched OONI Run (https://run.ooni.io/) with support for
9 languages (beyond English): Farsi, Russian, Thai, Turkish, Chinese
(Taiwan), Chinese (China), German, Portuguese (Brazil), Spanish. This work
is available here: https://github.com/ooni/run/pull/132/files

We thank the Localization Lab community (https://www.localizationlab.org/)
for making OONI Run available in so many languages, and for their ongoing
OONI Run localization efforts (https://explore.transifex.com/otf/ooni-run/).

*## OONI Probe CLI*

We started preparing the release of OONI Probe CLI 3.18.1, which should
include (a) better support for measuring throttling with Web Connectivity
LTE; (b) updated dependencies and assets.

We also continued to make progress towards OONI Probe CLI v3.19.0.
Following up from the work done in previous sprints, we have continued to
take advantage of the netem QA framework and we have started migrating some
nettests to use it for QA purposes. By using netem, we are able to exercise
more censorship conditions in an easier way compared to the previous
framework we used (Jafar, based on Linux and iptables).

For now, we have migrated the following experiments to netem:

* dnsping (https://github.com/ooni/probe-cli/pull/1177);
* facebook_messenger (https://github.com/ooni/probe-cli/pull/1178);
* sniblocking (https://github.com/ooni/probe-cli/pull/1179);
* whatsapp (https://github.com/ooni/probe-cli/pull/1181);
* telegram (https://github.com/ooni/probe-cli/pull/1182).

As a result, the availability of better QA tests opens up the possibility
of being more confident in rewriting these experiments to use the DSL
introduced as part of the richer input error.

*## Expanding OONI’s testing model to support richer testing input*

We continued to work on supporting richer testing input (
https://github.com/ooni/ooni.org/issues/1291). In previous months, we
worked towards writing a DSL (
https://en.wikipedia.org/wiki/Domain-specific_language) to support richer
testing input (https://github.com/ooni/probe/issues/2494) in the context of
our richer testing prototype (https://github.com/ooni/2023-05-richer-input).
We completed this task in early July 2023 (
https://github.com/ooni/2023-05-richer-input/pull/7,
https://github.com/ooni/2023-05-richer-input/pull/8).

We now have a DSL that is both external and internal. The internal DSL is
executable code but can also be serialized to JSON. We can read the JSON
and execute it as an external DSL. We basically use the same code for both
DSLs. The internal DSL uses Go generics and each stage in the measurement
pipelines has specific input and output types. The external DSL wraps each
internal DSL stage with a stage that performs type checking at runtime.

The final DSL (
https://github.com/ooni/2023-05-richer-input/releases/tag/v0.3.0) is not
Turing complete and only supports stage composition. We have primitive
stages (e.g., DNS lookups, TCP connect, TLS handshake, QUIC handshake, HTTP
transactions). We have stages for composing together measurement pipelines
(e.g., we can compose a TCP connect operation and a TLS handshake
operation). And we have primitives to compose pipelines that perform DNS
lookups with pipelines that perform endpoint measurements (e.g., TCP
connect, TLS handshake).

Now that we have finished researching, we have also reached a better degree
of clarity on what we can potentially achieve by expressing richer input
using this DSL:

1) We can run code directly (internal DSL) or we can generate JSON that we
can serve to OONI Probes (external DSL). By serving code to probes, we can
update the definition of nettests. So, in the future, we would need less
emergency releases when IM apps (such as Signal) change their testing
requirements (e.g., they change the CA they use).

2) We can incrementally roll out better implementations of experiments
without making new OONI Probe releases. For example, we can (easily!) add
support for TLS handshakes to the Facebook Messenger nettest and run this
extended version of the nettest only in a subset of probes, closely
monitoring whether everything is working as intended, ready to back out the
changes in case of issues, or roll out the functionality more extensively
if all looks good.

3) We can support the use case of using OONI Probe for research. We (or
anyone else concerned) will potentially be able to feed a specific
measurement DSL pipeline to OONI Probe consisting of the operations that
matter for research. For example, we can imagine serving extra follow-up
tests that would run as part of the experimental suite on an opt-in basis
in case there is a need to further investigate some censorship findings.

To investigate the suitability of the DSL, we reimplemented the Facebook
Messenger (https://github.com/ooni/2023-05-richer-input/pull/7) and
RiseupVPN (https://github.com/ooni/2023-05-richer-input/pull/12)
experiments. The latter, in particular, seems a good candidate to start
upgrading production code to use the DSL since it is currently disabled due
to concerns involving false positives. For this reason, we also sketched
out an initial plan to upgrade it to use the DSL (
https://github.com/ooni/probe/issues/2502).

In subsequent months, we will continue to investigate this topic, refine
our implementation, engage in internal discussions, work on streamlining
possible use cases and on defining a path to start rolling this feature out
in production. In the meanwhile, we have started to feel more confident
about the DSL implementation, as we started adding integration testing
using netem (https://github.com/ooni/netem) to it (
https://github.com/ooni/2023-05-richer-input/pull/7,
https://github.com/ooni/2023-05-richer-input/pull/9).

*## Creating a throttling measurement methodology*

Regarding measuring throttling (https://github.com/ooni/ooni.org/issues/1296),
we finished improving support for measuring the download speed (
https://github.com/ooni/probe/issues/2493). As Web Connectivity LTE now
implements collecting download speed samples during a download, we started
preparing for making it the default Web Connectivity implementation and for
retiring version v0.4. To this end, we started adding support for
netem-based integration testing (https://github.com/ooni/probe-cli/pull/1170)
and we backported the patch to collect download speed samples (
https://github.com/ooni/probe-cli/pull/1166) to the release/3.18 branch so
that we can include this functionality in the upcoming 3.18.1 OONI Probe
release.

*## OONI Explorer*

In July 2023, we released OONI Explorer in the following 2 additional
languages:

* Swahili: https://explorer.ooni.org/sw
* Traditional Chinese: https://explorer.ooni.org/zh-Hant

Beyond English, OONI Explorer now supports 10 languages!

We thank the Localization Lab (https://www.localizationlab.org/) community
for making OONI Explorer – and real-time open data on internet censorship
worldwide – available in additional languages!

*## Creating a Social Media Censorship Alert System*

We continued to make progress towards building a Social Media Censorship
Alert System (https://github.com/ooni/backend/pull/651). Based on internal
review of the code, we updated the internal design document for further
review.
We added support for real-time notifications on Slack in the Social Media
blocking event detector, along with a summary HTML page. We also created a
dedicated Jupyter notebook to show a summary of recent events flagged as
social media blocking alerts by our system.

*## Creating a Censorship Incident Reporting Platform*

In July 2023, we continued working on the UI implementation of the (new)
Censorship Incident Reporting Platform (
https://github.com/ooni/explorer/pull/862). Specifically, we added form
validations, we improved the layouts (also) by further expanding our
component library (https://github.com/ooni/design-system/pull/163), and we
added the report index view.

*## Automating censorship detection and characterization based on OONI
measurements*

Improvements were made to how errors are handled while writing observations
to the database. Specifically, support was added to dump failing rows to
file and retry writing the affected row in the event of a failure (
https://github.com/ooni/data/pull/33). Through this we were able to
identify some bugs in the processing of certain types of measurements and
implement fixes for them.

We started reaching out to third party researchers interested in using this
new observation database and training them on how to use it. As part of
this, we came up with a setup that allows independent researchers to
connect to the database used by the OONI data analysis tool (
https://github.com/ooni/data) and run queries on the platform.

*## OONI backend*

We improved our Jupyter notebook that tracks test helper failures grouped
by datacenter location. It detected a 1-hour long event impacting
0.th.ooni.org. The event was investigated and found to be benign.

We continued ongoing work to predict the amount of incoming measurements on
a per-country basis using basic machine learning in order to send alarms to
the team in the event of a sudden loss of measurements.

*## Published OONI Outreach Kit printing materials*

In July 2023, we updated the OONI Outreach Kit to include materials
designed for printing (https://github.com/ooni/ooni.org/pull/1394).

The updated OONI Outreach Kit (including printing materials) is available
here: https://ooni.org/support/ooni-outreach-kit/

*## Interviewed Project Manager candidates*

In July 2023, we reviewed the final applications and we completed the
second round of interviews for the OONI Project Manager position (
https://ooni.org/post/2023-job-opening-ooni-project-manager/). Based on
these interviews, we reached a hiring decision and followed up with
candidates.

*## FOCI 2023 Community Award*

On 10th July 2023, OONI’s Maria received the Free and Open Communications
on the Internet (FOCI) 2023 Community Award, recognizing the impact of her
work in the field of Internet freedom on the wider community.

We dedicate this award to the whole OONI community, without whom this work
would not be possible (
https://twitter.com/agrabeli_/status/1678722087953760256).



*## Community use of OONI data### Report of the Tehran Electronic Commerce
Association*

In July 2023, the Tehran Electronic Commerce Association published a report
on the low quality of the Internet in Iran, citing OONI in many pages
(pages 4, 5, 6, 16, 20, 35, 37). Their report is available here:
https://etchamber.ir/wp-content/uploads/2023/07/Iran-Internet-v0.9.pdf

OONI was also cited in an article by Peivast (one of Iran’s largest tech
outlets), discussing the findings of the report of the Tehran Electronic
Commerce Association: https://peivast.com/p/170337

*### Blocking of satirical news website in Jordan*

Our partner, JOSA (https://ooni.org/partners/josa/), reported on the
blocking of a satirical news media website (AlHudood) in Jordan, citing
OONI data: https://twitter.com/jo_osa/status/1677041740107198465

*### TEDIC’s research report examining internet shutdowns in Northern
Paraguay*

Our partner, TEDIC (https://ooni.org/partners/tedic/), published a new
research report investigating potential disruptions to internet access in
the northern zone of Paraguay. We supported this study by providing
relevant OONI data analysis.

Their report was published in:
* Spanish:
https://www.tedic.org/wp-content/uploads/2023/07/Informe-Apagones-de-Internet-2023.pdf

* English:
https://www.tedic.org/wp-content/uploads/2023/07/Internet-Shutdowns-Report-2023.pdf

TEDIC also published a blog post about the study in:
* Spanish:  https://www.tedic.org/apagones-de-internet-en-paraguay/
* English: https://www.tedic.org/en/internet-shutdowns-in-paraguay/

*### Sinar Project’s report on the blocking of MalaysiaNow*

Our partner, Sinar Project, published a report documenting the blocking of
MalaysiaNow in Malaysia, through the use of OONI tools and data. Their
report is available here:
https://imap.sinarproject.org/news/blocking-of-malaysianow-website

*### Sinar Project’s report on the blocking of a website of a former
Malaysian MP*

Sinar Project published a report documenting the blocking of a website (
weechookeong.com) which belongs to a former Malaysian Member of Parliament.
Their report makes use of OONI data, and is available here:
https://imap.sinarproject.org/news/internet-censorship-update-blocking-of-weechookeong-com-website

This report received press coverage from MalaysiaNow:
https://www.malaysianow.com/news/2023/07/28/online-freedom-monitor-confirms-mcmc-behind-latest-site-block

*### EngageMedia’s Advocacy Toolkit for Civil Society Organizations*

In July 2023, EngageMedia published “Addressing Internet Censorship and
Content Filtering: An Advocacy Toolkit for Civil Society Organizations”
which features OONI tools and resources (
https://engagemedia.org/2023/toolkit-internet-censorship/).

Their toolkit is available here:
https://engagemedia.org/wp-content/uploads/2023/07/GIF-Regional-Toolkit-Addressing-Internet-Censorship-and-Content-Filtering.pdf

*### PhD thesis on modeling and characterizing internet censorship
technologies*

In July 2023, a PhD thesis (titled “Modeling and Characterization of
Internet Censorship Technologies”) by Alexander Master was published
(making use of OONI data): https://www.doi.org/10.25394/PGS.23666784

*### FOCI 2023 academic paper studying worldwide view of nation-state
Internet censorship*

In July 2023, an academic paper studying a worldwide view of nation-state
Internet censorship was published and presented at FOCI 2023. This paper
makes use of OONI data for its analysis.

The paper is available here:
https://www.petsymposium.org/foci/2023/foci-2023-0008.pdf

*### Other FOCI 2023 papers citing OONI*

OONI was also cited in the following FOCI 2023 papers:

* “Crowdsourcing the Discovery of Server-side Censorship Evasion
Strategies”: https://www.petsymposium.org/foci/2023/foci-2023-0016.pdf
* "Towards a Comprehensive Understanding of Russian Transit Censorship":
https://www.petsymposium.org/foci/2023/foci-2023-0012.pdf
* "Detecting Network Interference Without Endpoint Participation":
https://www.petsymposium.org/foci/2023/foci-2023-0010.pdf



*## Community activities### FOCI 2023*

On 10th July 2023, OONI’s Arturo attended the Free and Open Communications
on the Internet (FOCI) 2023 in Lausanne, Switzerland. Information about the
workshop is available here: https://foci.community/

*### Keynote at HotPETs 2023*

On 14th July 2023, OONI’s Maria delivered a keynote (titled “The Power of
Community Participation in Shaping Digital Rights Tech”) at HotPETs 2023 in
Lausanne, Switzerland.

Information about the workshop is available here:
https://petsymposium.org/2023/hotpets.php

*### OONI workshop for Legal Resource Center in South Africa*

On 24th July 2023, OONI’s Elizaveta facilitated an online OONI workshop for
the team members of South Africa’s Legal Resource Center (
https://lrc.org.za/). The goal of the workshop was to introduce lawyers to
the use of OONI tools and data.

*### OONI workshop for civil society in Zimbabwe*

On 31st July 2023, OONI’s Elizaveta facilitated an online OONI workshop for
civil society groups in Zimbabwe, in preparation for the country’s upcoming
elections (which may trigger censorship events).

*### OONI Community Meeting*

On 25th July 2023, we hosted the monthly OONI Community Meeting on our
Slack channel (https://slack.ooni.org/), during which we discussed the
following topics:

1) Updates from the OONI team
2) IODA Update: Design Standards and Outage Severity Map (
https://ioda.inetintel.cc.gatech.edu/reports/ioda-update-designstandards-outagemap/
)
3) Community feedback on the thematic presentation of censorship findings
on OONI Explorer
4) Community feedback on push notifications: How should OONI utilize them
going forward? What would be most useful?

*## Measurement coverage*

In July 2023, 62,256,645 OONI Probe measurements were collected from 3,135
networks in 172 countries around the world.

This information can also be found through our measurement stats on OONI
Explorer (see chart on “monthly coverage worldwide”):
https://explorer.ooni.org/

~ OONI team.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.torproject.org/pipermail/tor-project/attachments/20231016/6f8fbba2/attachment-0001.htm>