[ooni-talk] OONI Monthly Report: May 2022

[Maria Xynou]

Hello,

OONI's May 2022 status report is shared below.

*# OONI Monthly Report: May 2022*

Throughout May 2022, the OONI team worked on the following sprints:

* Sprint 64 (1st - 8th May 2022)
* Sprint 65 (9th - 22nd May 2022)
* Sprint 66 (23rd - 31st May 2022)

Our work can be tracked through the various OONI GitHub repositories:
https://github.com/ooni

Highlights are shared in this report below.

*## Job opening for OONI Community Coordinator*

In May 2022, we published a job opening for hiring a Community Coordinator
for OONI: https://ooni.org/post/2022-job-opening-ooni-community-coordinator/

In an effort to help ensure that our job opening reached interested
candidates around the world, we worked on quite extensive outreach efforts.
In particular, we posted the job opening on:
* All OONI social media channels (Twitter, Mastodon, Facebook, Instagram,
Linkedin);
* All OONI mailing lists;
* Community mailing lists (such as those of the #KeepItOn campaign, Global
Voices Advocacy, and NetRights);
* OONI Slack channel (and other relevant Slack channels);
* Digital Rights Job Board (https://www.digitalrights.community/job-board);
* Hasgeek Job Board (https://hasjob.co/openobservatory.org/12jm2);
* FOSS Job Board (
https://www.fossjobs.net/job/10936/ooni-community-coordinator-at-open-observatory-of-network-interference-ooni/
).

We also reached out to our partners and many other community members,
requesting that they help share this opportunity with their networks.

Following the publication of this job opening, we worked on tracking and
reviewing incoming applications, and following-up with applicants.

*## OONI Probe Mobile*

Throughout May 2022, we worked on various improvements to the OONI Probe
mobile app in preparation for the next release.

Specifically, we worked on:
* Including the Vanilla Tor experiment as part of the Experimental card of
OONI Probe (https://github.com/ooni/probe/issues/2103);
* Ensuring that the Vanilla Tor and Tor Snowflake experiments only run as
part of automated testing (https://github.com/ooni/probe/issues/2102);
* Providing a working sample for Flutter on OONI Probe apps with probe-cli
integration (https://github.com/aanorbel/probe-shared);
* Enabling users to see all logs (https://github.com/ooni/probe/issues/1777
, https://github.com/ooni/probe-android/pull/516);
* Fixing an iOS bug affecting the icons displayed when all tests are run (
https://github.com/ooni/probe/issues/2055).

*## OONI Run*

We implemented a fix related to the copy-pasting of URLs into the OONI Run
web interface (https://github.com/ooni/run/pull/105). Based on community
feedback, we explored how we can add support to OONI Run for saving custom
test lists (https://github.com/ooni/probe/issues/867), and we had team
meetings to discuss possible implementation approaches. We also wrote an
internal design document on implementation approaches for addressing the
most pressing OONI Run community needs (as identified through a previous
OONI Run usability study and ongoing community interactions).

*## OONI Probe Desktop*

We made some improvement to the description of the Experimental card in
OONI Probe Desktop (https://github.com/ooni/probe-desktop/pull/294) and we
worked towards ensuring that the autorun directory is deleted when a hard
reset is performed (https://github.com/ooni/probe-desktop/pull/295).

*## OONI Probe Web*

We added support to OONI Probe Web for using the check-in API to retrieve
information about the probe IP address country code and ASN (
https://github.com/ooni/probe-web/commit/bc680323df0c7404b14651acea11a281c7353f96).
We also worked on improving the UI of OONI Probe Web.

We circulated an internal testing beta of OONI Probe Web (
https://ooni.github.io/probe-web/) to collect community feedback that can
help inform further development and improvements.



*## Expanding censorship measurement methodologies### Website measurement*

After some initial experimentation in April 2022, we sketched out a plan
for integrating some methodologies from the “Understanding the Practices of
Global Censorship through
Accurate, End-to-End Measurements” (
https://shhaos.github.io/papers/sigmetrics22.pdf) study into OONI Probe (
https://github.com/ooni/probe/issues/2086).

We started implementing the plan for merging the Websteps prototype (
https://github.com/bassosimone/websteps-illustrated/) into OONI Probe. To
this end, we merged several improvements for netxlite, the ooniprobe
package that provides basic networking functionality (
https://github.com/ooni/probe/issues/2096).

We also drafted and implemented a plan for observing duplicate DNS replies (
https://github.com/ooni/probe/issues/2099). This plan differs from the
Websteps implementation because community feedback (in particular from
Jigsaw’s Vinicius Fortuna and Sarah Laplante) convinced us that the
implementation of this functionality that we had sketched out in the
Websteps prototype could be further simplified and streamlined.

*### New experiments*

We implemented several new exploratory experiments:
* dnsping:
https://github.com/ooni/spec/blob/master/nettests/ts-035-dnsping.md
* simplequicping:
https://github.com/ooni/spec/blob/master/nettests/ts-034-simplequicping.md
* tcpping:
https://github.com/ooni/spec/blob/master/nettests/ts-032-tcpping.md
* tlsping:
https://github.com/ooni/spec/blob/master/nettests/ts-033-tlsping.md

These are “exploratory” in the sense that we expect users to manually run
them from the command line when they are actively researching censorship.
We introduced these four new experiments based on community feedback and
requests.

*### Vanilla Tor experiment*

We implemented the Vanilla Tor experiment (
https://github.com/ooni/probe-cli/pull/718), which used to be part of the
legacy version of OONI Probe. We reintroduced this experiment following
discussions with Tor developers, who mentioned that it would be quite
useful to them.

*### Data quality improvements*

We investigated and fixed a rather surprising and complex data quality
issue. It turns out that the Android implementation of `getaddrinfo`
behaves differently from any other existing `getaddrinfo` implementation
and only returns one of two return codes: 0 in case of success, and
`EAI_NODATA` in case any other possible error occurs, including cases where
a domain name does not exists, cases where the server is temporarily
unavailable, cases where the server is misconfigured, and cases of
censorship.

In practical terms, this means that the result of `getaddrinfo` on Android
is unreliable and we need to improve our heuristics on the backend and our
algorithms on the probe to work around this. The pull request where we
fixed this bug refers to our investigation, as well as to our follow up
plans to further improve DNS measurements on Android (
https://github.com/ooni/probe-cli/pull/764). The reference issue describing
this problem (https://github.com/ooni/probe/issues/2029) is still open
because we have not included the fix into the 3.15.x release cycle (which
was being finalized while we were still investigating the root cause of
this surprising bug).

*## OONI Explorer*

We fixed a couple of OONI Explorer bugs. Specifically, we added support for
searching URLs with a longer TLD (https://github.com/ooni/explorer/pull/757)
and we updated the copyright year in the footer (
https://github.com/ooni/explorer/pull/756).

*## Building a web platform for updating test lists*

Based on community feedback on the private beta version of our new test
list web platform (https://test-lists.ooni.org/), we added backend support (
https://github.com/ooni/api/pull/286) to enable users to make changes to
open pull requests (once a pull request has been opened through a test list
web platform submission).

In anticipation of a potentially increased volume of test list pull
requests following the upcoming public launch of the test list web
platform, we (along with the Citizen Lab) invited Censored Planet and
Netalitica team members to get added to the Citizen Lab test list
repository as reviewers of pull requests.

*## OONI backend*

*### Migration to the Clickhouse database*

We continued to work on the ongoing migration to the new OLAP database. In
May 2022, we migrated 12 API entry points to the new backend host.

*### Reprocessing legacy measurements*

We continued to work on the reprocessing of measurements collected before
2021 to improve data quality and ensure all measurements are reachable from
OONI Explorer.

*### OONI API*

We reviewed and deployed a change to support the detection of the ASN and
country code of probes based on their IP address on the API side. This will
allow the removal of bulky GeoIP database files from the probes. We also
improved the account_metadata API call to display login status on web UIs (
https://github.com/ooni/api/pull/292).

*### Other operations*

We added blackbox exporter-based testing and support for the alarming of
various API entry points. We finished moving the test helper rotation
script to the new backend host (https://github.com/ooni/pipeline/pull/386).

*## Test list updates*

In May 2022, we contributed a few updates to the Citizen Lab test lists. In
particular, we updated the US test list (
https://github.com/citizenlab/test-lists/pull/984) and we removed dead URLs
from the Global test list (https://github.com/citizenlab/test-lists/pull/995).




*## Notable community use of OONI Probe and OONI data### Sinar Project
report*

Malaysia’s Sinar Project published a report on the blocking of a NFT
fundraising campaign website (of graphics designer and political activist,
Fahmi Reza):
https://sinarproject.org/digital-rights/measuring-and-detecting-network-interference/nft-fundraising-campaign-website-of-malaysian-artist-fahmi-reza-blocked

Their report makes use of OONI Probe and OONI data.



*## Community activities### OONI workshop for civil society groups in Latin
America*

On 19th May 2022, OONI’s Maria facilitated a 2-hour OONI workshop for civil
society groups in Latin America. This workshop (organized by Derechos
Digitales) was facilitated in collaboration with Venezuela Inteligente.

*### OONI presentation at Tactical Tech*

On 24th May 2022, OONI’s Maria presented OONI at Tactical Tech (
https://tacticaltech.org/) in Berlin. This presentation also involved a
live demo of using the Measurement Aggregation Toolkit (MAT) to investigate
various forms of internet censorship around the world.

*### OONI presentation for journalists*

On 25th May 2022, OONI’s Maria presented OONI’s tools, methods, research,
and dataset to a group of journalists from Russia and Belarus.

*### OONI Community Meeting*

On 31st May 2022, we hosted the monthly OONI Community Meeting on our Slack
channel (https://slack.ooni.org/), during which we discussed the following
topics:

1) Job Opening: OONI Community Coordinator (
https://ooni.org/post/2022-job-opening-ooni-community-coordinator/)

2) Detecting regional censorship, which is increasingly becoming an urgent
issue (https://github.com/ooni/probe/issues/1581#issuecomment-1125221383)

In particular, we discussed the following questions with the community:

* Should there be an option to also include the region or should we include
the region by default?
* Should we invest some legal research time to figure out if we could use
MaxMind?
* Should we try to have location permission in the app and use that? Should
we have two apps, one with location and the other without?
* Should we allow users to manually annotate/correct the geolocation
results if they are inaccurate?

3) Copyright law for enforcing internet censorship

*## Userbase*

In May 2022, 41,882,898 OONI Probe measurements were collected from 2,905
AS networks in 171 countries around the world.

This information can also be found through our measurement stats on OONI
Explorer (see chart on “monthly coverage worldwide”):
https://explorer.ooni.org/

~ OONI team.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.torproject.org/pipermail/ooni-talk/attachments/20220715/9474d6ba/attachment-0001.htm>