[tor-project] OONI Monthly Report: February 2023
Maria Xynou
maria at openobservatory.org
Wed Mar 29 02:57:20 UTC 2023
Hello,
This email shares OONI's monthly report for February 2023.
*# OONI Monthly Report: February 2023*
Throughout February 2023, the OONI team worked on the following sprints:
* Sprint 84 (1st-12th February 2023)
* Sprint 85 (13th-28th February 2023)
Our work can be tracked through the various OONI GitHub repositories:
https://github.com/ooni
Highlights are shared in this report below.
*## Published research report on internet censorship in Russia*
On 24th February 2023 (1 year since the start of the war in Ukraine), we
co-published a new research report, in collaboration with Roskomsvoboda,
which documents how Internet censorship in Russia changed over the last
year based on OONI data analysis.
Read our report in:
* English: https://ooni.org/post/2023-russia-a-year-after-the-conflict/
* Russian: https://ooni.org/ru/post/2023-russia-a-year-after-the-conflict/
You can also read Roskomsvoboda's publication about the report here:
https://roskomsvoboda.org/post/rks-ooni-odin-god/
Overall, OONI data analysis shows the blocking of 494 domains in Russia.
Many more websites are also blocked, beyond those tested. The blocked
domains fall under 28 categories, suggesting pervasive levels of internet
censorship in Russia.
New blocks that emerged in Russia over the last year include:
* Blocking of Russian & international human rights websites (e.g. Human
Rights Watch & Amnesty International)
* Blocking of investigative journalism & news media sites
* Blocking of Instagram, SoundCloud and Patreon
Access to the Tor Project's website was temporarily unblocked in Russia
between 15th to 28th July 2022. Several discontinued sites have been
unblocked over the last year (possibly as part of a "blocklist cleanup").
We confirmed the blocking of 48 domains (including a Let's Encrypt domain)
based on OONI data, which do not appear to be included in Russia's official
blocking registry. OONI data also suggests that access to a major CDN was
blocked in the past year.
*## Published research report on internet censorship in Azerbaijan*
On 28th February 2023, we co-published a new research report on internet
censorship in Azerbaijan, in collaboration with our partner, Azerbaijan
Internet Watch. As part of this research, we analyzed OONI data collected
from Azerbaijan over the last year.
The report was published by:
* OONI: https://ooni.org/post/2023-azerbaijan-internet-censorship/
* Azerbaijan Internet Watch:
https://www.az-netwatch.org/news/ooni-measurements-show-ongoing-internet-censorship-in-azerbaijan/
As part of our analysis of OONI measurements collected from Azerbaijan
between January 2022 to February 2023, we found:
** Blocking of news media websites. *Azerbaijan continues to block access
to several independent news media websites. OONI data also suggests that
some ISPs in Azerbaijan may have started blocking access to the Guardian on
25th December 2022.
** Azerbaijan and Russia block each other’s news media. *In early June
2022, Azerbaijan started blocking access to Russia’s state-run RIA Novosti
media website. Since (at least) 7th June 2022, Russian ISPs have been
blocking access to Azerbaijani news media websites (haqqin.az, minval.az,
oxu.az, ru.oxu.az, ru.baku.ws). These blocks remain ongoing.
** Temporary blocking of TikTok amid border clashes with Armenia. *During
the September 2022 border clashes, both Azerbaijan and Armenia blocked
access to TikTok. While the TikTok block was lifted in Armenia by 21st
September 2022 (only lasting about a week), the TikTok block remained in
place in Azerbaijan until November 2022 (lasting about 2 months).
** Blocking of circumvention tool sites.* Azerbaijan continues to block
access to numerous circumvention tool websites, potentially limiting the
ability to circumvent internet censorship in Azerbaijan. However, most OONI
measurements suggest that tested circumvention tools (Tor and Psiphon)
appear to be reachable.
** Variance of censorship across networks. *While most ISPs in Azerbaijan
appear to adopt similar censorship techniques (as we continued to observe
connection timeouts in most anomalous measurements across ASNs), different
ISPs block access to different websites over time.
*## Published report on social media blocks in Ethiopia*
On 15th February 2023, we published a report on the blocking of social
media platforms in Ethiopia.
OONI data from Ethiopia showed the ongoing blocking of:
* Facebook and Telegram (since 9th February 2023)
* YouTube (since 10th February 2023)
Read the report here:
https://ooni.org/post/2023-ethiopia-blocks-social-media/
*## Published report on DW and Wikipedia blocking in Pakistan*
On 15th February 2023, we published a report on the blocking of Wikipedia
and Deutsche Welle (DW) in Pakistan.
OONI data from Pakistan showed:
* Access to Wikipedia restricted between 1st-6th February 2023 (48-hour
degradation, followed by block)
* Ongoing blocking of Deutsche Welle (DW) since (at least) 16th January 2023
Read the report here:
https://ooni.org/post/2023-pakistan-blocks-wikipedia-and-dw/
*## Published report on Twitter throttling in Turkey*
On 15th February 2023, we published a report on the throttling and DNS
blocking of Twitter in Turkey in the aftermath of a deadly earthquake.
On 8th February 2023, OONI data from Turkey showed that access to Twitter
was throttled on at least 4 networks, and blocked by means of DNS
interference on other networks.
Read the report here:
https://ooni.org/post/2023-turkey-throttling-blocking-twitter/
*## OONI reports for the ISOC Pulse project on Internet shutdowns*
As a data partner for ISOC’s Pulse Shutdown project (
https://pulse.internetsociety.org/shutdowns), we have been working on
creating “OONI reports” (with relevant OONI data, interpretation, and
charts) for all of the “service blocking” entries listed on the Pulse
shutdown timeline (ranging from March 2018 to February 2023) over the last
year.
In February 2023, we completed this work, having written the “Data and
analysis” section for all “service blocking” entries of the ISOC Pulse
Shutdown timeline (where relevant OONI data exists). As part of this work,
we found that OONI data is available for almost all blocking events
reported worldwide over the last 5 years (at least based on those listed on
the ISOC Pulse shutdown timeline).
In total, we wrote 49 OONI reports for the ISOC Pulse shutdown timeline.
These reports will be included in the “Data and analysis” section of
“service blocking” entries.
*## OONI Probe Mobile*
In February 2023, we continued to improve upon the OONI Probe mobile app.
Specifically, we:
* Prepared the next OONI Probe Mobile release (3.8.0) for internal testing;
* Fixed a bug ensuring that the network name is displayed along with the
ASN (https://github.com/ooni/probe/issues/2371);
* Worked towards addressing a bug affecting automated testing on OONI Probe
iOS (https://github.com/ooni/probe/issues/2259);
* Worked towards enabling the transfer of the OONI Probe Android app to a
SD card (https://github.com/ooni/probe/issues/2019);
* Worked towards ensuring that the OONI Probe iOS app does not crash when
OONI backend services are down (https://github.com/ooni/probe/issues/2201).
*## OONI Run*
OONI Run (https://run.ooni.io/) can be translated via the Transifex (
https://explore.transifex.com/otf/ooni-run/) platform.
In February 2023, thanks to the Localization Lab community, OONI Run was
translated into 6 languages: Chinese, German, Thai, Russian, Spanish, and
Turkish.
*## OONI Probe CLI*
In February 2023, we released OONI Probe CLI 3.17.0:
https://github.com/ooni/probe-cli/releases/tag/v3.17.0
Notably, OONI Probe CLI 3.17.0 includes an updated version (v0.5.20) of the
Web Connectivity experiment, a TLS middlebox experiment, backend proxy
support for Tor Snowflake, OONI Run improvements, and many other
improvements (https://github.com/ooni/probe-cli/releases/tag/v3.17.0).
This new release will be the first one to support providing richer input to
experiments. We now support dynamically instructing the probes to enable
experimental Web Connectivity functionality through the OONI backend.
As part of this release cycle, we investigated two issues related to
running tor on Android devices. The first issue (
https://gitlab.torproject.org/tpo/core/tor/-/issues/40747,
https://github.com/ooni/probe/issues/2405) deals with the OONI Probe
Android app receiving a SIGABRT signal because tor closes a file descriptor
twice, thus triggering Android’s fdsan (
https://android.googlesource.com/platform/bionic/+/master/docs/fdsan.md).
We patched the problem as part of the OONI Probe CLI 3.17.0 release. The
second issue deals with a mysterious SIGABRT crash that we are still
investigating.
While testing OONI Probe CLI 3.17.0, we also noticed and investigated
excessive CPU usage in the Web Connectivity test helper (
https://github.com/ooni/probe/issues/2413). We modified the codebase to add
support for profiling, started fixing the most obvious issues, and
increased the number of test helper hosts to spread the load among them.
As part of testing, we also triaged the following data quality issues:
https://github.com/ooni/probe/issues/2410
https://github.com/ooni/probe/issues/2411
https://github.com/ooni/probe/issues/1925#issuecomment-1429373491
https://github.com/ooni/probe/issues/2412
https://github.com/ooni/probe/issues/2420
https://github.com/ooni/probe/issues/2421
We fixed some of these data quality issues, as documented below:
https://github.com/ooni/probe-cli/pull/1110
https://github.com/ooni/probe-cli/pull/1111
https://github.com/ooni/probe/issues/2293
Where feasible, we backported to the OONI Probe CLI 3.17.0 branch.
After M-Lab deprecated their locatev1 API, we also migrated the DASH
experiment to use the locatev2 API and included this patch into the 3.17.0
branch (https://github.com/ooni/probe/issues/2398).
We also started working on exposing the OONI engine API, which is part of
the probe-cli repository, as a dynamic library (
https://github.com/ooni/probe/issues/2414).
*## Expanding OONI’s testing model to support richer testing input*
The OONI Probe CLI 3.17.0 release (
https://github.com/ooni/probe-cli/releases/tag/v3.17.0) starts adding
support for richer testing input (
https://github.com/ooni/ooni.org/issues/1291). We also continued to
improve such support in the main development branch of OONI Probe CLI and
OONI API. Specifically, we modified the API to directly return test helper
information inside the check-in API response (
https://github.com/ooni/probe/issues/2392).
We also started working on a design document regarding the richer testing
input functionality and we began working on the related PoC (
https://github.com/ooni/probe-cli/pull/1075).
*## Creating a throttling measurement methodology*
As part of our plans to create a methodology for measuring throttling (
https://github.com/ooni/ooni.org/issues/1296), we started sketching out a
possible integration testing strategy.
At the end of this reporting period, we wrote a diff for probe-cli (
https://gist.github.com/bassosimone/3ce52a37fc7394f7cce82391685c5477) which
works as follows:
* We hijack the core networking code to use a GVisor-based (
https://gvisor.dev/) TCP/IP stack in userspace;
* We connect this stack, representing the OONI Probe client with other
GVisor stacks representing servers;
* We add latency and throttling on those virtual channels when the network
traffic meets specified conditions (e.g., depending on the SNI used during
the TLS handshake).
Writing good integration tests for this upcoming functionality is crucial
to ensure that we do not break the Web Connectivity experiment while
applying the necessary changes to add support for collecting network
performance metrics (as planned:
https://github.com/ooni/ooni.org/issues/1297).
*## OONI Explorer*
We continued to work with a designer on the new OONI Explorer domain and
network centric pages (https://github.com/ooni/explorer/pull/830). We
updated the copy for the user feedback reporting system and we worked on
end-to-end tests. We made improvements to the third-party data integration,
notably adding support for the caching of proxied Cloudflare requests and
other UI changes (https://github.com/ooni/explorer/pull/831). We also
released a new MAT filter that allows showing data for different time
intervals: hourly, daily, weekly and monthly.
*## OONI backend*
Through February 2023, we continued to work on a series of backend
improvements.
Specifically, we:
* Implemented datacenter filtering in rotation (
https://github.com/ooni/pipeline/pull/408);
* Scaled up our infrastructure to 4 test helpers (
https://github.com/ooni/api/pull/320) and correlated the test runtime with
the deployment of the new test helpers to measure their effectiveness;
* Added more networks to the Web Connectivity v0.5 feature flag (
https://github.com/ooni/api/pull/318);
* Fixed a bug affecting authentication in the Test Lists Editor (
https://github.com/ooni/test-lists-ui/issues/71);
* Fixed the pipeline CI permissions (
https://github.com/ooni/pipeline/pull/409);
* Extracted more values from measurements including software and test
version, engine details, CPU architecture and test run time (
https://github.com/ooni/pipeline/pull/411,
https://github.com/ooni/pipeline/pull/410);
* Implemented an initial database backup tool.
*## Automating censorship detection and characterization based on OONI
measurements*
We made stable progress on the OONI data analysis tool (
https://github.com/ooni/data). Specifically, we:
* Made several improvements to the tooling that can be used for creating
plots that allow us to assess how well the analysis performs (
https://github.com/ooni/data/pull/25);
* Added support for mapping several “unknown_failures” that are a result of
software bugs, ensuring that the data is more clean and can be better
analyzed (
https://github.com/ooni/data/pull/25/commits/dc6517e0148be6c571a9e0768b50b5c3bccfee48
):
* Started exploring the possibility of performing the analysis directly
inside of the database engine (as opposed to doing in python code). This
resulted in a 10x performance increase, allowing us to analyze one day's
worth of OONI data in just over 5 minutes (compared to 2 hours of the
python implementation).
We also ingested new blocking fingerprints (
https://github.com/ooni/pipeline/pull/407) into the fastpath data
processing pipeline, thereby automatically detecting and confirming more
cases of website blocking around the world.
To evaluate how effectively OONI Probe measures and detects cases of
internet censorship, we compared measurements from Indonesia (that show
blocking) with Indonesia’s (“DNS-Trust”) blocking registry (
https://github.com/alsyundawy/dnstrust-apjii). We found that OONI
measurements that show DNS-based censorship are consistent with Indonesia’s
blocklist.
*## Creating a Social Media Censorship Alert System*
We continued to make progress on the initial event detector developed for
the new Social Media Censorship Alert System (
https://github.com/ooni/backend/issues/629). Specifically, we ran
experiments using Pandas, added more tests, and generated PNG/SVGs as part
of the testing.
*## Creating a new OONI Outreach Kit*
Throughout February 2023, we coordinated with Ura Design (
https://ura.design/), who worked on designing the new OONI Outreach Kit
materials. We are aiming to publish the Outreach Kit in March 2023.
*## Coordination of translation of OONI documents*
Throughout February 2023, we coordinated with translators with regards to
the translation of several OONI documents into 5 languages: Arabic, Farsi,
Russia, Swahili, and Spanish. We are aiming to publish the translated
documents in March 2023.
*## Test list updates*
We coordinated with our partner, TEDIC (https://ooni.org/partners/tedic/),
who provided extensive updates for the test list of Paraguay:
https://github.com/citizenlab/test-lists/pull/1224
The test list for Cambodia was also updated extensively by API Cambodia
(iMAP partner): https://github.com/citizenlab/test-lists/pull/1214
We reviewed and merged many other test list pull requests contributed by
community members:
https://github.com/citizenlab/test-lists/pulls?q=is%3Apr+is%3Aclosed
*## Rapid response### Wikipedia blocking in Pakistan*
Access to Wikipedia was temporarily restricted in Pakistan between 1st to
6th February 2023.
We rapidly responded by sharing relevant OONI data collected from Pakistan
(and encouraging further testing) on Twitter:
https://twitter.com/OpenObservatory/status/1620845976645963778 and
https://twitter.com/OpenObservatory/status/1621449233852252160
We subsequently published a report:
https://ooni.org/post/2023-pakistan-blocks-wikipedia-and-dw/
*### Social media blocks in Iraq*
On 6th February 2023, community members in Iraq reported to the #KeepItOn
mailing list that access to social media platforms (WhatsApp, Facebook,
Twitter, Instagram) would be banned during the country’s final exams,
starting on 5th February 2023 and ending on 13th February 2023 (from 4 am
to 12 pm local time). The decision was made by the PMO based on the
ministry of education recommendation.
We shared relevant OONI data with the #KeepItOn campaign on these blocks.
*### Twitter throttling in Turkey*
On 8th February 2023, in the aftermath of the earthquakes, Turkey throttled
access to Twitter.
Real-time OONI data collected from Turkey showed the block:
https://explorer.ooni.org/chart/mat?probe_cc=TR&test_name=web_connectivity&domain=twitter.com&since=2023-01-09&until=2023-02-09&axis_x=measurement_start_day
We rapidly responded by analyzing relevant OONI data collected from Turkey
and sharing our analysis through a Twitter thread:
https://twitter.com/OpenObservatory/status/1623398643595182083
OONI data showed that access to Twitter was throttled on at least 4
networks, and blocked by means of DNS on other networks.
We subsequently published a report:
https://ooni.org/post/2023-turkey-throttling-blocking-twitter/
*### Social media blocks in Ethiopia*
On 9th February 2023, amid church split tensions and calls for
anti-government protests, access to social media platforms was reportedly
blocked in Ethiopia.
We rapidly responded by sharing relevant OONI data collected from Ethiopia
(and encouraging further testing) on Twitter:
https://twitter.com/OpenObservatory/status/1624045402679873540 and
https://twitter.com/OpenObservatory/status/1624761465453379584
We subsequently published a report:
https://ooni.org/post/2023-ethiopia-blocks-social-media/
*## Community use of OONI data### Paper on internet censorship in Russia*
Censored Planet (in collaboration with other researchers) published a paper
(“Network Responses to Russia’s Invasion of Ukraine in 2022: A Cautionary
Tale for Internet Freedom”) which analyzes OONI data (along with Censored
Planet data) to examine internet censorship in Russia following the
February 2022 invasion of Ukraine.
Their paper (which was accepted to appear in USENIX Security 2023) is
available here:
https://censoredplanet.org/assets/russia-ukraine-invasion.pdf
*### Paper on censorship data analysis*
Censored Planet and Jigsaw published a paper (“Advancing the Art of
Censorship Data Analysis”) which discusses the challenges involved in
analyzing censorship measurement data. This paper shares examples from OONI
data and other public censorship datasets (such as Censored Planet).
Their paper (which was published by FOCI) is available here:
https://www.petsymposium.org/foci/2023/foci-2023-0003.pdf
*### Data Journalism guide for measuring internet shutdowns*
Journalist Sabrina Faramarzi published an article (“Kill switch: reporting
on and during internet shutdowns”) which aims to serve as a guide for data
journalists interested in measuring internet shutdowns. This article
features an interview with OONI’s Maria, where she discusses why
journalists should use OONI data to investigate internet censorship around
the world.
The article is available here:
https://datajournalism.com/read/longreads/internet-shutdowns-data-reporting
*## Community activities### OONI workshop for civil society in Azerbaijan*
On 1st February 2023, OONI’s Elizaveta facilitated an OONI workshop for
civil society groups in Azerbaijan. This workshop was hosted in
collaboration with our partner, Azerbaijan Internet Watch (
https://ooni.org/partners/azerbaijan-internet-watch/). The goal of the
workshop was to introduce civil society groups in Azerbaijan to OONI tools
to help boost OONI measurement coverage in Azerbaijan.
*### FOSDEM 2023*
On 4th and 5th February 2023, OONI’s Federico attended FOSDEM (
https://fosdem.org/2023/) in Brussels.
*### OONI workshop for human rights defenders in Nigeria*
On 16th February 2023, OONI’s Elizaveta facilitated an OONI workshop for
human rights defenders in Nigeria. This workshop was hosted in
collaboration with our partner, Paradigm Initiative (
https://ooni.org/partners/paradigm-initiative/). The goal of the workshop
was to introduce participants to OONI tools and data, encouraging
censorship measurement leading up to and during Nigeria’s 2023 elections
(which may trigger new censorship events).
*### OONI Run demo for the Localization Lab community*
On 22nd February 2023, OONI’s Elizaveta provided an OONI Run (
https://run.ooni.io/) demo for the Localization Lab community to encourage
the translation of the platform.
*### Live OONI session for the OPTIMA community in Senegal*
On 23rd February 2023, OONI’s Elizaveta briefly presented OONI and
addressed questions as part of Internews’ live OPTIMA session for community
members in Senegal.
*### OONI Community Meeting*
On 28th February 2023, we hosted the monthly OONI Community Meeting on our
Slack channel (https://slack.ooni.org/), during which we discussed the
following topics:
1. Updates from the OONI team.
2. Upcoming conferences and opportunities for censorship measurement
sessions.
3. Community feedback on early beta testing of OONI tools.
*## Measurement coverage*
In February 2023, 57,841,940 OONI Probe measurements were collected from
2,844 AS networks in 164 countries around the world.
This information can also be found through our measurement stats on OONI
Explorer (see chart on “monthly coverage worldwide”):
https://explorer.ooni.org/
~ OONI team.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.torproject.org/pipermail/tor-project/attachments/20230329/b9b931e2/attachment-0001.htm>
More information about the tor-project
mailing list