[tor-project] OONI Monthly Report: October 2018

Fri Nov 16 13:30:10 UTC 2018

Hello Tor world,

In October 2018, the OONI team co-published a new research report, in
collaboration with DefendDefenders, examining Uganda's social media tax
through the analysis of network measurements. We wrote new OONI Probe
nettest specifications, improved our heuristics for semi-automated
blockpage and blockserver detection, completed a list of improvements to
the monitoring of OONI services, and released a private internal beta of
the revamped OONI Probe mobile pps. We also facilitated an OONI workshop
at MozFest and participated at the OTF Summit. 

## Report on Uganda's Social Media Tax

In collaboration with our Ugandan partners, DefendDefenders, we
co-published a new research report tited: "Uganda's Social Media Tax
through the lens of network measurements".

Our joint report is available here:
https://ooni.io/post/uganda-social-media-tax/

As part of our latest study, we examine the blocking of social media &
circumvention tools across ISPs in Uganda through the analysis of OONI
network measurements.

Over the last months, we have published the following 5 research reports
(in collaboration with our partners):

1. The State of Internet Censorship in Venezuela:
https://ooni.torproject.org/post/venezuela-internet-censorship/

2. The State of Internet Censorship in Egypt:
https://ooni.torproject.org/post/egypt-internet-censorship/

3. South Sudan: Measuring internet censorship in the world's youngest
nation: https://ooni.torproject.org/post/south-sudan-censorship/

4. Mali: Social media disruptions amid 2018 presidential election?
https://ooni.torproject.org/post/mali-disruptions-amid-2018-election/

5. Uganda's Social Media Tax through the lens of network measurements:
https://ooni.io/post/uganda-social-media-tax/

More research studies are available on our website:
https://ooni.torproject.org/post/

## Revamping OONI Probe mobile apps

Both Android and iOS apps have been implemented based on improvements to
Measurement Kit and the finalized design and copy. We released a private
internal beta to further test and polish the apps.

Relevant pull requests include:

* https://github.com/ooni/probe-ios/pull/187

* https://github.com/ooni/probe-ios/pull/186

* https://github.com/ooni/probe-ios/pull/185

* https://github.com/ooni/probe-ios/pull/184

* https://github.com/ooni/probe-android/pull/159

We are now preparing for a public beta release of the apps.  

## Research and write specifications for new OONI Probe nettests

We wrote specifications for a series of techniques that we plan on using
in next generation OONI Probe nettests. These techniques will allow us
to collect more rich network measurement data, further reduce the number
of false positives in measurements and confirm, with a higher level of
confidence, incidents of internet censorship.

Our new nettest specifications are available via the following pull
request: https://github.com/ooni/spec/pull/118

## Implementation of semi-automated blockpage and blockserver detection

We have added SimHash support (https://en.wikipedia.org/wiki/SimHash) to
enable the semi-automated blockpage and blockserver detection.

When several unrelated URLs present a similar page, this usually means
that the page is either a common web-server error page or a blockpage.
Human intelligence is still required to distinguish anti-DDoS service
provider captcha pages from generic "403 Forbidden" web-server errors
and a "genuine" blockpage.

We therefore added support for SimHash to use it as the first
approximation of webpage similarity, to avoid pair-wise comparision of
hundreds of millions of webpages stored in the OONI dataset. See:
https://github.com/ooni/pipeline/pull/124

We also added support for the partial re-processing of data, to avoid
processing the whole dataset when the list of known blockpages changes.
See:
https://github.com/ooni/pipeline/commit/a86e4fe42dec869b02618ba2c3e282812052cab9

## Improvements to the monitoring of OONI services

To detect issues with our infrastructure earlier and to therefore
minimise downtime and be more responsive to incidents, we have done
several improvements to monitoring.

We have completed all activities the aimed at improving the monitoring
of OONI services, listed in the following issue:
https://github.com/ooni/sysadmin/issues/226

## Community activities

### Community meeting

We held a community meeting on Slack
(https://slack.openobservatory.org/) on Wednesday, 24th October 2018. As
part of the meeting, the OONI team provided updates on what we worked on
during the past month. We also addressed community questions pertaining
to running OONI Probe on Linux and Raspberry Pis.

### OONI Probe workshop at MozFest

OONI's Elio traveled to London to facilitate an OONI Probe workshop at
Mozilla Festival
(https://twitter.com/mozTechSpeakers/status/1056851429766414336). This
provided a great opportunity to engage more community members with
OONI's work.

### OTF Summit

OONI's Maria traveled to Taiwan to participate at the OTF Summit (29th &
30th October) and the Open Internet Day (31st October 2018). This
provided an opportunity to connect with community members, learn more
about their projects, and brainstorm on potential avenues for
collaboration.

## Userbase

In October 2018, OONI Probe was run 391,014 times from 5,062 different
vantage points in 214 countries around the world.

This information can also be found through our stats:
https://api.ooni.io/stats

~ The OONI team.

-- 
Maria Xynou
Research & Partnerships Director
Open Observatory of Network Interference (OONI)
https://ooni.torproject.org/
PGP Key Fingerprint: 2DC8 AFB6 CA11 B552 1081 FBDE 2131 B3BE 70CA 417E

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: OpenPGP digital signature
URL: <http://lists.torproject.org/pipermail/tor-project/attachments/20181116/33322586/attachment.sig>