[tor-project] OONI Monthly Report: November 2017

Thu Dec 7 13:25:11 UTC 2017

Hello,

The OONI team made steady progress in November 2017.

We reported on recent censorship events in Pakistan, as well as on cases
of DNS misconfiguration. We also updated various test lists in
collaboration with community members, and we presented OONI at the Free
Society Conference and Nordic Summit (FSCONS). We are currently working
on improving the design and usability of our mobile apps, and we
completed the first draft of the onboarding process for the OONI Probe
Android app. We also made an alpha release of node.js bindings for
measurement-kit, made progress on implementing dynamic serving of test
lists via the orchestration backend, and we started writing an
architecture and threat model document for probe orchestration.

# Published research report on recent censorship events in Pakistan

In collaboration with Bytes for All we published a research report
examining recent censorship events in Pakistan. 

This report is available
here: https://ooni.torproject.org/post/how-pakistan-blocked-social-media/

Through the analysis of OONI network measurements collected from
Pakistan, we confirmed the blocking of 14 news websites and various
social media sites and instant messaging apps during Islamist protests.

Our report received the following press
coverage: https://www.thenews.com.pk/print/250865-internet-body-issues-report-on-news-social-media-blackout

# Published research report on cases of DNS misconfiguration* *

As part of our analysis of OONI measurements and investigation into DNS
failures, we uncovered various cases of DNS misconfiguration.

We reported these cases in the following
post: https://ooni.torproject.org/post/not-quite-network-censorship/

# Progress on mobile app redesign

We are currently working on improving the design and usability of our
mobile apps. During November we completed the first draft of the
re-designed onboarding process for the OONI Probe Android app, and we
worked on the Interaction Design elements.

# Updated test lists

We collaborated with community members on updating various test lists.
During November, we updated the following test lists:

1. Sierra Leone: https://github.com/citizenlab/test-lists/pull/262

2. Pakistan: https://github.com/citizenlab/test-lists/pull/259

3.
Nigeria: https://github.com/citizenlab/test-lists/pull/255 & https://github.com/citizenlab/test-lists/pull/257

4. India: https://github.com/citizenlab/test-lists/pull/261

5. Algeria: https://github.com/citizenlab/test-lists/pull/249

# Architecture & Threat model for Probe Orchestration

We started writing an architecture and threat model document for the
Probe Orchestration system and we also started collaborating with Cure53
to get it audited by them.

# OONI data publishing

Amazon accepted us into their Open Data program
(https://aws.amazon.com/government-education/open-data/)
<https://aws.amazon.com/government-education/open-data/%29> and as a
result all the public OONI data is available for free on s3.

You can access all our public JSON reports inside of this s3 bucket:

s3://ooni-data/autoclaved/jsonl/

Example: https://s3.amazonaws.com/ooni-data/autoclaved/jsonl/2017-11-06/20171031T181513Z-BE-AS6848-http_header_field_manipulation-20171105T011219Z_AS6848_pCOb02HpTZRd3PQ6lfwfgHJLk1lSnxiux2oNvNjITZTWy1a2uJ-0.2.0-probe.json

(If you were previously using s3 hosted data, the bucket name changed
from ooni-public to ooni-data)

Moreover we also offer, for batch import, lz4 compressed daily dumps of
all measurements.

These are available here:

s3://ooni-data/autoclaved/jsonl.tar.lz4/

Example: https://s3.amazonaws.com/ooni-data/autoclaved/jsonl.tar.lz4/2017-12-01/dash.0.tar.lz4

To get a listing and metadata of all the files in a given bucket, you
may look for the index.json.gz file

Example: https://s3.amazonaws.com/ooni-data/autoclaved/jsonl.tar.lz4/2017-12-01/index.json.gz

If you are using the AWS Command Line Interface
(https://docs.aws.amazon.com/cli/latest/reference/)
<https://docs.aws.amazon.com/cli/latest/reference/%29> and have a AWS
access key configured, be sure to pass in the --no-sign-request option.

Example:

$ aws s3 --no-sign-request ls s3://ooni-data/autoclaved/jsonl/2017-12-04/

We will be releasing a short blog post explaining how to use this data,
once Amazon updates their Open Data page to list us as a project.

# Test lists orchestration

Progress was made on implementing dynamic serving of tests lists via the
orchestration backend. See: https://github.com/TheTorProject/proteus/pull/29

# Node.js bindings for Measurement-kit

We made an alpha release of node.js bindings for measurement-kit:

https://github.com/measurement-kit/measurement-kit-node

It's not yet ready for usage by the general public, but we have been
using this as a foundation for the development of our new OONI Probe
desktop apps.

# Community activities

We presented OONI at the Free Society Conference and Nordic Summit
(FSCONS) in Norway on 4th November 2017. Information about our
presentation is available
here: https://frab.fscons.org/en/fscons17/public/events/309.

We also attended the OTF Summit in Valencia, and hosted our monthly
community meeting on 28th November 2017. As part of the community
meeting, we discussed strategies for monitoring emergent censorship
events and rapidly responding to them (in light of recent censorship
events in Pakistan, for example). 

# Research citing OONI data

During November OONI data was cited in the following research:

* Citizen Lab Submission to the United Nations Special Rapporteur on
violence against women, its causes and
consequences: https://citizenlab.ca/wp-content/uploads/2017/11/Final-UNSRVAG-CitizenLab.pdf

* Internet Censorship Capabilities in Cyprus: An investigation of online
gambling
blocklisting: https://www.researchgate.net/publication/319482512_Internet_Censorship_Capabilities_in_Cyprus_An_investigation_of_online_gambling_blocklisting

# Userbase

In November 2017 OONI Probe was run 184,800 times from 3,544 different
vantage points across 202 countries around the world.

This information can also be found through our stats
here: https://api.ooni.io/stats

~ The OONI team.

-- 
Maria Xynou
Research and Partnerships Coordinator
Open Observatory of Network Interference (OONI)
https://ooni.torproject.org/
PGP Key Fingerprint: 2DC8 AFB6 CA11 B552 1081 FBDE 2131 B3BE 70CA 417E

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: OpenPGP digital signature
URL: <http://lists.torproject.org/pipermail/tor-project/attachments/20171207/c26677c0/attachment.sig>