[tor-project] OONI Monthly Report: June 2017

[Arturo Filastò]

Hi Tor people!

Below you can learn about what OONI has been up to in June 2017.

# OONI Monthly Report: June 2017

The OONI team made steady progress in June 2017. 

We released the beta version of our measurements API and made significant progress on the re-engineering of our data processing pipeline and on probe orchestration. In collaboration with Access Now, we published a research report on recent censorship findings in Egypt. We also updated test lists in collaboration with our partners, and we continued organizing OONI's Partner Gathering.

## Progress on the re-engineering of our data processing pipeline

This month we made a lot of progress on the data processing pipeline. The new pipeline is now running on a daily basis side-by-side the existing data processing pipeline.

In particular, the measurements API is now using the data generated from the new data processing pipeline to produce the various API endpoints and data is being ingested and processed using our new compressed data format, making it much more space efficient and performant.

## Beta release of measurements API

We have released a beta version of the measurement API that is using as a backend the re-engineered data pipeline.
See: https://measurements-beta.ooni.io
     https://github.com/TheTorProject/ooni-measurements/releases/tag/v1.0.0-beta.3

This version of the measurements API also adds a couple of new endpoints including:

* Support for searching individual measurements (/api/v1/measurements?<query>)
* Support for fetching an individual measurement (/api/v1/measurement/<measurement_id>)

Other improvements include:
* Better overall performance
* Some bug fixes

## Progress on probe orchestration

In the area of probe orchestration we made a series of very important improvements, namely:

* Added support for creating alert type notifications (this allows us to send messages to users in specific regions, that when clicked on redirect them to some web page)
* Switched out the push notification backend with something more stable and reliable (gorush: https://github.com/appleboy/gorush)

## Publication of research findings in Egypt

We investigated recent censorship events in Egypt and, in collaboration with Access Now, published a research report with our findings. 

We were able to confirm the blocking of 10 news websites in Egypt, as well as the blocking of the Tor network, torproject.org, bridges.torproject.org, and even ooni.torproject.org in some networks.

The report can be found here: https://ooni.torproject.org/post/egypt-censors/

## Updated test lists

During June we updated the following test lists:

Zimbabwe: https://github.com/citizenlab/test-lists/pull/176
Azerbaijan: https://github.com/citizenlab/test-lists/pull/178
Palestine: https://github.com/citizenlab/test-lists/pull/180
Egypt: https://github.com/citizenlab/test-lists/pull/181
Venezuela: https://github.com/citizenlab/test-lists/pull/184

In collaboration with new partners, we also created the following test lists:

Lesotho: https://github.com/citizenlab/test-lists/pull/177
Mongolia: https://github.com/citizenlab/test-lists/pull/184

## Organizing OONI Partner Gathering

We worked on logistics pertaining to the organization of OONI's first Partner Gathering which took place on 10th & 11th July 2017 in Toronto.

### Community meeting

We hosted our monthly community meeting on https://slack.openobservatory.org/ on 28th June 2017. As part of the meeting, we discussed the following:

Resources for mapping information controls around the world
Creating an information controls database to contextualize network measurements from around the world
OONI's upcoming probe orchestration 

## Userbase

In June 2017 ooniprobe was run 64 times from 1,594 different vantage points across 158 countries around the world.

This information can also be found through our stats here: https://measurements.ooni.torproject.org/stats

~ The OONI team.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.torproject.org/pipermail/tor-project/attachments/20170724/c0d6e500/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 208 bytes
Desc: Message signed with OpenPGP using AMPGpg
URL: <http://lists.torproject.org/pipermail/tor-project/attachments/20170724/c0d6e500/attachment.sig>