[tor-dev] 24 hours worth of BridgeDB usage metrics

Philipp Winter phw at nymity.ch
Mon Jul 29 22:01:01 UTC 2019 

Over at <https://bugs.torproject.org/9316>, we are working on having
BridgeDB export metrics.  The patch is almost done and I deployed the
work-in-progress code on BridgeDB, so we can take a look at the metrics
and think of ways to improve them.  The metrics format encodes the
approximate number of requests per distribution mechanism per transport
per country per success/fail.  All numbers are rounded up to the next
multiple of 10.  The last field, "none", will be used for an anomaly
score and is currently unused.

For example, the line

  bridgedb-metric-count email.obfs4.riseup.success.none 10

tells us that there have been 1-10 successful email requests for obfs4
coming from Riseup addresses.

I attached 24 hours worth of metrics to this email.  Keep the following
issues in mind:

* My feature branch hasn't been reviewed yet and likely still has bugs,
  so take all numbers with a grain of salt.

* The country codes are based on Debian stretch's geoip-database, which
  is slightly outdated and uses Maxmind's far-from-perfect GeoLite
  database.

* The country code "??" refers to geo-location failure or lack of IP
  addresses (in the cast of moat).  The country code "zz" refers to a
  request from a Tor exit relay.

Some observations:

* Gmail sees much more use than Riseup.  That's no surprise.

* The email distributor sees more vanilla than obfs4 requests.
  I wonder to what degree this is caused by the poor UX of the email
  distributor.

* For HTTPS, many countries have a fail and success bucket of 10 each.
  I would expect this to be at least one user who failed the captcha at
  least once before finally getting it right.

* The captcha success rate for obfs4 over moat is 54%.  That's very low
  and must cause lots of frustration for users.  This is a known issue
  that's tracked in this ticket: <https://bugs.torproject.org/29695>

* Ignore the large amount of HTTPS requests from "zz" -- I expect the
  vast majority of these to be a bot that's interacting with BridgeDB
  over exit relays.

After a cursory look at the numbers, I would like to aggregate the data,
to make it easier to compare distributors, transports, and countries.
For example: how do moat, email, and HTTPS rank in popularity?  I'll
improve the patch to keep track of these numbers in separate metrics.

Any thoughts or suggestions?

Cheers,
Philipp
-------------- next part --------------
bridgedb-stats-end 2019-07-28 22:15:02 (86400 s)
bridgedb-metric-count email.fte.gmail.success.none 30
bridgedb-metric-count email.obfs3.gmail.success.none 10
bridgedb-metric-count email.obfs3.riseup.success.none 10
bridgedb-metric-count email.obfs4.gmail.fail.none 10
bridgedb-metric-count email.obfs4.gmail.success.none 70
bridgedb-metric-count email.obfs4.riseup.fail.none 10
bridgedb-metric-count email.obfs4.riseup.success.none 10
bridgedb-metric-count email.obs4.gmail.fail.none 10
bridgedb-metric-count email.obs4.gmail.success.none 10
bridgedb-metric-count email.obsf4.gmail.success.none 10
bridgedb-metric-count email.scramblesuit.gmail.fail.none 10
bridgedb-metric-count email.scramblesuit.gmail.success.none 10
bridgedb-metric-count email.vanilla.gmail.fail.none 60
bridgedb-metric-count email.vanilla.gmail.success.none 170
bridgedb-metric-count email.vanilla.riseup.success.none 10
bridgedb-metric-count https.fte.??.fail.none 10
bridgedb-metric-count https.fte.??.success.none 10
bridgedb-metric-count https.fte.by.success.none 10
bridgedb-metric-count https.fte.fr.success.none 10
bridgedb-metric-count https.fte.gb.fail.none 10
bridgedb-metric-count https.fte.ru.success.none 10
bridgedb-metric-count https.fte.us.fail.none 10
bridgedb-metric-count https.fte.us.success.none 10
bridgedb-metric-count https.fte.zz.fail.none 10
bridgedb-metric-count https.fte.zz.success.none 10
bridgedb-metric-count https.obfs2.jp.fail.none 10
bridgedb-metric-count https.obfs2.us.fail.none 10
bridgedb-metric-count https.obfs2.zz.fail.none 130
bridgedb-metric-count https.obfs2.zz.success.none 6270
bridgedb-metric-count https.obfs3.??.fail.none 10
bridgedb-metric-count https.obfs3.??.success.none 10
bridgedb-metric-count https.obfs3.by.fail.none 10
bridgedb-metric-count https.obfs3.by.success.none 10
bridgedb-metric-count https.obfs3.ca.fail.none 10
bridgedb-metric-count https.obfs3.ca.success.none 10
bridgedb-metric-count https.obfs3.ch.success.none 10
bridgedb-metric-count https.obfs3.cn.fail.none 10
bridgedb-metric-count https.obfs3.cn.success.none 10
bridgedb-metric-count https.obfs3.de.success.none 10
bridgedb-metric-count https.obfs3.fr.fail.none 10
bridgedb-metric-count https.obfs3.fr.success.none 10
bridgedb-metric-count https.obfs3.hk.fail.none 10
bridgedb-metric-count https.obfs3.kz.fail.none 10
bridgedb-metric-count https.obfs3.kz.success.none 10
bridgedb-metric-count https.obfs3.ru.fail.none 10
bridgedb-metric-count https.obfs3.ru.success.none 10
bridgedb-metric-count https.obfs3.tw.success.none 10
bridgedb-metric-count https.obfs3.ua.fail.none 10
bridgedb-metric-count https.obfs3.us.fail.none 10
bridgedb-metric-count https.obfs3.us.success.none 10
bridgedb-metric-count https.obfs3.zz.fail.none 180
bridgedb-metric-count https.obfs3.zz.success.none 6240
bridgedb-metric-count https.obfs4.??.fail.none 130
bridgedb-metric-count https.obfs4.??.success.none 70
bridgedb-metric-count https.obfs4.at.fail.none 20
bridgedb-metric-count https.obfs4.at.success.none 30
bridgedb-metric-count https.obfs4.au.fail.none 10
bridgedb-metric-count https.obfs4.au.success.none 10
bridgedb-metric-count https.obfs4.bd.fail.none 10
bridgedb-metric-count https.obfs4.bd.success.none 10
bridgedb-metric-count https.obfs4.be.success.none 10
bridgedb-metric-count https.obfs4.br.fail.none 10
bridgedb-metric-count https.obfs4.br.success.none 10
bridgedb-metric-count https.obfs4.by.fail.none 10
bridgedb-metric-count https.obfs4.by.success.none 10
bridgedb-metric-count https.obfs4.ca.fail.none 10
bridgedb-metric-count https.obfs4.ca.success.none 10
bridgedb-metric-count https.obfs4.ch.fail.none 10
bridgedb-metric-count https.obfs4.ch.success.none 10
bridgedb-metric-count https.obfs4.cn.fail.none 100
bridgedb-metric-count https.obfs4.cn.success.none 10
bridgedb-metric-count https.obfs4.co.success.none 10
bridgedb-metric-count https.obfs4.cu.fail.none 10
bridgedb-metric-count https.obfs4.cz.success.none 10
bridgedb-metric-count https.obfs4.de.fail.none 10
bridgedb-metric-count https.obfs4.de.success.none 20
bridgedb-metric-count https.obfs4.ee.success.none 10
bridgedb-metric-count https.obfs4.eg.fail.none 10
bridgedb-metric-count https.obfs4.eg.success.none 10
bridgedb-metric-count https.obfs4.fr.fail.none 20
bridgedb-metric-count https.obfs4.fr.success.none 20
bridgedb-metric-count https.obfs4.gb.fail.none 40
bridgedb-metric-count https.obfs4.gb.success.none 30
bridgedb-metric-count https.obfs4.hk.success.none 10
bridgedb-metric-count https.obfs4.hu.success.none 10
bridgedb-metric-count https.obfs4.in.fail.none 10
bridgedb-metric-count https.obfs4.in.success.none 10
bridgedb-metric-count https.obfs4.it.fail.none 10
bridgedb-metric-count https.obfs4.it.success.none 10
bridgedb-metric-count https.obfs4.jp.fail.none 10
bridgedb-metric-count https.obfs4.jp.success.none 10
bridgedb-metric-count https.obfs4.kz.fail.none 10
bridgedb-metric-count https.obfs4.kz.success.none 20
bridgedb-metric-count https.obfs4.lv.success.none 10
bridgedb-metric-count https.obfs4.md.success.none 10
bridgedb-metric-count https.obfs4.mx.success.none 10
bridgedb-metric-count https.obfs4.ng.success.none 10
bridgedb-metric-count https.obfs4.nl.success.none 10
bridgedb-metric-count https.obfs4.ph.success.none 10
bridgedb-metric-count https.obfs4.pl.fail.none 10
bridgedb-metric-count https.obfs4.pl.success.none 10
bridgedb-metric-count https.obfs4.ro.fail.none 10
bridgedb-metric-count https.obfs4.ro.success.none 10
bridgedb-metric-count https.obfs4.ru.fail.none 30
bridgedb-metric-count https.obfs4.ru.success.none 30
bridgedb-metric-count https.obfs4.sc.success.none 10
bridgedb-metric-count https.obfs4.se.fail.none 10
bridgedb-metric-count https.obfs4.se.success.none 10
bridgedb-metric-count https.obfs4.sg.fail.none 10
bridgedb-metric-count https.obfs4.sg.success.none 10
bridgedb-metric-count https.obfs4.tr.success.none 10
bridgedb-metric-count https.obfs4.tw.success.none 10
bridgedb-metric-count https.obfs4.ua.fail.none 10
bridgedb-metric-count https.obfs4.ua.success.none 10
bridgedb-metric-count https.obfs4.us.fail.none 380
bridgedb-metric-count https.obfs4.us.success.none 70
bridgedb-metric-count https.obfs4.ve.success.none 10
bridgedb-metric-count https.obfs4.zz.fail.none 580
bridgedb-metric-count https.obfs4.zz.success.none 6520
bridgedb-metric-count https.scramblesuit.??.fail.none 10
bridgedb-metric-count https.scramblesuit.??.success.none 10
bridgedb-metric-count https.scramblesuit.at.success.none 10
bridgedb-metric-count https.scramblesuit.br.fail.none 10
bridgedb-metric-count https.scramblesuit.br.success.none 10
bridgedb-metric-count https.scramblesuit.by.success.none 10
bridgedb-metric-count https.scramblesuit.ch.success.none 10
bridgedb-metric-count https.scramblesuit.co.success.none 10
bridgedb-metric-count https.scramblesuit.fr.success.none 10
bridgedb-metric-count https.scramblesuit.gb.fail.none 10
bridgedb-metric-count https.scramblesuit.gb.success.none 10
bridgedb-metric-count https.scramblesuit.jp.fail.none 10
bridgedb-metric-count https.scramblesuit.jp.success.none 10
bridgedb-metric-count https.scramblesuit.ng.success.none 10
bridgedb-metric-count https.scramblesuit.ru.success.none 10
bridgedb-metric-count https.scramblesuit.sg.success.none 10
bridgedb-metric-count https.scramblesuit.us.fail.none 10
bridgedb-metric-count https.scramblesuit.us.success.none 10
bridgedb-metric-count https.scramblesuit.zz.fail.none 470
bridgedb-metric-count https.scramblesuit.zz.success.none 6360
bridgedb-metric-count https.vanilla.??.fail.none 60
bridgedb-metric-count https.vanilla.??.success.none 40
bridgedb-metric-count https.vanilla.an.fail.none 10
bridgedb-metric-count https.vanilla.an.success.none 10
bridgedb-metric-count https.vanilla.at.fail.none 30
bridgedb-metric-count https.vanilla.at.success.none 20
bridgedb-metric-count https.vanilla.au.fail.none 10
bridgedb-metric-count https.vanilla.au.success.none 10
bridgedb-metric-count https.vanilla.bd.fail.none 10
bridgedb-metric-count https.vanilla.bd.success.none 10
bridgedb-metric-count https.vanilla.be.fail.none 10
bridgedb-metric-count https.vanilla.be.success.none 10
bridgedb-metric-count https.vanilla.br.fail.none 20
bridgedb-metric-count https.vanilla.br.success.none 20
bridgedb-metric-count https.vanilla.by.fail.none 10
bridgedb-metric-count https.vanilla.by.success.none 20
bridgedb-metric-count https.vanilla.ca.fail.none 20
bridgedb-metric-count https.vanilla.ca.success.none 20
bridgedb-metric-count https.vanilla.ch.fail.none 10
bridgedb-metric-count https.vanilla.ch.success.none 10
bridgedb-metric-count https.vanilla.cl.success.none 10
bridgedb-metric-count https.vanilla.cn.fail.none 10
bridgedb-metric-count https.vanilla.cn.success.none 10
bridgedb-metric-count https.vanilla.cz.success.none 10
bridgedb-metric-count https.vanilla.de.fail.none 20
bridgedb-metric-count https.vanilla.de.success.none 20
bridgedb-metric-count https.vanilla.dz.fail.none 10
bridgedb-metric-count https.vanilla.dz.success.none 10
bridgedb-metric-count https.vanilla.ec.success.none 10
bridgedb-metric-count https.vanilla.eg.fail.none 10
bridgedb-metric-count https.vanilla.eg.success.none 10
bridgedb-metric-count https.vanilla.es.fail.none 10
bridgedb-metric-count https.vanilla.es.success.none 10
bridgedb-metric-count https.vanilla.et.fail.none 10
bridgedb-metric-count https.vanilla.et.success.none 10
bridgedb-metric-count https.vanilla.eu.fail.none 10
bridgedb-metric-count https.vanilla.fi.fail.none 10
bridgedb-metric-count https.vanilla.fi.success.none 10
bridgedb-metric-count https.vanilla.fr.fail.none 10
bridgedb-metric-count https.vanilla.fr.success.none 20
bridgedb-metric-count https.vanilla.gb.fail.none 20
bridgedb-metric-count https.vanilla.gb.success.none 30
bridgedb-metric-count https.vanilla.ge.fail.none 10
bridgedb-metric-count https.vanilla.ge.success.none 10
bridgedb-metric-count https.vanilla.gr.fail.none 10
bridgedb-metric-count https.vanilla.gr.success.none 10
bridgedb-metric-count https.vanilla.hk.fail.none 10
bridgedb-metric-count https.vanilla.hk.success.none 20
bridgedb-metric-count https.vanilla.hr.success.none 10
bridgedb-metric-count https.vanilla.hu.fail.none 10
bridgedb-metric-count https.vanilla.hu.success.none 10
bridgedb-metric-count https.vanilla.id.fail.none 10
bridgedb-metric-count https.vanilla.id.success.none 10
bridgedb-metric-count https.vanilla.ie.fail.none 10
bridgedb-metric-count https.vanilla.ie.success.none 10
bridgedb-metric-count https.vanilla.il.success.none 10
bridgedb-metric-count https.vanilla.in.fail.none 30
bridgedb-metric-count https.vanilla.in.success.none 20
bridgedb-metric-count https.vanilla.iq.success.none 10
bridgedb-metric-count https.vanilla.is.fail.none 10
bridgedb-metric-count https.vanilla.is.success.none 10
bridgedb-metric-count https.vanilla.it.fail.none 20
bridgedb-metric-count https.vanilla.it.success.none 10
bridgedb-metric-count https.vanilla.jm.fail.none 10
bridgedb-metric-count https.vanilla.jm.success.none 10
bridgedb-metric-count https.vanilla.jp.fail.none 10
bridgedb-metric-count https.vanilla.jp.success.none 10
bridgedb-metric-count https.vanilla.ke.success.none 10
bridgedb-metric-count https.vanilla.kr.fail.none 10
bridgedb-metric-count https.vanilla.kr.success.none 10
bridgedb-metric-count https.vanilla.kw.fail.none 10
bridgedb-metric-count https.vanilla.kw.success.none 10
bridgedb-metric-count https.vanilla.kz.fail.none 20
bridgedb-metric-count https.vanilla.kz.success.none 20
bridgedb-metric-count https.vanilla.la.success.none 10
bridgedb-metric-count https.vanilla.lv.fail.none 10
bridgedb-metric-count https.vanilla.lv.success.none 10
bridgedb-metric-count https.vanilla.mx.fail.none 10
bridgedb-metric-count https.vanilla.mx.success.none 10
bridgedb-metric-count https.vanilla.ng.fail.none 10
bridgedb-metric-count https.vanilla.ng.success.none 10
bridgedb-metric-count https.vanilla.ni.fail.none 10
bridgedb-metric-count https.vanilla.ni.success.none 10
bridgedb-metric-count https.vanilla.nl.fail.none 10
bridgedb-metric-count https.vanilla.nl.success.none 20
bridgedb-metric-count https.vanilla.no.fail.none 10
bridgedb-metric-count https.vanilla.no.success.none 10
bridgedb-metric-count https.vanilla.np.fail.none 10
bridgedb-metric-count https.vanilla.nz.success.none 10
bridgedb-metric-count https.vanilla.pe.fail.none 10
bridgedb-metric-count https.vanilla.pe.success.none 10
bridgedb-metric-count https.vanilla.pg.fail.none 10
bridgedb-metric-count https.vanilla.pg.success.none 10
bridgedb-metric-count https.vanilla.ph.success.none 10
bridgedb-metric-count https.vanilla.pk.fail.none 10
bridgedb-metric-count https.vanilla.pk.success.none 10
bridgedb-metric-count https.vanilla.pl.fail.none 10
bridgedb-metric-count https.vanilla.pl.success.none 10
bridgedb-metric-count https.vanilla.re.fail.none 10
bridgedb-metric-count https.vanilla.re.success.none 10
bridgedb-metric-count https.vanilla.ru.fail.none 100
bridgedb-metric-count https.vanilla.ru.success.none 90
bridgedb-metric-count https.vanilla.se.fail.none 10
bridgedb-metric-count https.vanilla.se.success.none 10
bridgedb-metric-count https.vanilla.sg.success.none 10
bridgedb-metric-count https.vanilla.si.fail.none 10
bridgedb-metric-count https.vanilla.sk.fail.none 10
bridgedb-metric-count https.vanilla.sk.success.none 10
bridgedb-metric-count https.vanilla.so.fail.none 10
bridgedb-metric-count https.vanilla.tr.fail.none 10
bridgedb-metric-count https.vanilla.tr.success.none 10
bridgedb-metric-count https.vanilla.ua.fail.none 10
bridgedb-metric-count https.vanilla.ua.success.none 10
bridgedb-metric-count https.vanilla.us.fail.none 90
bridgedb-metric-count https.vanilla.us.success.none 100
bridgedb-metric-count https.vanilla.ve.success.none 10
bridgedb-metric-count https.vanilla.vn.fail.none 10
bridgedb-metric-count https.vanilla.vn.success.none 10
bridgedb-metric-count https.vanilla.ye.fail.none 10
bridgedb-metric-count https.vanilla.ye.success.none 10
bridgedb-metric-count https.vanilla.za.fail.none 10
bridgedb-metric-count https.vanilla.zz.fail.none 870
bridgedb-metric-count https.vanilla.zz.success.none 7100
bridgedb-metric-count moat.obfs4.??.fail.none 2030
bridgedb-metric-count moat.obfs4.??.success.none 2360
bridgedb-metric-count moat.obfs4.cn.fail.none 30
bridgedb-metric-count moat.obfs4.cn.success.none 20
bridgedb-metric-count moat.obfs4.us.fail.none 110
bridgedb-metric-count moat.obfs4.us.success.none 80

More information about the tor-dev mailing list