[tor-project] New Tor Metrics graph: Tor Browser downloads and updates

Karsten Loesing karsten at torproject.org
Thu Feb 2 09:30:37 UTC 2017 

On 30/01/17 09:32, Yawning Angel wrote:
> On Sun, 29 Jan 2017 17:30:23 +0100
> Karsten Loesing <karsten at torproject.org> wrote:
>> Okay, we rely on folks telling us which resource strings to look after
>> and how to interpret them.  If we should be looking for different
>> resource strings in the logs or change the documentation in any way,
>> please tell us!
> 
> Downloading (sandboxed-tor-browser application):
> 
>   https://dist.torproject.org/torbrowser/7.0a1/sandbox-0.0.3-linux64.zip
>   https://dist.torproject.org/torbrowser/7.0a1/sandbox-0.0.3-linux64.zip.asc

I ran a query on the webstats database to see how many downloads of the
sandboxed Tor Browser there are:

webstats=> SELECT log_date, resource_string, SUM(count) AS count
FROM files NATURAL JOIN requests NATURAL JOIN resources
WHERE resource_string LIKE '/torbrowser/%/sandbox-%-linux%.zip'
AND response_code = 200
AND method = 'GET'
AND log_date >= '2017-01-01'
GROUP BY 1, 2
ORDER BY 1, 2;

  log_date  |               resource_string               | count
------------+---------------------------------------------+-------
 2017-01-01 | /torbrowser/6.5a6/sandbox-0.0.2-linux32.zip |    32
 2017-01-01 | /torbrowser/6.5a6/sandbox-0.0.2-linux64.zip |    41
 2017-01-02 | /torbrowser/6.5a6/sandbox-0.0.2-linux32.zip |    35
 2017-01-02 | /torbrowser/6.5a6/sandbox-0.0.2-linux64.zip |    74
 2017-01-03 | /torbrowser/6.5a6/sandbox-0.0.2-linux32.zip |    42
 2017-01-03 | /torbrowser/6.5a6/sandbox-0.0.2-linux64.zip |    75
 2017-01-04 | /torbrowser/6.5a6/sandbox-0.0.2-linux32.zip |    39
 2017-01-04 | /torbrowser/6.5a6/sandbox-0.0.2-linux64.zip |    62
 2017-01-05 | /torbrowser/6.5a6/sandbox-0.0.2-linux32.zip |    52
 2017-01-05 | /torbrowser/6.5a6/sandbox-0.0.2-linux64.zip |    84
 2017-01-06 | /torbrowser/6.5a6/sandbox-0.0.2-linux32.zip |    38
 2017-01-06 | /torbrowser/6.5a6/sandbox-0.0.2-linux64.zip |    48
 2017-01-07 | /torbrowser/6.5a6/sandbox-0.0.2-linux32.zip |    36
 2017-01-07 | /torbrowser/6.5a6/sandbox-0.0.2-linux64.zip |    57
 2017-01-08 | /torbrowser/6.5a6/sandbox-0.0.2-linux32.zip |    35
 2017-01-08 | /torbrowser/6.5a6/sandbox-0.0.2-linux64.zip |    50
 2017-01-09 | /torbrowser/6.5a6/sandbox-0.0.2-linux32.zip |    27
 2017-01-09 | /torbrowser/6.5a6/sandbox-0.0.2-linux64.zip |    58
 2017-01-10 | /torbrowser/6.5a6/sandbox-0.0.2-linux32.zip |    43
 2017-01-10 | /torbrowser/6.5a6/sandbox-0.0.2-linux64.zip |    61
 2017-01-11 | /torbrowser/6.5a6/sandbox-0.0.2-linux32.zip |    59
 2017-01-11 | /torbrowser/6.5a6/sandbox-0.0.2-linux64.zip |    92
 2017-01-12 | /torbrowser/6.5a6/sandbox-0.0.2-linux32.zip |    41
 2017-01-12 | /torbrowser/6.5a6/sandbox-0.0.2-linux64.zip |    48
 2017-01-13 | /torbrowser/6.5a6/sandbox-0.0.2-linux32.zip |    35
 2017-01-13 | /torbrowser/6.5a6/sandbox-0.0.2-linux64.zip |    63
 2017-01-14 | /torbrowser/6.5a6/sandbox-0.0.2-linux32.zip |    34
 2017-01-14 | /torbrowser/6.5a6/sandbox-0.0.2-linux64.zip |    57
 2017-01-15 | /torbrowser/6.5a6/sandbox-0.0.2-linux32.zip |    45
 2017-01-15 | /torbrowser/6.5a6/sandbox-0.0.2-linux64.zip |   102
 2017-01-16 | /torbrowser/6.5a6/sandbox-0.0.2-linux32.zip |    43
 2017-01-16 | /torbrowser/6.5a6/sandbox-0.0.2-linux64.zip |    58
 2017-01-17 | /torbrowser/6.5a6/sandbox-0.0.2-linux32.zip |    30
 2017-01-17 | /torbrowser/6.5a6/sandbox-0.0.2-linux64.zip |    58
 2017-01-18 | /torbrowser/6.5a6/sandbox-0.0.2-linux32.zip |    23
 2017-01-18 | /torbrowser/6.5a6/sandbox-0.0.2-linux64.zip |    64
 2017-01-19 | /torbrowser/6.5a6/sandbox-0.0.2-linux32.zip |    34
 2017-01-19 | /torbrowser/6.5a6/sandbox-0.0.2-linux64.zip |    39
 2017-01-20 | /torbrowser/6.5a6/sandbox-0.0.2-linux32.zip |    38
 2017-01-20 | /torbrowser/6.5a6/sandbox-0.0.2-linux64.zip |    45
 2017-01-21 | /torbrowser/6.5a6/sandbox-0.0.2-linux32.zip |    35
 2017-01-21 | /torbrowser/6.5a6/sandbox-0.0.2-linux64.zip |    49
 2017-01-22 | /torbrowser/6.5a6/sandbox-0.0.2-linux32.zip |    45
 2017-01-22 | /torbrowser/6.5a6/sandbox-0.0.2-linux64.zip |    52
 2017-01-23 | /torbrowser/6.5a6/sandbox-0.0.2-linux32.zip |    52
 2017-01-23 | /torbrowser/6.5a6/sandbox-0.0.2-linux64.zip |    41
 2017-01-24 | /torbrowser/6.5a6/sandbox-0.0.2-linux32.zip |    35
 2017-01-24 | /torbrowser/6.5a6/sandbox-0.0.2-linux64.zip |    46
 2017-01-24 | /torbrowser/7.0a1/sandbox-0.0.3-linux64.zip |     1
 2017-01-25 | /torbrowser/6.5a6/sandbox-0.0.2-linux32.zip |     2
 2017-01-25 | /torbrowser/6.5a6/sandbox-0.0.2-linux64.zip |     2
 2017-01-25 | /torbrowser/7.0a1/sandbox-0.0.3-linux64.zip |    99
 2017-01-26 | /torbrowser/6.5a6/sandbox-0.0.2-linux32.zip |     1
 2017-01-26 | /torbrowser/6.5a6/sandbox-0.0.2-linux64.zip |     3
 2017-01-26 | /torbrowser/7.0a1/sandbox-0.0.3-linux64.zip |    76
 2017-01-27 | /torbrowser/6.5a6/sandbox-0.0.2-linux32.zip |     1
 2017-01-27 | /torbrowser/7.0a1/sandbox-0.0.3-linux64.zip |    75
 2017-01-28 | /torbrowser/6.5a6/sandbox-0.0.2-linux32.zip |     1
 2017-01-28 | /torbrowser/6.5a6/sandbox-0.0.2-linux64.zip |     1
 2017-01-28 | /torbrowser/7.0a1/sandbox-0.0.3-linux64.zip |    80
 2017-01-29 | /torbrowser/7.0a1/sandbox-0.0.3-linux64.zip |    61
 2017-01-30 | /torbrowser/6.5a6/sandbox-0.0.2-linux32.zip |     1
 2017-01-30 | /torbrowser/6.5a6/sandbox-0.0.2-linux64.zip |     1
 2017-01-30 | /torbrowser/7.0a1/sandbox-0.0.3-linux64.zip |    75
(64 rows)

And one more with totals per day:

webstats=> SELECT log_date, SUM(count) AS count
FROM files NATURAL JOIN requests NATURAL JOIN resources
WHERE resource_string LIKE '/torbrowser/%/sandbox-%-linux%.zip'
AND response_code = 200
AND method = 'GET'
AND log_date >= '2017-01-01'
GROUP BY 1
ORDER BY 1;

  log_date  | count
------------+-------
 2017-01-01 |    73
 2017-01-02 |   109
 2017-01-03 |   117
 2017-01-04 |   101
 2017-01-05 |   136
 2017-01-06 |    86
 2017-01-07 |    93
 2017-01-08 |    85
 2017-01-09 |    85
 2017-01-10 |   104
 2017-01-11 |   151
 2017-01-12 |    89
 2017-01-13 |    98
 2017-01-14 |    91
 2017-01-15 |   147
 2017-01-16 |   101
 2017-01-17 |    88
 2017-01-18 |    87
 2017-01-19 |    73
 2017-01-20 |    83
 2017-01-21 |    84
 2017-01-22 |    97
 2017-01-23 |    93
 2017-01-24 |    82
 2017-01-25 |   103
 2017-01-26 |    80
 2017-01-27 |    76
 2017-01-28 |    82
 2017-01-29 |    61
 2017-01-30 |    77
(30 rows)

When these numbers go up we should include them in the graph.  But for
now, they wouldn't change the totals much, and the SQL query is complex
enough already, so I'd rather not want to add more special cases to it.

All the best,
Karsten


> 
> sandboxed-tor-browser works similarly to tor-browser-launcher in
> certain regards, and handles downloading/installing an actual bundle.
> For the purposes of illustration, the `release` channel will be shown,
> and I will assume that substituting in the right thing is trivial.
> 
>   Installing tor browser (done only once):
> 
>     http://x3nelbld33llasqv.onion/torbrowser/update_2/release/downloads.json
>     OR    
>     https://aus1.torproject.org/torbrowser/update_2/release/downloads.json
> 
>     https://dist.torproject.org/torbrowser/6.5/tor-browser-linux64-6.5_en-US.tar.xz
>     https://dist.torproject.org/torbrowser/6.5/tor-browser-linux64-6.5_en-US.tar.xz.asc
> 
>     The HS is only used here if a system tor instance is
>     explicitly pre-configured.
> 
>     (This probably shouldn't be hitting up dist.tp.o, but the JSON file
>      is generated by the automated build process, and is beyond my
>      control.)
> 
> sandboxed-tor-browser also handles checking for updates, and keeping
> the installed browser up to date, because the sandboxed firefox instance
> doesn't have sufficient file system privileges to be able to rewrite
> itself (This is a good thing).
> 
>   Update check (every 2h):
> 
>     http://x3nelbld33llasqv.onion/torbrowser/update_2/release/Linux_x86_64-gcc3/6.5/en-US
> 
>     If the fetch from the HS fails, it will immediately attempt to
>     fetch:
> 
>       https://aus1.torproject.org/torbrowser/update_2/release/Linux_x86_64-gcc3/6.5/en-US
> 
>     Any further failures are retried after another 2 hours has elapsed.
> 
>     All other internal update checks present in Tor Browser are
>     disabled, so it will never fetch:
> 
>       https://www.torproject.org/projects/torbrowser/RecommendedTBBVersions
> 
>     If there is an update, it will grab the MAR as specified in the XML
>     response just fetched.
> 
>     This will probably be the main source of inconsistency.  Personally
>     I don't see why `RecommendedTBBVersions` still exists at all.
> 
>     nb: If the user closes sandboxed-tor-browser completely, on next
>     launch an update check cycle is forced before firefox is allowed to
>     execute if it has been over 2 hours since the last update check,
>     after the bootstrap process completes. Complete failure then
>     results in a failure to launch with a modal dialog box (Yes, I know
>     some people will consider this a misfeature.  No, I don't care).
> 
> Regards,
> 
> 
> 
> _______________________________________________
> tor-project mailing list
> tor-project at lists.torproject.org
> https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-project
> 


-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 495 bytes
Desc: OpenPGP digital signature
URL: <http://lists.torproject.org/pipermail/tor-project/attachments/20170202/e3b77f7a/attachment.sig>

More information about the tor-project mailing list