[tor-project] Requiring JavaScript on Tor Metrics

[Karsten Loesing]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

Hi teor,

On 14/11/16 10:34, teor wrote:
> 
>> On 14 Nov. 2016, at 02:54, Karsten Loesing
>> <karsten at torproject.org> wrote:
>> 
>> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256
>> 
>> Hi everyone,
>> 
>> I'm moving this (old but recently continued) thread from an
>> internal mailing list to this list as suggested by Roger.  Any
>> feedback would be most useful by Wednesday, November 16.
>> Thanks!
>> 
>> All the best, Karsten
>> 
>> 
>> On 07/12/15 14:09, Karsten Loesing wrote:
>>> Hi everyone,
>>> 
>>> we're discussing changing the graphing engine on Tor Metrics
>>> from generating static images on the server using R/ggplot2
>>> towards generating interactive visualizations using JavaScript
>>> on the client.
>>> 
>>> As a result, Tor Metrics will require Tor Browser users to
>>> switch to Medium-High Security or lower.
>>> 
>>> This switch has some potential with respect to visualizations,
>>> and the visualization people in the metrics team want to do it.
>>> It allows producing more graphs like Lunar's bubble graph:
>>> 
>>> https://metrics.torproject.org/bubbles.html
>>> 
>>> In fact, it would allow all kinds of visualizations like the
>>> ones seen in the D3 gallery:
>>> 
>>> https://github.com/mbostock/d3/wiki/Gallery
>>> 
>>> So, my question is: can you all live with this change?
>>> 
>>> The next step will be to ask users on tor-talk@, but I figured
>>> I should ask here first.  If I don't hear any strong objections
>>> by Thursday, I'll go ask on tor-talk at .
>>> 
>>> Thanks!
>>> 
>>> All the best, Karsten
>> 
>> 
>> On 07/12/15 15:22, somebody else wrote:
>>> I assume it would be infeasible to maintain non-Javascript 
>>> fallbacks, correct?
>>> 
>>> Likewise, I assume it would be too difficult (for now (?)) to
>>> try something crazy-fancy like render the javascript versions
>>> on the server and serve them as images in that case? [0]
>>> 
>>> [0] I'm hand-waving about whether or not this is possible, but
>>> it seems like it in theory, and at least one person has tried:
>>> 
>> http://blog.davidpadbury.com/2010/10/03/using-nodejs-to-render-js-charts-on-server/
>>
>>
>>
>> 
On 08/12/15 00:34, somebody wrote:
>>> I don't understand why you would want Tor users to lower their
>>>  security in order to get prettier graphs.  Is there some
>>> obvious thing I am missing?
>>> 
>>> Personally I run with Javascript disabled on the vast majority
>>> of web sites I access (via NoScript), and with third party
>>> "included" Javascript disabled virtually everywhere (via
>>> PrivacyBadger and sometimes RequestPolicy).  Having web sites
>>> run their own choice of programs in my computer seems like a
>>> really stupid idea.  I can only assume that its prevalence as a
>>> technique reflects how little respect the average web site has
>>> for the security of its users.
>> 
>> 
>> On 09/12/15 08:45, another person wrote:
>>> I'd like to know how much work it would be.
>> 
>> 
>> On 09/12/15 13:59, Karsten Loesing wrote:
>>> I'll try to find out.  I think that having non-JavaScript 
>>> fallbacks written in a different language would be infeasible.
>>> But I could imagine that we render D3.js graphs on the server
>>> using Node.js and return images to the client.
>>> 
>>> I'm mostly worried about the lack of interactivity.  I'd
>>> really want to get away from requiring a full round-trip from
>>> client to server and back just to change something in the
>>> displayed graph. Maybe we can keep them somewhat interactive,
>>> or at least add tooltips, by using SVG as image format.
>>> 
>>> But please understand that while I'm doing okay at processing 
>>> large amounts of data, I don't know much about web development.
>>> If people here have ideas, please let me know!
>>> 
>>> Note that investigating these options may take a while, mostly 
>>> because people in the metrics team are busy.  But we won't
>>> switch to client-side JavaScript before we know about possible 
>>> alternatives.
>>> 
>>> Thanks for the feedback.  This is very helpful!
>>> 
>>> All the best, Karsten
>> 
>> 
>> On 12/11/16 16:37, Karsten Loesing wrote:
>>> Hello everyone,
>>> 
>>> apologies for digging out such an old thread, but after almost
>>>  starting a new thread on the exact same topic I remembered
>>> that we had this discussion almost a year ago.  I figured it's
>>> better to continue this thread to avoid discussing the exact
>>> same things again and instead add new thoughts below.
>>> 
>>> So, on Thursday, Linda, iwakeh, and I met in Berlin to talk
>>> about making the Tor Metrics website more usable.  We came up
>>> with some pretty good ideas to better address the various user
>>> types---from journalists to data scientists---coming to the Tor
>>> Metrics website to learn interesting things about the deployed
>>> Tor network.  We'll share results with you in a few weeks from
>>> now when there's something to see and click on.
>>> 
>>> But in this context I want to bring up the topic of JavaScript 
>>> once more.
>>> 
>>> To be clear, our immediate plans---including reorganizing 
>>> information and displaying sparklines as quick entry points
>>> into the data---don't require JavaScript on Tor Metrics.  And
>>> even our planned improvements might work without
>>> JavaScript---including more customizable graphs and even a
>>> dashboard with user-selected graphs---can be made to work
>>> without JavaScript.  Though the latter will be a stretch.
>>> 
>>> The question is: do we have to keep stretching by avoiding a
>>> web techology that would make our lives and the lives of our
>>> users so much easier?
>>> 
>>> This isn't really something that we can work around by
>>> generating graphs with a JavaScript library on the server.  I'd
>>> want us to switch to another graphing framework such as R Shiny
>>> (which I used for the webstats prototype) or D3.js (which we
>>> currently use on Tor Metrics just for the bubbles graph, though
>>> not in the most efficient way).
>>> 
>>> - From a development perspective this switch would make a lot
>>> of sense, because we'd have to write a lot less code for new
>>> graphs and because there'd be potential contributors out there
>>> who'd appreciate working with a known framework.  Our current
>>> graphing engine doesn't scale much longer, and this does slow
>>> us down.
>>> 
>>> Note that I'm not arguing to use JavaScript in all places, just
>>>  because we can.  I'm in favor of keeping all the parts of Tor 
>>> Metrics where we provide textual or static information entirely
>>>  JavaScript-free, so that our data will still be available to
>>> users that don't have or don't want to use JavaScript.  And
>>> services like ExoneraTor could easily stay JavaScript-free.
>>> 
>>> But the parts of Tor Metrics where we're providing
>>> visualizations and letting users explore our data would require
>>> JavaScript.  This would include all graphs and tables, because
>>> we shouldn't be maintaining two graphing engines.
> 
> Are there any privacy or security advantages to having client-side 
> JavaScript?
> 
> For example, if we download data from the server, and then render
> it on the client using JavaScript, then the server knows less about
> what the client is requesting and visualising.
> 
> Are there ways of coding the metrics website in JavaScript that 
> improve client privacy in this way? (Or does it cost too much
> effort or too much bandwidth to pull down larger datasets just to
> hide what the client is looking at?)

Fine question.  I assume most Tor Metrics users don't care that much
about leaking to the server what part of the data they're looking at.
 And I assume that those who do might download the CSV files and look
at them locally.  But my assumptions might be wrong.

So, the switch to JavaScript may or may not address this.  If we pick
something like Shiny, graphs will still be generated on the server,
and there wouldn't be any difference with regard to client privacy.
If we use D3.js, the browser downloads the data it needs and produces
a graph locally.

I think, overall, I wouldn't add Tor Metrics client privacy towards
the server as a hard requirement, because we already have too many of
those.  If the solution we decide on offers more client privacy than
the current solution, great.  But if it doesn't, okay.

>>> So, how do we decide this?  I believe that this should be a 
>>> Tor-wide decision.  My main worry is that we're sinking weeks
>>> and weeks of development effort into this switch without many
>>> Tor people noticing, and then once we publish and they get
>>> aware, we need to roll back, wasting all the effort.
>>> 
>>> But to be honest, we're wasting effort right now by keeping the
>>>  workarounds and implementing hacks with dynamically generated
>>> HTML forms and potentially dozens of parameters just to avoid
>>> the devil called JavaScript.  This feels like a bad use of our
>>> time.
>>> 
>>> Here's my suggestion: unless somebody raises a valid concern
>>> how requiring JavaScript on Tor Metrics is *bad for Tor*, say,
>>> by next Wednesday, November 16, we're putting this on the hold
>>> again. (That's the day before the next metrics team meeting and
>>> Vegas meeting, and it should be enough time to raise your
>>> voice.)
>>> 
>>> Otherwise we'll switch.
>>> 
>>> Oh, and if you're in favor of switching, please consider
>>> saying that, too.  Thanks.
>>> 
>>> All the best, Karsten
>> 
>> 
>> On 13/11/16 05:04, Roger Dingledine wrote:
>>> Does this mean that we'd be breaking the "download a static
>>> version of the graph" feature too? I use that feature a lot for
>>> grabbing snapshots to put into presentations, and we also want
>>> to use it for pulling in metrics graphs in blog posts, e.g. 
>>> https://blog.torproject.org/blog/tracking-impact-whatsapp-blockage-tor
>>>
>>>
>>
>>> 
as well as for external journalists.
>>> 
>>> Oh, I should also say this would be a great topic for the 
>>> tor-project list, since it doesn't need to be sekrit (and
>>> since then other people could know that it's a topic we're
>>> considering, and maybe even help us make the right decision).
>> 
>> 
>> On 13/11/16 10:08, Karsten Loesing wrote:
>>> It's quite easy to implement that feature using Shiny, for 
>>> example. Either Shiny would produce a .png file that you can 
>>> download using your browser's "Save Image As..." feature, or
>>> there would be a "Download Graph" button.  It would be just a
>>> few lines of code.
>>> 
>>> And we'd be able to provide features like a "Download graph
>>> data" button with just a few more lines of code, which would
>>> require a lot more effort right now.
> 
> Just to clarify, would users need JavaScript turned on to use the 
> "Download Graph" or "Download graph data" buttons?
> 
> Would it be possible to give others a URL for a specific graph, 
> without having to save the graph on some other site? Would users
> need JavaScript enabled to view the graph?
> 
> If we can't do this, it would be bad for how I and many others 
> typically use Tor Metrics on mailing lists.

Right, very good point.  I don't know Shiny enough yet to give you an
answer with absolute certainty, but I think that we should be able to
provide a page with a static image that can be used on mailing lists.
 That page would just have no controls for further customizing the
graph, unless the browser supports JavaScript.

> (And if we do allow this feature, I'm not sure how that's any 
> different from server-side rendering of graphs for clients that 
> don't use JavaScript.)

One major difference is that Shiny, for example, allows us to generate
a user interface with just a few lines of code, whereas we'd otherwise
have to write and maintain that user interface ourselves.  And I'm not
very optimistic to find a framework similar to Shiny that works
without client-side JavaScript, mostly because that's not a
requirement for most websites.

> I'm also not sure what you mean by "tables" or "Download graph 
> data" - will there still be CSV data downloads available?
> 
> Is it the aggregated data that would be in the (JavaScript-only)
> tables, and the raw data in the static CSVs?

The following is not at all final yet, but ideally, all parts that are
updated twice per day in the background can be obtained without
JavaScript whereas any data that gets filtered or aggregated by user
request would require JavaScript.  So, the current CSV data would
still be available without JavaScript.

> In general, I would prefer to be able to use the Tor Metrics 
> website without enabling JavaScript. I don't like the idea that we
> provide Tor Browser, where we recommend(?) people turn JavaScript
> off, and then also provide websites where they need to turn it on.
> But if the advantages outweigh the security and consistency
> considerations, I'd be ok with it.
> 
> (Just like I use Atlas, and the Metrics Bubble graphs.)

Right, this possible inconsistency is what has stopped me in the past.
 And I think if we really want to be consistent here, we'll have to
rewrite Atlas and the Metrics bubble graphs to not require client-side
JavaScript anymore, which is certainly doable.  And we should probably
look at other torproject.org websites that require JavaScript in the
browser and rewrite those parts.

But do we really want this?  If we recommend that people turn
JavaScript off, why do we even support modes in Tor Browser where they
turn it on again, isn't that inconsistent, too?  I think no, because
we acknowledge that not everybody has the same requirements.  And in
this case I believe that the target audience of Tor Metrics is not
exactly the same as the set of high-security Tor Browser users.

On the other hand I see that weakening the no-JavaScript rule for the
Tor Metrics case might influence future decisions for other
torproject.org websites.  Not an easy decision.

> But my preference would be for users without JavaScript to still 
> have some level of basic functionality, even if it's only static 
> images and tables (which I think should be available for offline 
> use as well).

Right.  Two examples for parts of Tor Metrics that would require
JavaScript are (1) the graphs where users can change the x axis,
change lines, add new lines, and so on, and (2) a dashboard where
users can configure the graphs they want to see next to each other.

But users without JavaScript should still be able to navigate the
website, learn about where the data comes from, download the raw or
pre-processed data, and contemplate the graphs posted on mailing lists.

Thanks for your input here!

All the best,
Karsten

-----BEGIN PGP SIGNATURE-----
Comment: GPGTools - http://gpgtools.org

iQEcBAEBCAAGBQJYKZsCAAoJEC3ESO/4X7XBjy4IAK6f8GUhOk+tp4mAaJ8XFBAD
Y7XXFxjMClDH5g6Xi4VDEyVq4xytD/Y1u60un7oh57ZKnVSBH38TkZSCuZfNiLM8
GTWEZsYJRr53JLajCbHpZB+JRw6wvxuzwPoNp5SvY/2aDtvHm0ke379r3PfKDyNs
1SxCD6oAbg4kh9GFlbfBHPpdwyJftZ7k0Hu9oGdJE0erC0J0lLsviPk91eKiP8nf
BsaYNKXtcZ29bba/GkkMGxayzjJO8bJciNOIFZ7SymOVwKalquwqS7q2VCIkZnO0
P+wnGOrlRqb69HWynb1RM024922YSU3kDSsfUMBt6GIVcGTqFY3vNYzIUZc4aJs=
=P1q7
-----END PGP SIGNATURE-----