[metrics-team] plots

Thu Jun 1 18:08:29 UTC 2017

Is this Tor's Debian distro?
https://cran.rstudio.com/bin/linux/debian/

Maintain by Dirk Eddelbuettel
www.rstudio.com/resources/videos/extending-r-with-c-a-brief-introduction-to-rcpp/

On Thu, Jun 1, 2017 at 5:23 AM, Karsten Loesing <karsten at torproject.org>
wrote:

> Hi John, please give me another day or four (after the weekend) to
> figure out the best way forward.  Karsten
>
>
> On 29.05.17 18:29, John Williams wrote:
> > followup thought .... I know that deploying a shiny server has not been
> > your priority, but, replacing the 26 plots with shiny apps will sidestep
> > the Debian-R limitation issue , as shiny server runs its own instance of
> R
> > and won't affect legacy code running on the Debian R instance  ... a far
> > better solution, IMO, than updating R code with outdated R functions ....
> >
> > On Mon, May 29, 2017 at 9:01 AM, Karsten Loesing <karsten at torproject.org
> >
> > wrote:
> >
> >> Hi John,
> >>
> >> On 29.05.17 13:35, John Williams wrote:
> >>> Hi Karsten
> >>>
> >>> The modern R tool chain  has the tidyverse
> >>> <https://blog.rstudio.org/2016/09/15/tidyverse-1-0-0/> as it core.
> The
> >>> tidyverse library is a convenience library for installing and loading
> the
> >>> core set of R packages that comprise the tidyverse - ggplot2, dplyr,
> >> tidyr,
> >>> readr, purrr & tibble. A dozen other packages are installed but not
> >>> explicitly loaded by tidyverse.
> >>>
> >>> A high rate of R package innovation currently exists within & beyond
> >>> the tidyverse (e.g., the shiny tool chain) and it would be difficult to
> >>> exclude CRAN <https://cran.r-project.org/>-compliant packages from
> use.
> >>>
> >>> I have high confidence in the people & motivations
> >>> <https://rviews.rstudio.com/2016/10/12/interview-with-j-j-allaire/>
> >> behind
> >>> the tidyverse & shiny tool chains.
> >>>
> >>> Speaking of shiny, a shiny server
> >>> <https://www.rstudio.com/products/shiny/shiny-server> installed in the
> >> Tor
> >>> metrics subnet will enable rapid innovation in Tor metrics. Instead of
> >>> creating plot code to called by javascript, we'll create interactive
> >> shiny
> >>> apps linked by iframe,  and/or standalone, data-centric web apps, just
> >>> using R.
> >>
> >> Can you elaborate on that?  How would we use Shiny Server without
> >> JavaScript?  (I'm not asking, because I think it's impossible.  I'm
> >> asking because I haven't looked at all the details there and am
> curious.)
> >>
> >>> My library of Tor analytics is small but growing:
> >>>
> >>> Exit Nodes <http://rpubs.com/johnbwilliams/nodes>
> >>> Network Size <http://rpubs.com/johnbwilliams/refactor>
> >>> Network Size Delta <http://rpubs.com/johnbwilliams/network_size_delta>
> >>> hidserv-descs-per-hsdir
> >>> <http://rpubs.com/johnbwilliams/hidserv-descs-per-hsdir>
> >>>
> >>> This is all well and good but pales in comparison to what could be done
> >> in
> >>> Tor Metrics with an analytics (shiny) server.
> >>>
> >>> I see that a local cloud service provider, with whom I have done
> business
> >>> from time to time, is hosting a Tor exit address.  I was thinking that
> >> the
> >>> Project could make a business case to this provider's management for
> >>> hosting a shiny server for Tor metrics.
> >>>
> >>> Yes, it is possible but not desirable to revert to base R functions
> from
> >>> readr - a core package of the tidyverse.  I'll do that as requested.
> >>
> >> I can understand that going back to base functions is a bit painful.  My
> >> main goal here is to start making improvements as soon as possible.  I
> >> think it would take some time to get everyone involved in dependencies
> >> on Tor hosts to make a decision here, and I'd rather want to postpone
> >> that and start coding and reviewing.
> >>
> >> Two ideas:
> >>
> >>  - We include our own version of a read_csv that internally uses R base
> >> functions to do the job and that we can later replace by what's in
> readr.
> >>
> >>  - We try to get readr into Debian backports.  A Debian developer told
> >> me that it's not that difficult to do that, and he might help with the
> >> process.  (We could even go one step further and try to get tidyverse
> >> into Debian backports, but that might be a bigger project.)
> >>
> >> I think my preference would be the first option, because that's likely
> >> the quickest way to get the code updated.  What do you think?
> >>
> >>> Cheers,
> >>>
> >>> John
> >>
> >> All the best,
> >> Karsten
> >>
> >>
> >>>
> >>> On Mon, May 29, 2017 at 3:12 AM, Karsten Loesing <
> karsten at torproject.org
> >>>
> >>> wrote:
> >>>
> >>>> Hi John,
> >>>>
> >>>> On 26.05.17 05:02, John Williams wrote:
> >>>>> something like this?
> >>>>>
> >>>>> refactor of network size plot <http://rpubs.com/
> johnbwilliams/refactor
> >>>
> >>>>
> >>>> Looks great!  A lot shorter and clearer than before.
> >>>>
> >>>> By the way, did you change that code after posting here?  I remember
> >>>> seeing `library(tidyverse)` in the original version, which I couldn't
> >>>> find in Debian stable or backports.
> >>>>
> >>>> The current code doesn't contain `library(tidyverse)` anymore, but it
> >>>> has `library(readr)` which I couldn't find in Debian stable or
> backports
> >>>> either.
> >>>>
> >>>> The other libraries are all available in Debian stable or backports as
> >>>> far as I can see.
> >>>>
> >>>> We're trying to depend only on Debian packages for anything running on
> >>>> the server.  This is not an absolute requirement, and we might not
> keep
> >>>> it up anyway if we ever switch to a Shiny server.  But for the moment
> it
> >>>> would be great if we could keep this requirement.  Or at least we
> >>>> shouldn't give up on it too easily.
> >>>>
> >>>> Do you think you can change the code to avoid `library(readr)`?
> >>>>
> >>>> Thanks!
> >>>>
> >>>> All the best,
> >>>> Karsten
> >>>>
> >>>>
> >>>>> On Wed, May 24, 2017 at 7:57 PM, David Fifield <
> david at bamsoftware.com>
> >>>>> wrote:
> >>>>>
> >>>>>> On Wed, May 24, 2017 at 07:28:39PM -0400, John Williams wrote:
> >>>>>>> Thanks, Karsten, but the direct links to CSV files are not working
> -
> >>>> get
> >>>>>>> Oops! Something went wrong here! We encountered a 404 Not Found
> when
> >>>>>> processing
> >>>>>>> your request!
> >>>>>>
> >>>>>> The URLs were missing a "/stats".
> >>>>>>
> >>>>>> https://metrics.torproject.org/stats/servers.csv
> >>>>>> https://metrics.torproject.org/stats/bandwidth.csv
> >>>>>> https://metrics.torproject.org/stats/torperf-1.1.csv
> >>>>>> https://metrics.torproject.org/stats/connbidirect2.csv
> >>>>>> https://metrics.torproject.org/stats/advbwdist.csv
> >>>>>> https://metrics.torproject.org/stats/hidserv.csv
> >>>>>> https://metrics.torproject.org/stats/webstats.csv
> >>>>>>
> >>>>>
> >>>>
> >>>>
> >>>>
> >>>
> >>
> >>
> >>
> >
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.torproject.org/pipermail/metrics-team/attachments/20170601/efa60beb/attachment.html>