[metrics-bugs] #21588 [Metrics/Statistics]: Rewrite the censorship detector used by the Tor Metrics website in Java

[Tor Bug Tracker & Wiki]

#21588: Rewrite the censorship detector used by the Tor Metrics website in Java
--------------------------------+--------------------------
 Reporter:  karsten             |          Owner:  karsten
     Type:  enhancement         |         Status:  assigned
 Priority:  Medium              |      Milestone:
Component:  Metrics/Statistics  |        Version:
 Severity:  Normal              |     Resolution:
 Keywords:  metrics-2018        |  Actual Points:
Parent ID:                      |         Points:
 Reviewer:                      |        Sponsor:
--------------------------------+--------------------------

Comment (by karsten):

 Replying to [comment:10 iwakeh]:
 > Replying to [comment:9 karsten]:
 > > Hmm, okay, that sounds like we need to choose the least bad solution.
 > >
 > > If we go for calling scipy from Java, let's at least reduce that to a
 single call and not create a Python process for each `norm.fit` call. We
 could write all input data to a file, call Python to fit normal
 distributions and write an output file, and read that in Java. Not pretty,
 but that at least reduces overhead a bit.
 >
 > Yep, let's be pragmatic.  I would still create the new package for the
 calls in case a different solution comes up.

 Okay. Can we design this package in a way that math calls can either be
 executed directly, or computed in a batch fashion? Because, for scipy, I
 think we should really go for batch execution, whereas future solutions
 might not require that.

 > > Are there really no native Java libraries that do the same thing?
 >
 > Fortran is well suited for scientific computations, why re-implement?
 Apache commons does re-implement  one algorithm by minpack in a simplified
 way and w/o any means to judge the quality.  The algorithm they provide
 doesn't give the good results scipy's choice of minpack algorithm
 delivers.  I didn't encounter any free Java implementation that would
 provide the functionality.  Maybe, Apache commons math will one day
 provide all the optimization algorithms.

 Okay. Let's be pragmatic and assume it won't happen.

--
Ticket URL: <https://trac.torproject.org/projects/tor/ticket/21588#comment:12>
Tor Bug Tracker & Wiki <https://trac.torproject.org/>
The Tor Project: anonymity online