[tor-bugs] #2687 [Torperf]: Update filter.R to parse Torperf's new .mergedata format

[Tor Bug Tracker & Wiki]

#2687: Update filter.R to parse Torperf's new .mergedata format
-------------------------+--------------------------------------------------
 Reporter:  karsten      |          Owner:  karsten     
     Type:  enhancement  |         Status:  needs_review
 Priority:  major        |      Milestone:              
Component:  Torperf      |        Version:              
 Keywords:               |         Parent:              
   Points:  4            |   Actualpoints:              
-------------------------+--------------------------------------------------

Comment(by tomb):

 I broke karsten's and my code down into it's barest essentials.

 Even with everything but the actual parsing and output removed they are
 far too slow.

 I ran R's native profiling to find out what they were spending their time
 on, and found it to be string manipulation and output.

 I found no significant difference between aggregation of output in data
 frame, vector, or immediate appending to the native output buffer.  I went
 through about a dozen possible implementations and found none of them to
 be more than a small constant factor different in run time.

 I profiled memory consumption and found that all the versions I
 experimented with had modest O(n) memory consumption.

 Tentative conclusion:  R is ill suited to significant string manipulation
 Tentative recommendation: Let R crunch numbers and stats, but do the
 string manipulation in a different language.  Why not move the string
 manipulation into the programs that provide the .data and .mergedata?

-- 
Ticket URL: <https://trac.torproject.org/projects/tor/ticket/2687#comment:18>
Tor Bug Tracker & Wiki <https://trac.torproject.org/>
The Tor Project: anonymity online