[tor-bugs] #17939 [Onionoo]: Optimise the construction of details documents with field constraints

[Tor Bug Tracker & Wiki]

#17939: Optimise the construction of details documents with field constraints
-------------------------+---------------------
 Reporter:  fmap         |          Owner:
     Type:  enhancement  |         Status:  new
 Priority:  Low          |      Milestone:
Component:  Onionoo      |        Version:
 Severity:  Minor        |     Resolution:
 Keywords:               |  Actual Points:
Parent ID:               |         Points:
  Sponsor:               |
-------------------------+---------------------

Comment (by karsten):

 Thanks for starting this discussion.  I'm in favor of taking Gson out of
 the loop for two reasons: it's a potential performance bottleneck (though
 I never measured that), and it's a maintenance nightmare because it's just
 to easy to miss a new details document field in that hacked part of the
 code.

 Regarding the approach, I'd favor one that doesn't require keeping
 anything new in memory but instead process details document contents on
 the fly.  We'll have to read a details document from disk if we want to
 include part of it in a response anyway, and once it's in memory it's
 cheap to create such an index where fields start and end and only pick the
 ones we want.  It could just be that Gson adds some overhead that we could
 avoid here.  And of course the current approach has the downside of being
 hard to maintain, which we could fix.  Maybe we can try out different
 approaches and compare them with respect to performance and robustness?

 Bonus points: we could use this new approach to allow the `fields`
 parameter for other documents than details documents.

--
Ticket URL: <https://trac.torproject.org/projects/tor/ticket/17939#comment:3>
Tor Bug Tracker & Wiki <https://trac.torproject.org/>
The Tor Project: anonymity online