[tor-bugs] #21693 [Core Tor/Tor]: prop224: HS descriptors do wasteful double-base64 encoding

[Tor Bug Tracker & Wiki]

#21693: prop224: HS descriptors do wasteful double-base64 encoding
-----------------------------+------------------------------------
 Reporter:  asn              |          Owner:  asn
     Type:  task             |         Status:  assigned
 Priority:  Medium           |      Milestone:  Tor: 0.3.1.x-final
Component:  Core Tor/Tor     |        Version:
 Severity:  Normal           |     Resolution:
 Keywords:  tor-hs, prop224  |  Actual Points:
Parent ID:                   |         Points:  4
 Reviewer:                   |        Sponsor:  SponsorR-can
-----------------------------+------------------------------------

Comment (by dgoulet):

 That scheme looks sane.

 I'm curious on how much we'll win here to be honest. `inner_layer` is a
 part we can save a bit of space but it will be from ~3k to ~2k... (most
 common case). And then `client_auth_data` is pretty small (16 lines of
 small amount of text) so maybe 1k to 800B?

 If HSDir were serving descriptor compressed that is `gzip(outer-layer)`,
 we would go from 14k to ~11k for a client to download. (Same for uploading
 a desc for a service). So I'm guessing we can bring that 11k to maybe ~10k
 with compressing more inside the descriptor (inner-layer)?

 The above implies two things. First, we need to change how HSDir are
 serving descriptors which is fine because we could just make that "if
 request comes in with URL.z, send it compressed". And we make client fetch
 it that way by default. Second, we need to implement the
 compression/encryption changes for the descriptor encoding/decoding in
 hs_descriptor.c which is a bit involving considering testing (maybe a day
 of work).

 That being said and to be honest, I'm slightly unconvinced that all this
 is needed. Going between ~10k to ~14k for the common case, it is very
 little for something you "should" download every ~18h-24h. Nathan in AMS
 was telling me that the concern with mobile for instance is not much the
 bandwidth nowadays but rather the battery life which we are going to kill
 more with multiple gzip instead of download extra 3k *I think*.
 (speculation)

 I'm missing some data here on how much going from ~3k (legacy code) to
 ~14k will affect people around the world... :S

--
Ticket URL: <https://trac.torproject.org/projects/tor/ticket/21693#comment:10>
Tor Bug Tracker & Wiki <https://trac.torproject.org/>
The Tor Project: anonymity online