[tor-bugs] #25667 [Core Tor/Tor]: LZMA/ZSTD descriptor compression support

[Tor Bug Tracker & Wiki]

#25667: LZMA/ZSTD descriptor compression support
-------------------------------------------------+-------------------------
 Reporter:  atagar                               |          Owner:  (none)
     Type:  defect                               |         Status:  new
 Priority:  Medium                               |      Milestone:  Tor:
                                                 |  0.3.3.x-final
Component:  Core Tor/Tor                         |        Version:
 Severity:  Normal                               |     Resolution:
 Keywords:  regression? 033-must needs-analysis  |  Actual Points:
Parent ID:                                       |         Points:
 Reviewer:                                       |        Sponsor:
-------------------------------------------------+-------------------------

Comment (by teor):

 > The spec should say how we pick among the compression. That is to say,
 lzma > zstd > gzip > identity (or whatever the actual behavior is).

 This is the required behaviour::

 Relays MUST compress all directory documents with gzip. If lzma or zstd
 are available, each compression method MAY be used to compress some types
 of directory documents.

 This is how the current implementation works, but clients shouldn't rely
 on the exact details:

 Typically, compression methods are used for the documents that provide the
 best compression/CPU/RAM tradeoffs. Some compression methods are used to
 compress long-lived documents, then those documents are cached. Other
 methods are used for streaming compression when documents are requested.

 This is the required behaviour:

 When a directory mirror receives a request for compressed data, it MUST
 serve a format that is available and supported by the client.

 This is how the current implementation works, but clients shouldn't rely
 on the exact details:

 If multiple methods are supported, tor chooses the compression method in
 this order:

 Client requests contain all supported methods in this order:
 lzma > zstd > zlib > gzip > identity
 https://gitweb.torproject.org/tor.git/tree/src/or/directory.c#n3627

 Directory responses choose a common, supported method in this order:
 Precompressed, cached data: lzma > zstd > zlib > gzip > identity
 Streamed data: zstd > zlib > gzip > identity
 Anonymous connections (e.g. HSDirs): zlib > gzip > identity
 https://gitweb.torproject.org/tor.git/tree/src/or/directory.c#n3575

--
Ticket URL: <https://trac.torproject.org/projects/tor/ticket/25667#comment:4>
Tor Bug Tracker & Wiki <https://trac.torproject.org/>
The Tor Project: anonymity online