[tor-bugs] #28877 [Core Tor/Tor]: Paginate large controller commands like 'GETINFO desc/all-recent'

[Tor Bug Tracker & Wiki]

#28877: Paginate large controller commands like 'GETINFO desc/all-recent'
--------------------------+--------------------------
 Reporter:  wagon         |          Owner:  atagar
     Type:  defect        |         Status:  assigned
 Priority:  Medium        |      Milestone:
Component:  Core Tor/Tor  |        Version:
 Severity:  Normal        |     Resolution:
 Keywords:                |  Actual Points:
Parent ID:                |         Points:
 Reviewer:                |        Sponsor:
--------------------------+--------------------------

Comment (by wagon):

 > The usual way of dealing with this is to have a paginated API where the
 caller can provide a starting index and number of responses. This would
 let Nyx fetch smaller batches without hosing the control connection.
 I'm not sure I understand what you mean. There are other commands which
 return descriptors by id or by name like `desc/id/FINGERPRINT`, output of
 these commands is small. However, a purpose of `desc/all-recent` is to get
 the full content of the file with descriptors, so how it can be
 "paginated"? Do you mean some internal TCP transport level?

 > Unfortunately without that there's not much I can do on my end.
 Maybe you can. I started from finding a source of this problem. Initially
 I thought it may be a Tor side. I checked
 [[https://stem.torproject.org/faq.html#i-m-using-cookie-
 authentication|your instructions]] for `telnet`, but `telnet` works fine
 with `desc/all-recent` when invoked from interactive shell. It returns the
 whole output without any problem. The same is true for `netcat` and
 `socat` if they are used to connect to `ControlPort`.

 However, if any of these tools are not used interactively, but are called
 inside some wrappers or functions, they all fail. They return some part of
 `desc/all-recent` output (each time it is a different part) after which
 the connection hangs. I looked for a way to solve this problem and found
 it. Probably, so powerful language as python also has some way to solve
 this problem.

--
Ticket URL: <https://trac.torproject.org/projects/tor/ticket/28877#comment:2>
Tor Bug Tracker & Wiki <https://trac.torproject.org/>
The Tor Project: anonymity online