[tor-dev] OnionOO protocol questions

[Karsten Loesing]

Hi Norman,

On 8/3/12 9:42 PM, Norman Danner wrote:
> If I understand the weights documents correctly, there is no one
> consensus weight for a given router.  What exactly is the ordering we
> should use when the request asks to order by consensus weight?
> 
> It seems like determining the consensus weights of the selected relays
> could be a relatively slow process, because at least with the
> filesystem-backed approach we are using right now, each individual
> weights file has to be opened and read.  Is this correct, or am I
> missing something here?

Ordering by consensus weight (or any other field) is independent of the
returned document type, like weights documents.  The fact that weights
documents contain consensus weight histories doesn't change that.  You'd
order by the latest known consensus weight of a relay.  You don't have
to open and parse all the weights files for that, which would be a
performance disaster.

The way how the current Java Onionoo works is that we do the filtering
and ordering only based on the summary helper file.  The result is an
ordered list of fingerprints.  Only in the last step we look at the
requested document type and fetch documents from disk by fingerprint.

The Python Onionoo should probably do the same thing but using a
database.  There would be a single database table that can be used to
filter and sort results.  Those results can then be joined with document
tables by fingerprint to fetch all the JSON strings to return.

Best,
Karsten