[tor-dev] HSDir Auth and onion descriptor scraping

[grarpamp]

On Sun, Nov 9, 2014 at 3:22 PM, Gareth Owen <gareth.owen at port.ac.uk> wrote:
> I have several hundred thousand (or million? Haven't counted) hs descriptors
> saved on my hard disk from a data collection experiment (from 70k HSes).
> I'm a bit nervous about sharing these en masse as whilst not confidential
> they're supposed to be difficult to obtain in this quantity.  However, if
> someone wants to write a quick script that goes through all of them and
> counts the number of authenticated vs nonauthed then I do not mind running
> it on the dataset and publishing the results.  I have a directory where each
> file is a hs descriptor.
>
> The introduction point data is base64 encoded plaibtext when unauthed or has
> high entropy otherwise.

What version descriptors are you collecting?

There are a few reports I could think to run against your dataset, even if
the IntroPoints were replaced with 127.0.0.n (n set to 1, 2, 3, n for each
IntroPoint in respective descriptors list)... or even 1:1 mapped for all
descriptors either a) randomly into a new parallel IPv4/IPv6 space (dot-quad),
or b) serially into a respective 32 or 128 bit number (not dot-quad).

Whether on or off list I could use your collection patches, and a raw
sample of a single recent on disk descriptor from a public service such as
hbjw7wjeoltskhol or kpvz7ki2v5agwt35 so we know your data format.

It's effectively public info anyways, I'll get to it sooner or later, others
already have.