[tor-dev] onionoo: historic details.json data

[Karsten Loesing]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 07/04/15 17:39, Nusenu wrote:
>> On 05/04/15 23:35, Nusenu wrote:
>>>> can one download historic onionoo documents (details.json) 
>>>> archived somewhere or would one have to setup onionoo + feed 
>>>> old data into it to achieve that?
>> There are no archives of Onionoo's documents.
> 
>> But of course there are CollecTor's archives which you'd feed
>> into Onionoo.  Though it's very likely easier to parse those
>> directly (possibly using Stem) rather than setting up an Onionoo
>> instance for the exact time you're interested in.
> 
>> Speaking of, what historic data are you looking for?  Maybe it's
>>  something that we should add to Onionoo itself?
> 
> I've a few use case for onionoo data, one of them uses onionoo to
> find groups of relays run by one entity. First_seen combined with 
> last_restarted has proven to be a rather good datapoint for that. 
> Using an array of all restarts (instead of just one) would likely 
> reduce false-positives even more.

So, the problem here is that an array of restarts doesn't scale, so it
would have to be limited to the last 10 restarts or so.  But even that
is not trivial to implement in Onionoo and, as you note below, it's
quite specific and not very useful for the average Onionoo client.

What you could try is evaluate uptime documents and see if two relays
had similar uptime patterns over time:

https://onionoo.torproject.org/protocol.html#uptime

> That is one field but I'll consider (historic) changes to other
> fields (contactinfo, orport, dirport, ...) as well.
> 
> Although I could imagine atlas displaying data like "these were 
> previously provided contact details: ..." most of my use cases are
> to uncommon to add to onionoo.

I think I agree with you here.  Also, you're much more flexible by
using descriptors directly and adapting what you're extracting from
them rather than having to wait for me to add a new field to an
Onionoo document.

By the way, are you aware of Philipp Winter's work on a better Sybil
attack detector?  He's cc'ed in case you want to brainstorm about good
criteria for comparing relays.

All the best,
Karsten
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1
Comment: GPGTools - http://gpgtools.org

iQEcBAEBAgAGBQJVK3uSAAoJEJD5dJfVqbCr3uoH/2WyEtrm6af/1LTZ2cBW9CgD
wWoDpo17zRXse4A17IRYaD2P+8yaKHRS2KDnfewCmNoW52JHUWEvzkODMSEmLjTf
l6sEWo3gvYcmp6TGVMpcjTtYcXlj093MzD749p3dDglkB/HkJjamO1bKfzzbRw51
Z00s7aN1bCbdjTYenreJreAHUeUNGWD+c7y8VG4tKU12TbZq8AZD1x5B3jPfg1n7
cQEvSvpr4Y7SHgPkndwtwliBsg51B+oALQk2wEmh1neVDEBG8Rb8i4hLjZB84DsD
eMCtteEUVm0TinIoKNO3uGv+hk1Ls5MQF906JeB3ctrwPIbZcPRCwolGLUgmXkA=
=xUvB
-----END PGP SIGNATURE-----