[tor-dev] Relay Database: Existing Schemas?

[Karsten Loesing]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 15/04/15 21:18, nusenu wrote:
> Hi,
> 
> I'm planing to store relay data in a database for analysis. I
> assume others have done so as well, so before going ahead and 
> designing a db schema I'd like to make sure I didn't miss
> pre-existing db schemas one could build on.
> 
> Data to be stored: - (most) descriptor fields - everything that
> onionoo provides in a details record (geoip, asn, rdns, tordnsel,
> cw, ...) - historic records
> 
> I didn't find something matching so far, so I'll go ahead, but if
> you know of other existing relay db schemas I'd like to hear about
> it.
> 
> thanks, nusenu
> 
> 
> 
> 
> "GSoC2013: Searchable Tor descriptor archive" (Kostas Jakeliunas) 
> https://www.google-melange.com/gsoc/project/details/google/gsoc2013/wfn/
>
> 
5866452879933440
> https://lists.torproject.org/pipermail/tor-dev/2013-May/004923.html
>
> 
https://lists.torproject.org/pipermail/tor-dev/2013-September/005357.htm
> l https://github.com/wfn/torsearch (btw, someone knows the license
> of this?)

Cc'ing Kostas for this question.

>> This is true: the summary/details documents (just like in Onionoo
>>  proper) deal with the *last* known info about relays.
> 
> 
> ernie 
> https://gitweb.torproject.org/metrics-db.git/plain/doc/manual.pdf 
> (didn't find db/tordir.sql mentioned in the pdf)

That file lives here now:

https://gitweb.torproject.org/metrics-web.git/tree/modules/legacy/db/tordir.sql

A better schema might be the following one though.  It's smaller, but
it's better documented:

https://gitweb.torproject.org/exonerator.git/tree/db/exonerator.sql

> "Instructions for setting up relay descriptor database" 
> https://lists.torproject.org/pipermail/tor-dev/2010-March/001783.html

That's
> 
five years old.  I'd say ignore that one.

> "Set up descriptor database for other researchers" 
> https://trac.torproject.org/projects/tor/ticket/1643

Also five years old.  Better ignore.

Hope that helps.

All the best,
Karsten
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1
Comment: GPGTools - http://gpgtools.org

iQEcBAEBAgAGBQJVL9rcAAoJEJD5dJfVqbCrFZgIAIEv/Yi4sNoa8clYVAxuk0Sh
FFbRDT0kLs19t/DgTwUtB6jD4Lh0akMc806AaIFgfCdL+QwcG0llBfZnSsrbszoH
Xoi226PRx9lPITrA7KYds4PUZfqIqg3ECpNsKNa4PLB7SlQdNfJQ1wDngcwu2CrF
Hk+zHbu0gfSkfZRBqxt5aJLTFXR0aBYybF4d6sPJ4OW5Al2U8r9DYysXc0xALvwq
bvEDFctV1wkDgA3mP3guRrXImXYT1AQPFFlz0TR1eBruuSJBiPKIv7Fs/ocns4aR
OhxIEaKBaAO+HkvyxDcZ1ukXldR13s3MUPD0XvvZ8xQRCBZpNMygqTMi6pIjTN4=
=a0Nb
-----END PGP SIGNATURE-----