[tor-dev] onionoo details document deterministic output

[Karsten Loesing]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 25/04/15 19:58, nusenu wrote:
> Karsten Loesing:
>>>> That timestamp is updated as last step of the hourly update
>>>>>> process. The details documents that you're fetching may 
>>>>>> have been updated before.  That would also explain some 
>>>>>> differences.
>>>> 
>>>> Wouldn't it make sense to mention/use the timestamp of the 
>>>> consensus that has been used to generate the output instead 
>>>> then?
>> It *is* the timestamp of the consensus that has been used to 
>> generate the output.  But generating the documents that go into
>> the output is not an atomic step.  It's an hourly cronjob that
>> runs for 15--30 minutes and writes documents for all relays to
>> disk, and only after that is done, the relays-published timestamp
>> is updated on disk.  As I'm saying on one of the tickets, one way
>> to change this would be to use a database and update all
>> documents in a single transaction, but that's a major change to
>> the current design.  Which doesn't mean we shouldn't do it, but
>> it's not trivial, and maybe it's not the most pressing missing
>> feature.
> 
> Would it be a quick and dirty fix to state the timestamp for every 
> record separately (to become "more atomic") or does that not fix 
> anything/is not possible? (I have no onionoo insides) "fix" in
> terms of: a record with a given timestamp should not change over
> time and multiple instances would provide the same record for a 
> given timestamp.

You mean instead of:

{"version":"2.3",
"relays_published":"2015-04-25 18:00:00",
"relays":[
{"n":"shadowmourne","f":"1F515F1D420B498D9687658F4A3D176F88DD4910","a":["91.219.236.218","80.255.11.213"],"r":true},
{"n":"StinTheHuman","f":"7C05C5D24577CA4C3A904470AE5526C32290FCF0","a":["108.216.89.93"],"r":true}
],
"bridges_published":"2015-04-25 17:52:43",
"bridges":[
]}

something like this (note the "u" field):

{"version":"2.3",
"relays_published":"2015-04-25 18:00:00",
"relays":[
{"n":"shadowmourne","f":"1F515F1D420B498D9687658F4A3D176F88DD4910","a":["91.219.236.218","80.255.11.213"],"r":true,"u":"2014-05-25
18:39:15"},
{"n":"StinTheHuman","f":"7C05C5D24577CA4C3A904470AE5526C32290FCF0","a":["108.216.89.93"],"r":true,"u":"2014-05-25
18:32:50"}
],
"bridges_published":"2015-04-25 17:52:43",
"bridges":[
]}

Yes, that would be possible, but it would not fix the
`If-Modified-Since` problem.

However, I this is probably just a minor problem for most users.  It's
still a bug, but nothing too serious.  (Your use case of finding
differences between two Onionoo servers is probably the exception there.)

All the best,
Karsten
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1
Comment: GPGTools - http://gpgtools.org

iQEcBAEBAgAGBQJVO+8tAAoJEJD5dJfVqbCr54kH/i7MrhlXx1xZqSG8BgAd3EG+
5InPtXMyblDRYGh+Ded5uFbLVQRmWnQXISwwJ/IWsvMiGrepraFLoNv0ltl2hkN0
GXwemeI2ya0OKh068vCvSsf/H6EaFoukUo76Eb3IKA10bzXZnfu3hD2BUeEPj/Pv
K5JC/YBBDKBCVKZaiZKPMKfX6lpRsZNoy/iSBkrQMqLZIPuKiFRJ2Qf++SgC/BE7
u52X8rDvnAz3KrrQLecMruDnXi+cIqLuMIX2dJYBtugWpOd2/WO1MxW8ufQy8K2e
7UXL8SEu8jX51EFV2Vf3byS6pVuUyFhYVMrA7+4TBdqeZf1y+ry4UxQgwiaUkXg=
=QGPS
-----END PGP SIGNATURE-----