[tor-dev] New Fallback Directory File Format

[teor]

> On 24 Dec 2017, at 08:16, Damian Johnson <atagar at torproject.org> wrote:
> 
> Hi Tim, added preliminary Stem parsing support for the v2 fallback format...
> 
>  https://gitweb.torproject.org/stem.git/commit/?id=ea55eaa
> 
> Few thoughts are...
> 
>  * It would be nice if the 'extrainfo=' lines were mandatory since I
> need a delimiter between the entries.

We have a few alternatives here:

C requires a comma as the delimiter between entries.
I can guarantee there will always be a comma delimiter after every entry,
Including the last entry. I will also make sure that any comment fields
come before this delimiter. I can guarantee there will never be a comma
inside the C string or comments in the entry.

Alternately, I can make extrainfo mandatory, and if I can't fetch a relay's
descriptor (a rare case?), I will mark is as 0.

But I'd like to be able to add extra fields in future without breaking
parsers, so I don't want parsers relying on field order.

Do you want me to add an explicit end of record comment, or is the
comma sufficient?

Is there a delimiter you'd like me to add before the first entry?

>  * Maybe we should start the document with a format version number?

I thought of that, too. Thanks for reminding me.

https://trac.torproject.org/projects/tor/ticket/24725

> Cheers! -Damian
> 
> 
> 
>> On Fri, Dec 22, 2017 at 6:53 AM, teor <teor2345 at gmail.com> wrote:
>> Hi all,
>> 
>> Some time in the next few weeks, the Tor fallback directory mirror file
>> format will change. This affects stem and Relay Search, which parse this
>> file.
>> 
>> 
>> Change Description
>> 
>> Here is a list of changes to the file format:
>> * the "weight" line has been removed, and replaced with a Tor config
>>  default (#24679, #24681)
>> * the comma that separates fallback C strings is now on its own line
>> * a "nickname" comment has been added (#24600)
>> * an optional "extrainfo" comment has been added (#22759)
>> 
>> The added fields will be populated with placeholders until the list is
>> rebuilt (#22271). This will hopefully happen some time in the next few
>> weeks.
>> 
>> 
>> Requesting More Extra Info Caches
>> 
>> There are only a few fallbacks that cache extra-info documents.
>> I checked 67, and only 4 cached extra-info documents.
>> 
>> atagar, do you want me to ask some fallback operators to set
>> DownloadExtraInfo 1?
>> 
>> What number or proportion would you like?
>> 
>> (We allow approximately 25% of fallbacks to go down before we start to rebuild
>> the list. In the worst case, this can mean that ~40% are down at some point.)
>> 
>> 
>> Example Entries
>> 
>> A sample entry in the new format, using actual relay info:
>> 
>> "5.9.110.236:9030 orport=9001 id=0756B7CD4DFC8182BE23143FAC0642F515182CEB"
>> " ipv6=[2a01:4f8:162:51e2::2]:9001"
>> /* nickname=rueckgrat */
>> /* extrainfo=1 */
>> ,
>> 
>> The current fallback file in the new format, with placeholders:
>> https://github.com/teor2345/tor/blob/ticket22759_tree/src/or/fallback_dirs.inc
>> 
>> A small sample fallback file in the new format, with actual relay info:
>> https://trac.torproject.org/projects/tor/attachment/ticket/22759/fallback_dirs_new_format.inc
>> 
>> Please let me know if you would like any changes to the format.
>> 
>> T
>> 
>> --
>> Tim Wilson-Brown (teor)
>> 
>> teor2345 at gmail dot com
>> PGP C855 6CED 5D90 A0C5 29F6 4D43 450C BA7F 968F 094B
>> ricochet:ekmygaiu4rzgsk6n
>> xmpp: teor at torproject dot org
>> ------------------------------------------------------------------------
>> 
>> 
>> 
> _______________________________________________
> tor-dev mailing list
> tor-dev at lists.torproject.org
> https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.torproject.org/pipermail/tor-dev/attachments/20171224/c1140917/attachment-0001.html>