[tor-project] Intent to Minimise Effort: Fallback Directory Mirrors

Tue Jan 9 02:56:25 UTC 2018

Hi all,

I'd like to move this thread to the tor-dev list. I made a list of
Core Tor tickets at the end of this email. Let's talk about them
on tor-dev.

Here's some background:

Rebuilding the fallback list takes a lot of effort!

So I'm picking up this tor-project thread while the latest fallback rebuild
is fresh in my mind.

The rebuild took a day or two of my time for regular changes.
And another day or two for once-off changes.

We made a few changes to streamline the process:
* atagar and I wrote scripts that make it easier to generate fallback lines,
  and get operator contact details [0]
* pastly, atagar and irl helped out with code reviews and list format changes,
  which should make the list easier to read and parse
* pastly and nickm helped out with fallback list updates and backport

Having this help meant that I had time to make the process better.
Thanks!

But there's still more we can do:

> On 29 Aug 2017, at 11:12, teor <teor2345 at gmail.com> wrote:
> 
>> On 28 Aug 2017, at 18:15, Linus Nordberg <linus at torproject.org> wrote:
>> 
>> teor <teor2345 at gmail.com> wrote
>> Sat, 26 Aug 2017 22:39:52 +1000:
>> 
>>> So I can't see any reasons to keep running opt-ins.
>> 
>> Are we listing non-exits?
> 
> Yes. Non-exits and under-used exits are the best fallback directory
> mirrors, because they are under less load.
> 
>> I suppose those might see a radical change in
>> their "IP reputation" when they end up on the blocklists after the next
>> ransomware attack. Exits are already in the lists.
> 
> Yes, there is a higher risk that a fallback will end up on one of these
> lists. But all public Tor relays are at risk of ending up on a list like
> this.

Because of the higher risk, we will continue to ask relay operators to
opt-in [1]. So we need to find people to help with this process every
6-12 months. And make it more efficient.

>> On 28 Aug 2017, at 19:34, Silvia [Hiro] <hiro at torproject.org> wrote:
>> 
>> On 26/08/17 15:44, nusenu wrote:
>>>>> use ContactInfo
>>>>> as a way to collect the intention of relay operators to opt-in/opt-out
>>>>> of the fallback directory mirrors list.
>>>> This is a good idea, *if* we still think an opt-in is necessary.
>>> 
>>> Isn't that way to collect operator intentions also useful in opt-out
>>> mode? (instead of continuing to use emails) Or how do you plan to
>>> collect the opt-out info?
>> 
>> Why are emails preferred over other methods? Another options would be
>> wrapping a script that ops can run to create a cyberpunk trac ticket and
>> opt-out. This is probably easier than merging emails. But maybe we do
>> not want to handle this through trac.
>> 
>> I'd be happy to help w/ that eventually, and the fallback script.
> 
> This is a good idea.
> 
> At the moment, I take emails sent to me or tor-relays, and turn them
> into trac comments or trac tickets. If we put it directly on trac,
> anyone could handle the ticket.

The most time-consuming part of the rebuild is the opt-in process.

Here's what we do at the moment: [2]

0. Send an email to tor-relays to ask operators to opt-in
1. Run a script to find fallbacks whose details have changed, and email
   tor-relays to ask operators if their details are permanent

Here are the most time-consuming steps:

2a. Receive personal emails and emails to the list with fallback details
2b. Generate an updated fallback line for each relay [0]
2c. Update the whitelist

And these steps are quick:

3. Run the update script to generate a new list
4. Write a changes file
5. Announce the new list
6. Backport the list to all supported Tor versions

Here are some things I'd like to change:

Just have relay fingerprints in the fallback whitelist [3]

The fallback script checks relay details, but it can just rely on Onionoo
to say when the address last changed.

This removes step 1 and step 2b.

Add a torrc option and descriptor line to opt-in as a FallbackDir [4]

This is inspired by nusenu's "ContactInfo" idea, and Hiro's "bot" idea.
Having a descriptor line and torrc option gets us a more reliable
implementation, and it's easier for relay operators to use. And
there's no need for an email bot or trac bot.

This removes step 2a and step 2c, for tor versions with this feature.
For tor versions without this feature, we can use the simpler fingerprint
list.

Make the hard-coded authorities into a separate include file with a standard format [5]

We'll also change the fallback file format (again, sorry!) to make
them consistent. This makes it easier for stem, metrics-lib, and
other Tor implementations to use the same authorities and fallbacks.

This makes step 5 easier (for libraries that use fallbacks).

Let's discuss these changes on the tor-dev list!

T

[0]: https://gitweb.torproject.org/tor.git/tree/scripts/maint/generateFallbackDirLine.py
     https://gitweb.torproject.org/tor.git/tree/scripts/maint/lookupFallbackDirContact.py
[1]: https://trac.torproject.org/projects/tor/ticket/24789
[2]: https://trac.torproject.org/projects/tor/wiki/doc/UpdatingFallbackDirectoryMirrors
[3]: https://trac.torproject.org/projects/tor/ticket/24838
[4]: https://trac.torproject.org/projects/tor/ticket/24839
[5]: https://trac.torproject.org/projects/tor/ticket/24818

--
Tim / teor

PGP C855 6CED 5D90 A0C5 29F6 4D43 450C BA7F 968F 094B
ricochet:ekmygaiu4rzgsk6n
------------------------------------------------------------------------

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: Message signed with OpenPGP
URL: <http://lists.torproject.org/pipermail/tor-project/attachments/20180109/9a405df2/attachment-0001.sig>