[tor-dev] Using Stem's descriptor fetching module to replace the Java consensus-health checker

[Karsten Loesing]

On 8/20/13 4:41 AM, Damian Johnson wrote:
> Thanks, Karsten, for the code review! I decided to take a long weekend so
> was able to address all of it...

Hi Damian,

neat, thanks for your quick response!

Replying inline, but leaving out the parts where I'd only write "okay".

>> Yes, I agree that we should have your script start sending results to
>> the mailing list.  Want to set that up?  Not sure if your mails will
>> bounce until somebody approves your sender address, but I guess we'll
>> find out.
> 
> Nope, not getting through. Asking for my addresses to be whitelisted....
> 
> https://trac.torproject.org/projects/tor/ticket/9537

Looks like this is already solved.

>> 1. Do the authorities voting on BadExit agree about relays they assign
>> this flag to?  The warning might contain the diff of relay fingerprints
>> they voted or didn't vote BadExit on.  We'll probably want to rate-limit
>> this warning to every 6 or 12 hours.
> 
> Done...
> 
> https://gitweb.torproject.org/atagar/tor-utils.git/commitdiff/e6d378a
> 
> Btw, it looks like they presently *are* out of sync...
> 
> NOTICE: Authorities disagree about the BadExit flag for
> 7E6A3AA70A156167E7AE543E50EF54321EC80AF0 (with flag: Faravahar, without
> flag: tor26, moria1)
> NOTICE: Authorities disagree about the BadExit flag for
> ADF62D3A1305F0B5404D41EEDADA68ECD294FC60 (with flag: Faravahar, without
> flag: tor26, moria1)

Are you sure?  AFAIK, Faravahar never voted on BadExit.  Which
valid-after time was this?

>> How about we start a new thread on this list discussing only the part
>> where we're planning to kill the website and move everything to the
>> status emails?
> 
> Personally I don't see value in forking this thread, but I don't care
> strongly either.

It seems I learned some input from Peter on IRC, and Sebastian promised
some feedback via email to you.  Guess that's a fine start.  We'll
probably have a new discussion once we run both DocTors in parallel or
when we finally switch over.

>> - DocTor produces two output files containing warnings, one with new
>> warnings and one with all warnings (which is empty if there are zero new
>> warnings).  New warnings are sent to the IRC bot and all warnings are
>> sent to the mailing list.  I agree that this approach is somewhat
>> complicated, so maybe it's sufficient to just send the new warnings to
>> both the IRC bot and the mailing list.  Unless you didn't realize there
>> was such a distinction and think it's worth adding to your script.  Up
>> to you.
> 
> I've ignored the IRC bot notifications since I'm both unfamiliar with it
> and presently lack a mechanism to provide it with notifications.
> 
> I take it as if the bot periodically reads files from disk then dumps the
> contents into an IRC channel? Who maintains the bot and might we switch it
> to another notification mechanism? Maybe it should read
> tor-consensus-health@ instead?
> 
> Alternatively I can change my send() function to do whatever the bot
> maintainer wants, though dumping files to disk seems a bit odd to me.

Peter runs the IRC bot.  The interface on which it accepts input is
simply an email address.  You could send it the same content that you
send to the mailing list, or a different output.

> PS. A couple other things that need to be addressed at some point are:
> 
> * Running this on a TPO host rather than my desktop, and maybe also use a
> TPO address to send the emails too.

Sure, we can do that.  The easiest way would be that I run your script
on yatei.  Should I do that, similar to how I run DocTor on it?

We can later give you access to yatei (which you'll also need for
co-co-maintaining metrics services), or we could run the script on a
new, tiny VM.

> * Moving these scripts to a real repository rather than my 'tor-utils' user
> repo. Not sure though if we should keep the name 'DocTor' or opt for
> something more descriptive like 'DescriptorMonitors'.

I prefer repository names that aren't too descriptive, because if we
ever want to extend or narrow scope of a tool, the name might confuse
new people.  That's what happened to metrics-db which doesn't even use a
database anymore since I split off the metrics-web part.

How about we add your script to the DocTor repository?  Again, to be
quick, you could simply send me a patch produced with `git format-patch`
or tell me a repository to pull from.

And if we want to do this right, we should give you a personal
doctor.git and push rights to the official doctor.git.

Of course, if your preference is to start a new repository, I'm fine
with that, too.


Thanks for spending your long weekend on this!

All the best,
Karsten