[tor-dev] Get Stem and zoossh to talk to each other

[tordev123 at Safe-mail.net]

> > zoossh's test framework says that it takes 36364357 nanoseconds to
> > lazily parse a consensus that is cached in memory (to eliminate the I/O
> > bottleneck).  That amounts to approximately 27 consensuses a second.
> >
> > I used the following simple Python script to get a similar number for
> > [...]
> > This script manages to parse 24 consensus files in ~13 seconds, which
> > amounts to 1.8 consensuses a second.  Let me know if there's a more
> > efficient way to do this in Stem.
> 
> Interesting! First thought is 'wonder if zoossh is even reading the
> file content'. Couple quick things to try are...
> 
> with open(file_name) as consensus_file:
>   consensus_file.read()
> 
> ... to see how much time is disk IO verses parsing. Second is to try
> doing something practical (say, count the number of relays with the
> exit flag). Stem does some bytes => unicode normalization which might
> account for some difference but other than that I'm at a loss for what
> would be taking the time.

Why are you surprised? Python is a very, very slow language. My very simple unoptimized C++ parsing code is more than 10x faster than Stem (parsing from memory). Go has some nice string parsing features, so zoossh should have an easy time to get fast and simple parsing code.