[tor-dev] TorBEL migration

[Sebastian Hahn]

On Apr 30, 2013, at 3:17 AM, Sreenatha Bhatlapenumarthi <sreenatha.dev at gmail.com> wrote:
> Hi,
> 
> My name is Sreenatha Bhatlapenumarthi. I'd like to migrate TorBEL 
> from TorCtl to Stem as a part of GSoC'13 project this summer.

Hi Sreenatha,

great to hear from you!

> Damian pointed out that there is still some improvement to be made to
> TorBEL to replace DNSEL. I've downloaded the source code from [0] and 
> started looking around for missing functionality and components that use 
> TorCtl(and hence need migration) and here is what I've observed:
> 	• TorBEL components that use TorCtl
> 		• Controller - control.py
> 		• Logger - logger.py
> 		• Router - router.py
> 		• Tests - tests.py
> 		• The main interface - torbel
> 	• Exit policies of narrow_exit routers can be checked in Router.will_exit_to[1]
> 	• RouterRecord.update_to()[2] can use "Router Status Entry" descriptors[3] to update the router's attributes
> 	• I think the following line[4] in network.py has a bug - socket.inet_aton works only for IPv4 addresses[5].
> 		•  (ip,) = struct.unpack(">I", socket.inet_aton(peer.host))

Quite possibly! ipv6 support in Tor was far away when that code
was written.

> Please let me know if I've missed something or got something wrong.

You might want to check out the TorBEL trac tickets [6] if you haven't,
to get some further ideas about things that need doing.

> I donot have any background on Haskell as such but if you recommend that I
> learn it inorder to understand TorDNSEL, I am prepared to spend the required
> amount of time learning it during May before the coding period starts. Also, can 
> you give me a rough estimate of how long you think the migration will take so that 
> I can plan the project timeline accordingly?

Learning Haskell will be much less valuable than investing the time to
learn what's needed from a dnsel replacement. Also note that unless
you have experience with functional programming languages, Haskell will
not be "just another language", but rather you'd first spend some time
wrapping your head around functional programming.

As for the timeline, that's a bit tricky. Migrating to stem will
probably require a fair deal of refactoring, as the current
implementation makes heavy use of TorCtl internals, and especially the
Router class is tailored to TorCtl. Damian might have some input here
on how much TorCtl and stem differ here, and what (some of) the
architectural differences will be. That said, if you invest some time
before the actual coding period to get up to speed on what the tool
is actually supposed to do, you have a great shot at porting in 6
weeks. Three weeks learning stem and porting, 3 weeks testing and
fixing issues you'd encounter along the way, I'd assume. The rest of
the summer could be spent on improving stem, or on implementing
additional functionality. All this depends on your Python skills tho,
of course. Ultimately, it's  your job to figure out and propose a
timeline (but don't worry, it's not like that'll be set into stone
the moment the program starts).

> Thanks for taking your time out to read this.

My pleasure, thanks for your interest in TorBEL!

> Cheers,
> Sreenatha

All the best
Sebastian

[6]: https://trac.torproject.org/projects/tor/query?status=accepted&status=assigned&status=needs_information&status=needs_review&status=needs_revision&status=new&status=reopened&component=TorDNSEL%2FTorBEL&order=priority