[tor-dev] grabbing Tor circuit (node) data- Tor stem, torrc and Tor control port

Tue Jun 11 12:32:55 UTC 2013

Damian, thanks. Your summary pretty well sums up where i am trying to start. I will try this out and go from there. 

On Jun 10, 2013, at 11:08 PM, Damian Johnson <atagar at torproject.org> wrote:

> Hi Sarah. I'm not really sure what you're trying to ask in most of
> these questions. Assuming that your goal is simply 'I want to connect
> to port 9051 and dump information about all the relays I'm presently
> using in my circuits' then the following should do the trick.
> 
> Take the following with a grain of salt - I just wrote it, I haven't
> tried running it. Should be close though. ;)
> 
> from stem.control import Controller
> 
> with Controller.from_port(port = 9051) as controller:
>  my_relay_fingerprints = []  # fingerprints of all relays in our circuits
> 
>  for circ in controller.get_circuits():
>    my_relay_fingerprints += [fp for (fp, nickname) in circ.path]
> 
>  for fingerprint in my_relay_fingerprints:
>    desc = controller.get_network_status(fingerprint)
>    country = controller.get_info("ip-to-country/%s" % desc.address, "unknown")
> 
>    print "relay: %s:" % fingerprint
>    print "  address: %s:%s" % (desc.address, desc.or_port)
>    print "  locale: %s" % country
> 
> 
> Did you have any other questions?
> 
> On Mon, Jun 10, 2013 at 9:43 AM, SARAH CORTES <sarah at lewis.is> wrote:
>> Damian, thanks, this is very helpful.
>> 
>> is there a way to do this in torrc? Else, i suppose i will need to:
>> 
>> 1) create a socket or connection to my port 9051 ; do i need/can i use
>> TORRC_CONTROL_SOCKET ?
>> 2) call get_circuits() ; grab the relay fingerprints
>> Do I need circuit = controller.get_circuit(circ_id)
>> 3) return class:`stem.events.CircuitEvent` ; for the given circuit
>> Not sure whether or where to use the "path" attribute
>> 4) call controller.get_network_status() to get IP address, nickname, ORPort,
>> Should I use:
>> desc_by_fingerprint = controller.get_network_status(test_relay.fingerprint)
>> 
>> 	+      desc_by_nickname =
>> controller.get_network_status(test_relay.nickname)
>> 
>> 5) use Maxmind- i already have the GeoIPLite DB to grab AS and country, and
>> onion code also from Arturo
>> 
>> 
>> Any guidance is appreciated
>> 
>> 
>> https://lists.torproject.org/pipermail/tor-commits/2012-December/051174.html
>> 
>> get_circuit(self, circuit_id, default = UNDEFINED):
>> +    """
>> +    Provides a circuit presently available from tor.
>> +
>> +    :param int circuit_id: circuit to be fetched
>> +    :param object default: response if the query fails
>> +
>> +    :returns: :class:`stem.events.CircuitEvent` for the given circuit
>> +
>> +    :raises:
>> +      * :class:`stem.ControllerError` if the call fails
>> +      * ValueError if the circuit doesn't exist
>> +
>> +      An exception is only raised if we weren't provided a default
>> response.
>> +    """
>> +
>> +    try:
>> +      for circ in self.get_circuits():
>> +        if circ.id == circuit_id:
>> +          return circ
>> +
>> +      raise ValueError("Tor presently does not have a circuit with the id
>> of '%s'" % circuit_id)
>> +    except Exception, exc:
>> +      if default: return default
>> +      else: raise exc
>> +
>>   def get_circuits(self):
>>     """
>>     Provides the list of circuits Tor is currently handling.
>> 
>> 
>> 
>> On Jun 10, 2013, at 10:34 AM, Damian Johnson <atagar at torproject.org> wrote:
>> 
>> Hi, Damian, thanks. I am happy to discuss it on tor-dev at . But I want to keep
>> off spam, which some of my questions at first may be, essentially Qs. But,
>> if you think they would be of interest to tor-dev, or others could help,
>> just let me know, and i will sign up for it.
>> 
>> 
>> They certainly are! If you're interested in tor and development then I
>> would definitely suggest being on that list. Including it for this
>> thread.
>> 
>> I am trying to figure out how to pull in the nodes that are actually used in
>> my Tor circuits. They are the nodes reflected in the Network map function.
>> 
>> 
>> You want the get_circuits() method. As you mentioned the 'path'
>> attribute has the relays in your present circuits...
>> 
>> https://stem.torproject.org/api/control.html#stem.control.Controller.get_circuits
>> 
>> I have created a MySql DB of some of my Tor circuits and nodes which i am
>> analyzing. I grabbed 48 circuits with their 144 nodes and info (IP address,
>> nickname, country) manually from my laptop's Tor network map.
>> 
>> 
>> That certainly sounds painful. The circuit paths will provide the
>> relay fingerprints which you can use for get_network_status() to get
>> the address, nickname, ORPort, etc...
>> 
>> https://stem.torproject.org/api/control.html#stem.control.Controller.get_network_status
>> 
>> As for locales that would be done via get_info('ip-to-country/<address>')...
>> 
>> https://gitweb.torproject.org/torspec.git/blob/HEAD:/control-spec.txt#l672
>> 
>> ... and ultimately to AS and country.
>> 
>> 
>> AS will require the Maxmind AS database or something else. I know that
>> Onionoo includes the AS information so the options that come to mind
>> are either to (a) see how it does it or (b) query Onionoo for this
>> information.
>> 
>> https://onionoo.torproject.org/
>> 
>> And i have read much of the control-spec, don't know how much is out of date
>> 
>> 
>> The control-spec should be up to date. If there are any inaccuracies
>> then please let us know!
>> 
>> Did you have any other questions? -Damian
>> 
>> On Mon, Jun 10, 2013 at 4:44 AM, SARAH CORTES <sarah at lewis.is> wrote:
>> 
>> Hi, Damian, thanks. I am happy to discuss it on tor-dev at . But I want to keep
>> off spam, which some of my questions at first may be, essentially Qs. But,
>> if you think they would be of interest to tor-dev, or others could help,
>> just let me know, and i will sign up for it.
>> 
>> For example, I am trying to figure out how to pull in the nodes that are
>> actually used in my Tor circuits. They are the nodes reflected in the
>> Network map function. I have read your fine stem tutorials, it is so
>> impressive you have dome so much work and documentation on your own
>> initiative. It is certainly a huge help to me and otter researchers like me.
>> 
>> I have created a MySql DB of some of my Tor circuits and nodes which i am
>> analyzing. I grabbed 48 circuits with their 144 nodes and info (IP address,
>> nickname, country) manually from my laptop's Tor network map. I am analyzing
>> their country, so I am seeking to map nickname or fingerprint of the node to
>> IP address and ultimately to AS and country. Ultimately, I am analyzing
>> jurisdictional arbitrage and the hostility factors of differ countries to
>> Tor traffic, and whether it is possible to incorporate country or AS in to
>> the Tor path algorithm.
>> 
>> I can see the entry guard nicknames and fingerprints in my state file and
>> pick them up there, if necessary. Wish i could just edit the logs to display
>> all nodes, then figure out how to map the node fingerprint or nickname to
>> its IP address them AS and country. I understand there is a Tor atlas
>> function but have not read it carefully yet.
>> 
>> I have read documentation on the contents of the consensus file, the
>> descriptors, geoip and state files, and played around with them a bit to
>> understand them. I grasp that a fingerprint is a hash of the node's public
>> key, and one or both of which seem to be the most reliable ID for a node.
>> 
>> I can see from your documentation that variable path(nickname, fingerprint)
>> contains the data i want. Just need to be able to get that data easily. Not
>> sure if editing my torrc file is the way or what. If you have any
>> suggestions, they are much appreciated.
>> 
>> And i have read much of the control-spec, don't know how much is out of date
>> https://gitweb.torproject.org/torspec.git/blob/HEAD:/control-spec.txt
>> 
>> I have read this on modifying torrc, and played around with modifying torrc,
>> on an elementary basis:
>> https://www.torproject.org/docs/faq.html.en#torrc
>> 
>> and some tickets on modifying torrc
>> https://trac.torproject.org/projects/tor/ticket/6147
>> 
>> and what I believe is your excellent documentation on stem events;
>> https://stem.torproject.org/api/response.html#stem.response.events.BandwidthEvent
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> On Jun 8, 2013, at 2:31 PM, Damian Johnson <atagar at torproject.org> wrote:
>> 
>> Hi Sarah. Yup, I'd be more than happy to help if you have any stem
>> questions. Email would definitely be better since we look to keep
>> missing each other on irc. I'd appreciate it though if we discussed it
>> on the tor-dev@ email list...
>> 
>> https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
>> 
>> On Sat, Jun 8, 2013 at 3:42 AM, SARAH CORTES <sarah at lewis.is> wrote:
>> 
>> Hi Damian, I'm Sarah Cortes, a researcher. I tried you on irc on #tor-dev
>> but was unable to reach you. We met at the tor dev meeting which i attended
>> in March. I have some Qs for you regarding stem. Please let me know if you
>> are available, thanks
>> 
>> 
>> 