[tor-dev] Proc Unit Testing & Next Project

Tue Jun 26 21:41:16 UTC 2012

Hi Damian,

> In reading the following code I suspect that this could be clearer if
it accepted a single argument that was the dict of 'argument => return
value'.

All set -- mock_fn() now takes a dictionary rather than two lists as an
argument.  The docstring suggestions have also been implemented.

> Why do you fall through to the target function? Unit tests need to be
os independent, so if falling though to an os will break on some
platforms then we'll need to find an alternative.

We do this in order to accommodate a call of open() that is being made
somewhere within the call of test_get_connections().  We had no easy way of
seeing what exactly was making this call, but it was giving us a large
backtrace and error if we did not provide a way for the original open
funciton to be called.

We will run this code under windows tomorrow when we have access to a
windows machine.

All of these changes are in Megan's Github at:
https://github.com/meganchang/Stem/blob/proc-tests/test/unit/util/proc.py

Per your suggestion to use itertools.product(), we tried this but ran into
a problem.  Our subsets() function returns a list of lists, where each
sub-list contains tuples that bind input values to return values -- we want
these to be persistent throughout.  itertools doesn't seem to be able to
take a single list of tuples and create all possible combinations of these
tuples (from each individual tuple in its own list to all four given tuples
in a single list).  It is challenging to explain, but let us know if you
think it is still a better option when you have a chance to look over the
code again.

As for Onionoo, we had a brief discussion about it today, and we are
definitely considering it.  We will take a closer look at it tomorrow and
give a more definite answer then.

Have a good evening,
Erik & Megan

On Tue, Jun 26, 2012 at 4:25 PM, Damian Johnson <atagar at torproject.org>wrote:

> Hi Erik, hi Megan.
>
> > Megan and I have finished a first run at writing unit tests for the proc
> utilities in stem.
>
> Looks great! I only got down to 'test_get_memory_usage()' before
> needing to run to catch the bus but the tests that I've seen so far
> look good. The mocking though confused me for a bit.
>
> > +def mock_fn(exp_args, return_vals, target=None):
> > +  """
> > +  Provides a lambda function that may be used to mock another function.
>
> In reading the following code I suspect that this could be clearer if
> it accepted a single argument that was the dict of 'argument => return
> value'.
>
> > +  :param list of tuples exp_args: expected input value(s) to be used
> for comparison
>
> The ReStructuredTest format for param entries are...
> :param [type ]arg_name:
>
> So to properly compile this should be...
> :param list exp_args:
>
> Usually I say what the list contains in the following description.
>
> > +  :param function target: target function to be called if mocking
> doesn't cover this input
>
> Why do you fall through to the target function? Unit tests need to be
> os independent, so if falling though to an os will break on some
> platforms then we'll need to find an alternative.
>
> Speaking of os independence, if it isn't too hard would you mind
> running this on a warty platform like Windows? If you don't have a
> Windows system laying around then don't worry about it.
>
> Don't worry if the expand_path unit test breaks - that's something
> Beck is currently fixing.
>
> > +  :precondition: len(exp_args) = len(return_vals)
>
> I checked the sphinx index [1] and python domain [2] for
> "precondition" and didn't find it. Are you sure that directive exists?
> If not then we can simply provide it in the above function
> description.
>
> > +                                                 return_vals[i]
>  a=exp_args[i]
> > +  :returns:  function _mocker such that: f(*a) = target(*a)        a !=
> exp_args[i] and target != N
> > +                                                 raise TargetError a !=
> exp_args[i] and target = No
>
> Nice, though to make this render the way that you want in sphinx it
> should be changed to...
>
> :returns:  function _mocker such that...
>  * return_vals[i]    a = exp_args[i]
>  * target(*a)        a != exp_args[i] and target != N
>  * raise TargetError a != exp_args[i] and target = No
>
> > +  Used with the builtin zip function to create all possible combinations
> > +  of two lists. Called in test_get_stats().
>
> That sounds a lot like itertools.product().
>
> >>> import itertools
> >>> list(itertools.product([1, 2, 3], ['a', 'b', 'c']))
> [(1, 'a'), (1, 'b'), (1, 'c'), (2, 'a'), (2, 'b'), (2, 'c'), (3, 'a'),
> (3, 'b'), (3, 'c')]
>
> > Looking further into the future, we discussed our next project with Stem
> with Professor Danner today, and agreed that the "Export Tor Desciptors"
> project would be ideal.
>
> Ok. Little more context on that project in case you're not on tor-dev at ...
> https://lists.torproject.org/pipermail/tor-dev/2012-June/003634.html
>
> I was gonna suggest a python port of Onionoo [3][4], which is the
> service that supports Atlas [5]. Karsten would be able to help mentor
> that, and it would involve filling in the missing bits of descriptor
> parsing that we'll need for it (such as network status entries).
> However, that would be far more ambitious and might be a bit too
> large. Up to you.
>
> Cheers! -Damian
>
> [1] http://sphinx.pocoo.org/genindex.html
> [2] http://sphinx.pocoo.org/domains.html#directive-py:class
> [3] https://www.torproject.org/projects/onionoo.html
> [4] https://gitweb.torproject.org/onionoo.git/blob/HEAD:/DESIGN
> [5] https://atlas.torproject.org/
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.torproject.org/pipermail/tor-dev/attachments/20120626/d99d0ffb/attachment-0001.html>