[tor-dev] Stem Descriptor Parsers

[Megan Chang]

Hi Damian,

After looking at possible use cases, wouldn't it make sense to allow the
caller to specify a file to be written to?  Regardless, we were thinking of
creating two methods, one that takes a list of descriptors, and one that
takes a single descriptor. This would remove the need to check for a list
versus an object, allowing more consistent typing.

Just to clarify, the include_fields and exclude_fields parameters would
have default values of none and since we are taking in descriptors are a
list rather than a *arg, we don't need to worry about specifying the
keyword parameters. That said, if a caller doesn't specify either, all
parameters would be returned. Otherwise, it is expected that only one of
these parameters would be specified by the caller.

Also, going back to features expected by the community, would users want a
csv header to be written? Or simply a csv file?

- Erik & Megan

On Mon, Jul 9, 2012 at 1:22 PM, Damian Johnson <atagar at torproject.org>wrote:

> On Mon, Jul 9, 2012 at 8:40 AM, Erik I Islo <eislo at wesleyan.edu> wrote:
> > Hello,
> >
> > Megan and I have been working on the CSV export functionality that was
> being
> > discussed a little over a week ago, and given the recent discussion, we
> > would like to clarify the expected/desired implementation of this
> feature.
> >
> > We have created an export.py module within /stem/descriptor, which
> contains
> > a single method as of now that takes a descriptor object and two possible
> > lists of fields.  These lists are to be specified as either the
> explicitly
> > included attributes of the descriptor or the attributes to be excluded.
>  As
> > we continue to work on this code, Megan and I were wondering if it
> wouldn't
> > be better to accept a file object as well, in addition to accepting any
> > number of descriptor objects (i.e. def csv_exp(..., *descriptors)).  Or
> are
> > there other suggestions request concerning what sort of input such a
> method
> > should take?
> >
> > -Erik & Megan
> >
> > On Fri, Jul 6, 2012 at 1:49 PM, Damian Johnson <atagar at torproject.org>
> > wrote:
> >>
> >> > So is export intended to be an instance method of descriptor, one that
> >> > just dumps a single csv line of the instance attributes (maybe
> subject to
> >> > some selection of those attributes)?  Or a static method that takes a
> >> > collection?
> >>
> >> Either would work fine. I was envisioning the former, though on
> >> reflection stem/descriptor/export.py module would probably be better
> >> since that localizes this functionality and allows for better
> >> expansion in the future (other formats such as json, or the inclusion
> >> of import functionality).
> >>
> >> > It seems like it might be awkward to have to hack stem itself to add a
> >> > new export format (for example).  Is this a concern?
> >>
> >> That depends on how useful users would find it to be. If researchers
> >> commonly want csv export functionality then we might as well support
> >> it. However, if it's a rarely desired feature then there's little
> >> reason to clutter our API. My understanding is that this feature is
> >> mostly for researchers and sysadmins, so as part of the target
> >> audience I'm happy to defer to you on how we handle this.
> >>
> >> > Do all the known use-cases make need both an interface to Tor Control
> >> > and a descriptor utility library?
> >>
> >> No, you're completely right. Stem's controller functionality utilizes
> >> its descriptor functionality but not vice versa. Another design that
> >> we could go with is to make several smaller libraries (descriptors,
> >> controller, response parsing, shared utilities, etc) if stem grows
> >> unwieldy. However, we're nowhere near that yet and keeping stem as a
> >> single library makes development, testing, installation and usage far
> >> easier.
> >>
> >> Stem is a library to make working with Tor easier for developers and
> >> researchers, with the current scope of the Tor control and dir specs.
> >> My plan is to complete that, release it to the community, then see
> >> based on feedback where we should go from there.
> >
> >
>
> Naif: This was your feature request. Thoughts?
>
> > Megan and I were wondering if it wouldn't
> > be better to accept a file object as well, in addition to accepting any
> > number of descriptor objects (i.e. def csv_exp(..., *descriptors)).
>
> If we can make it work then that would be nice, though having a *list
> entry generally doesn't work well for optional keyword fields. Ie, if
> you had the signature...
>
> def csv_exp(include_fields = None, exclude_fields = None, destination
> = None, *descriptors)
>
> Then the caller needs to provide all of those keyword fields which
> kinda defeats the purpose of them being optional. For instance, to
> call it with the defaults and a single descriptor it would be...
>
> csv_exp(None, None, None, my_descriptor)
>
> My suggestion is to just accept a single argument that can either be a
> single descriptor or a list of descriptors.
>
> Cheers! -Damian
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.torproject.org/pipermail/tor-dev/attachments/20120710/4d50c850/attachment.html>