[tor-bugs] #18910 [Metrics/CollecTor]: distributing descriptors accross CollecTor instances

[Tor Bug Tracker & Wiki]

#18910: distributing descriptors accross CollecTor instances
-------------------------------+---------------------------------
 Reporter:  iwakeh             |          Owner:  iwakeh
     Type:  enhancement        |         Status:  needs_review
 Priority:  High               |      Milestone:  CollecTor 1.1.0
Component:  Metrics/CollecTor  |        Version:
 Severity:  Normal             |     Resolution:
 Keywords:  ctip               |  Actual Points:
Parent ID:                     |         Points:
 Reviewer:                     |        Sponsor:
-------------------------------+---------------------------------

Comment (by iwakeh):

 Please clarify the path differences from comment:25 above.  I've been
 looking at the paths and the protocol for too long now that I caught some
 path-blindness (temporary).
 It would be very helpful, if you could use the test-cases listed
 [https://gitweb.torproject.org/user/iwakeh/collector.git/tree/src/test/java/org/torproject/collector/sync/SyncPersistenceTest.java?h=task-18910
 -first-sync#n59 here] and indicate what should be changed, i.e. basically
 which paths are expected.  Then the tests can verify the structure.

 The protocol might need to be changed, if this path `./out/bridge-
 descriptors/2016/10/extra-
 infos/0/0/0000000001764ef8b8b5bc9ed70b9e99225112ffd04` is  correct. It
 [https://gitweb.torproject.org/collector.git/tree/src/main/resources/docs/PROTOCOL#n284
 says]:
 {{{
 5.2 'bridge-descriptors' below 'out'

    'bridge-descriptors' contains the following subdirectories:

    * extra-infos
    * server-descriptors
    * statuses

 5.2.1
    'extra-infos' and 'server-descriptors' have the following
    subdirectory structure

    year SEP month SEP first SEP second

    Where year is derived from the published date.
    'first' and 'second' are the first and second symbol from the
    router-digest, which also serves as the filename for the files
    in the 'second' level directories.

    Tars are named according to section 2.3 and have the following
    substructure using the definitions from 2.3:

    BRIDGE DASH marker DASH year DASH month SEP first SEP second

 5.2.2
    'statuses' have a different substructure

    year SEP month SEP day

 }}}

--
Ticket URL: <https://trac.torproject.org/projects/tor/ticket/18910#comment:44>
Tor Bug Tracker & Wiki <https://trac.torproject.org/>
The Tor Project: anonymity online