[tor-bugs] #18798 [Metrics/CollecTor]: analysis of descriptor completeness

[Tor Bug Tracker & Wiki]

#18798: analysis of descriptor completeness
-------------------------------+-----------------------------------
 Reporter:  iwakeh             |          Owner:  iwakeh
     Type:  task               |         Status:  needs_information
 Priority:  Medium             |      Milestone:
Component:  Metrics/CollecTor  |        Version:
 Severity:  Normal             |     Resolution:
 Keywords:  ctip               |  Actual Points:
Parent ID:                     |         Points:
 Reviewer:                     |        Sponsor:
-------------------------------+-----------------------------------

Comment (by karsten):

 Replying to [comment:8 iwakeh]:
 > I added the referencing descriptor analysis to the todo list.

 Great!

 > Another question regarding the logging statements:
 >
 > What do the following numbers mean, i.e. how reliable are these numbers?
 > Also, the difference between "importing" and "downloading"?
 >
 > [...]

 CollecTor receives its descriptors from two sources:
  1. it first fetches copies of cached descriptors from directory authority
 gabelmoo using `wget` on the command line outside of the directory
 protocol and imports them (the "import" step);
  1. it looks which descriptors it's still missing and fetches them via the
 directory protocol from most directory authorities except a few that have
 been too slow in the past (the "download" step).

 I believe that the numbers in the logs are reliable, but I don't know for
 certain.

 > And:
 > Why are there so many "404 Not found" like
 > [...]
 >
 > Does this mean that just one of the requested descriptors wasn't found?
 > Or, that there was no answer, or ...?
 > Hope I didn't overlook anything obvious in the tor-spec.

 Fine question, let's simply try it out.  The current extra-info descriptor
 digest of TorLand1 is `352c235cd4134d0d8ff41c1f88c653a450bd4137`:

 {{{
 curl -v
 http://154.35.32.5/tor/extra/d/352c235cd4134d0d8ff41c1f88c653a450bd4137 >
 /dev/null   # 200 OK
 curl -v
 http://154.35.32.5/tor/extra/d/0000000000000000000000000000000000000000 >
 /dev/null   # 404 Not found
 curl -v
 http://154.35.32.5/tor/extra/d/352c235cd4134d0d8ff41c1f88c653a450bd4137+0000000000000000000000000000000000000000
 > /dev/null   # 200 OK
 }}}

 So, looks like "404 Not found" means that none of the requested descriptor
 was found.  Maybe the spec agrees, I didn't look.

--
Ticket URL: <https://trac.torproject.org/projects/tor/ticket/18798#comment:9>
Tor Bug Tracker & Wiki <https://trac.torproject.org/>
The Tor Project: anonymity online