[metrics-team] CollecTor exit-list archive bug

[Wojtek Porczyk]

Hello, Metrics Team,

I belive I found a bug in CollecTor. Since 2016-11-01-09-02-00 the exit list
archive features "@type torperf" instead of "tordnsel", which makes it
unreadable in at least stem (python) parser.

> [woju at REDACTED ~/tmp/exit-list-2016-11/01]$ head 2016-11-01-08-02-00
> @type tordnsel 1.0
> Downloaded 2016-11-01 08:02:00
> ExitNode 0011BD2485AD45D984EC4159C88FC066E5E3300E
> Published 2016-10-31 13:21:33
> LastStatus 2016-10-31 14:03:03
> ExitAddress 162.247.72.201 2016-10-31 14:03:21
> ExitNode 004FDAC2B5B966F53361FA34D7E6E48D512FE869
> Published 2016-10-31 21:59:17
> LastStatus 2016-10-31 23:03:01
> ExitAddress 103.14.68.50 2016-10-31 23:06:46
> [woju at REDACTED ~/tmp/exit-list-2016-11/01]$ head 2016-11-01-09-02-00
> @type torperf 1.0
> Downloaded 2016-11-01 09:02:00
> ExitNode 0011BD2485AD45D984EC4159C88FC066E5E3300E
> Published 2016-10-31 13:21:33
> LastStatus 2016-11-01 08:03:04
> ExitAddress 162.247.72.201 2016-10-31 14:03:21
> ExitNode 004FDAC2B5B966F53361FA34D7E6E48D512FE869
> Published 2016-10-31 21:59:17
> LastStatus 2016-10-31 23:03:01
> ExitAddress 103.14.68.50 2016-10-31 23:06:46

I believe this is caused by commit 7e80f72 "Implements task-18910 sync-functionality."
https://gitweb.torproject.org/collector.git/commit/?id=7e80f72393dd98792fc25fcaced6a97a3bafb973:

> diff --git a/src/main/java/org/torproject/collector/conf/Annotation.java b/src/main/java/org/torproject/collector/conf/Annotation.java
> new file mode 100644
> index 0000000..7f301bb
> --- /dev/null
> +++ b/src/main/java/org/torproject/collector/conf/Annotation.java
> @@ -0,0 +1,38 @@

> +  ExitList("@type tordnsel 1.0\n"),

> +  Torperf("@type torperf 1.0\n"),

> diff --git a/src/main/java/org/torproject/collector/exitlists/ExitListDownloader.java b/src/main/java/org/torproject/collector/exitlists/ExitListDownloader.java
> index 5609cea..81fab3d 100644
> --- a/src/main/java/org/torproject/collector/exitlists/ExitListDownloader.java
> +++ b/src/main/java/org/torproject/collector/exitlists/ExitListDownloader.java

> @@ -69,7 +76,7 @@ public class ExitListDownloader extends CollecTorMain {
>      try {
>        logger.debug("Downloading exit list...");
>        StringBuilder sb = new StringBuilder();
> -      sb.append("@type tordnsel 1.0\n");
> +      sb.append(Annotation.Torperf.toString());
>        sb.append("Downloaded " + dateTimeFormat.format(downloadedDate)
>            + "\n");
>        URL url = config.getUrl(Key.ExitlistUrl);

Patch fixing this:

diff --git a/src/main/java/org/torproject/collector/exitlists/ExitListDownloader.java b/src/main/java/org/torproject/collector/exitlists/ExitListDownloader.java
index 3c2f403..26fbe4b 100644
--- a/src/main/java/org/torproject/collector/exitlists/ExitListDownloader.java
+++ b/src/main/java/org/torproject/collector/exitlists/ExitListDownloader.java
@@ -76,7 +76,7 @@ public class ExitListDownloader extends CollecTorMain {
     try {
       logger.debug("Downloading exit list...");
       StringBuilder sb = new StringBuilder();
-      sb.append(Annotation.Torperf.toString());
+      sb.append(Annotation.ExitList.toString());
       sb.append("Downloaded " + dateTimeFormat.format(downloadedDate)
           + "\n");
       URL url = config.getUrl(Key.ExitlistUrl);

You may also pull tag fix-exit-list from https://github.com/woju/collector
(commit e885283).

Can you also fix the already posted files at collector.torproject.org?

-- 
pozdrawiam / best regards       _.-._
Wojtek Porczyk               .-^'   '^-.
Invisible Things Lab         |'-.-^-.-'|
                             |  |   |  |
 I do not fear computers,    |  '-.-'  |
 I fear lack of them.        '-._ :  ,-'
    -- Isaac Asimov             `^-^-_>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: Digital signature
URL: <http://lists.torproject.org/pipermail/metrics-team/attachments/20170110/8de52a73/attachment.sig>