[metrics-bugs] #22594 [Metrics/Onionoo]: Escape characters in contact lines break hourly updater
Tor Bug Tracker & Wiki
blackhole at torproject.org
Tue Jun 13 15:39:53 UTC 2017
#22594: Escape characters in contact lines break hourly updater
---------------------------------+--------------------------
Reporter: karsten | Owner: metrics-team
Type: defect | Status: new
Priority: High | Milestone:
Component: Metrics/Onionoo | Version:
Severity: Normal | Keywords:
Actual Points: | Parent ID:
Points: | Reviewer:
Sponsor: |
---------------------------------+--------------------------
Onionoo's hourly updater broke on June 12 at around 13:30 UTC. Turns out
it couldn't process the following server descriptor (look out for the
contact line):
{{{
@type server-descriptor 1.0
router HarukaMiddle 46.101.62.152 9001 0 0
identity-ed25519
-----BEGIN ED25519 CERT-----
AQQABlsyAa97mM9YlSM9a8RHbgNwUduV8zMYrUs/uXhfk3fg7ZPEAQAgBAA/B+AS
R+3S5M09GNQjE9EzvGR/FS6s+WjMs9bNdxTideK2fjKqU3mR+QqCvOP7DYEh8/2w
VMChyxEjyKWBRo4iFyVTICqeuStIRLqPAVY/ODcvHbQNbzOY1F8OZSEWWwc=
-----END ED25519 CERT-----
master-key-ed25519 PwfgEkft0uTNPRjUIxPRM7xkfxUurPlozLPWzXcU4nU
platform Tor 0.2.9.11 on Linux
proto Cons=1-2 Desc=1-2 DirCache=1 HSDir=1 HSIntro=3 HSRend=1-2 Link=1-4
LinkAuth=1 Microdesc=1-2 Relay=1-2
published 2017-06-12 12:39:39
fingerprint EABD 6C28 2F28 C6F3 EBB1 AA59 3B50 2071 3B33 D131
uptime 0
bandwidth 2048000 2048000 2220032
extra-info-digest 290AF585354448C448A748CBC632D933CD2880D6
s5rHOgcE+7SAeDUUiQbb9OaYgpfPFOBgW/uJ4SVuXuk
onion-key
-----BEGIN RSA PUBLIC KEY-----
MIGJAoGBAMaI+nbAiMpcMZRXhV6ai9ccHwhp9mr5nNwYunUzdNSyCIB0N56ODZ3c
xme3mG1QQ08um1ewXb6vMxxsiZzzYVxdyawOa+oHMYQEISWyZlvPw+7PDtAg5bGR
8gWqUAIXSTe3K1YXnaXpNcEiwVOO35jDih5HkzkVCvnzz8BCYDUHAgMBAAE=
-----END RSA PUBLIC KEY-----
signing-key
-----BEGIN RSA PUBLIC KEY-----
MIGJAoGBAMygGYMk7d+pYQu5JyCROtRS2LtURSSgfaAGsPUblTGD+ZoSlDDGIu2Q
AUC7QzQxO1HxbUqkoQwo0Im3jsQPcHznIuKevJEzMCPkErn1DIwOidYdW9tTLgcG
4+q5pvwpyd0uDTV2Z8SqXnIyVfAFpAGt8LOOFahJiTMHRRI8ABsFAgMBAAE=
-----END RSA PUBLIC KEY-----
onion-key-crosscert
-----BEGIN CROSSCERT-----
HAn8AxeqoyWnI0h/JhnKoEEs6Lgz/JwmhruEjOC488bOF5G1HJrkM675Sj4B8A3Z
5pGPn0hi1Owbza72nlUtxPuK8pa6ItP/A8Q1v9AvwYhIkmEvHVaCH/FzthW+8AzG
lGiH6FcP/VQ3htQ2kpNUyMrkqLVgDHmVXgT/bXBGsto=
-----END CROSSCERT-----
ntor-onion-key-crosscert 1
-----BEGIN ED25519 CERT-----
AQoABllNAT8H4BJH7dLkzT0Y1CMT0TO8ZH8VLqz5aMyz1s13FOJ1ADVfDOXFoxvI
Zre9gmhKEuPq10ioGbGaKtvj/YrdVsFwNc76mGRSpaM2Ar2DBkxpZBh/p3Cwqe0V
Bbcre5sQOgo=
-----END ED25519 CERT-----
hidden-service-dir
contact Haruka iampsychopath/\@/\live/\./\co/\./\uk
ntor-onion-key 98H8kDWwomC0z+goDIgI2MH0fMkW1I1vbme2dBoo8TY=
reject *:*
router-sig-ed25519
UFYUr1vTeiRwA4grYD/LGLHC8xuzKMSNDijHIcZkckqXdxiXLwRasQagyXuKxwKS+q6A7uIxRUcwdjxq7t1sBA
router-signature
-----BEGIN SIGNATURE-----
wngC1BBTQUrSkZejdr9pYEGJmAreph8x0g0UvA5jjTX7do/ltRrmPN6VSgxbd36y
nHEe+cL8jYXUyLBENxnD4cA4pVxYgqFYWlhgtrDIonmeMWGXfirJBHIbZG/zKeVv
EXRdkh13OHEtUWU1PEGRSNNt7oSQf6rl//Qwz3Andx8=
-----END SIGNATURE-----
}}}
Here's the exception:
{{{
java.lang.NumberFormatException: \uk","
at
com.google.gson.stream.JsonReader.readEscapeCharacter(JsonReader.java:1466)
~[onionoo-4.0-1.2.0-dev.jar:4.0-1.2.0-dev-595f87a]
at
com.google.gson.stream.JsonReader.nextQuotedValue(JsonReader.java:993)
~[onionoo-4.0-1.2.0-dev.jar:4.0-1.2.0-dev-595f87a]
at
com.google.gson.stream.JsonReader.nextString(JsonReader.java:811)
~[onionoo-4.0-1.2.0-dev.jar:4.0-1.2.0-dev-595f87a]
at
com.google.gson.internal.bind.TypeAdapters$13.read(TypeAdapters.java:358)
~[onionoo-4.0-1.2.0-dev.jar:4.0-1.2.0-dev-595f87a]
at
com.google.gson.internal.bind.TypeAdapters$13.read(TypeAdapters.java:346)
~[onionoo-4.0-1.2.0-dev.jar:4.0-1.2.0-dev-595f87a]
at
com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$1.read(ReflectiveTypeAdapterFactory.java:93)
~[onionoo-4.0-1.2.0-dev.jar:4.0-1.2.0-dev-595f87a]
at
com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$Adapter.read(ReflectiveTypeAdapterFactory.java:172)
~[onionoo-4.0-1.2.0-dev.jar:4.0-1.2.0-dev-595f87a]
at com.google.gson.Gson.fromJson(Gson.java:803)
~[onionoo-4.0-1.2.0-dev.jar:4.0-1.2.0-dev-595f87a]
at com.google.gson.Gson.fromJson(Gson.java:768)
~[onionoo-4.0-1.2.0-dev.jar:4.0-1.2.0-dev-595f87a]
at com.google.gson.Gson.fromJson(Gson.java:717)
~[onionoo-4.0-1.2.0-dev.jar:4.0-1.2.0-dev-595f87a]
at com.google.gson.Gson.fromJson(Gson.java:689)
~[onionoo-4.0-1.2.0-dev.jar:4.0-1.2.0-dev-595f87a]
at
org.torproject.onionoo.docs.DocumentStore.retrieveParsedDocumentFile(DocumentStore.java:539)
[onionoo-4.0-1.2.0-dev.jar:4.0-1.2.0-dev-595f87a]
at
org.torproject.onionoo.docs.DocumentStore.retrieveDocumentFile(DocumentStore.java:505)
[onionoo-4.0-1.2.0-dev.jar:4.0-1.2.0-dev-595f87a]
at
org.torproject.onionoo.docs.DocumentStore.retrieve(DocumentStore.java:378)
[onionoo-4.0-1.2.0-dev.jar:4.0-1.2.0-dev-595f87a]
at
org.torproject.onionoo.updater.NodeDetailsStatusUpdater.processRelayServerDescriptor(NodeDetailsStatusUpdater.java:151)
[onionoo-4.0-1.2.0-dev.jar:4.0-1.2.0-dev-595f87a]
at
org.torproject.onionoo.updater.NodeDetailsStatusUpdater.processDescriptor(NodeDetailsStatusUpdater.java:130)
[onionoo-4.0-1.2.0-dev.jar:4.0-1.2.0-dev-595f87a]
at
org.torproject.onionoo.updater.DescriptorSource.readDescriptors(DescriptorSource.java:132)
[onionoo-4.0-1.2.0-dev.jar:4.0-1.2.0-dev-595f87a]
at
org.torproject.onionoo.updater.DescriptorSource.readDescriptors(DescriptorSource.java:97)
[onionoo-4.0-1.2.0-dev.jar:4.0-1.2.0-dev-595f87a]
at org.torproject.onionoo.cron.Main.updateStatuses(Main.java:180)
[onionoo-4.0-1.2.0-dev.jar:4.0-1.2.0-dev-595f87a]
at org.torproject.onionoo.cron.Main.run(Main.java:129)
[onionoo-4.0-1.2.0-dev.jar:4.0-1.2.0-dev-595f87a]
at
org.torproject.onionoo.cron.Main.runOrScheduleExecutions(Main.java:103)
[onionoo-4.0-1.2.0-dev.jar:4.0-1.2.0-dev-595f87a]
at org.torproject.onionoo.cron.Main.main(Main.java:35)
[onionoo-4.0-1.2.0-dev.jar:4.0-1.2.0-dev-595f87a]
}}}
This issue was quite well hidden, because `Gson.fromJson` apparently
doesn't catch `NumberFormatException`, so the main thread just dies.
Here's the (pretty-printed) string that it attempted to parse:
{{{
{
"desc_published": "2017-06-12 12:39:39",
"last_restarted": "2017-06-12 12:39:39",
"bandwidth_rate": 2048000,
"bandwidth_burst": 2048000,
"observed_bandwidth": 2220032,
"advertised_bandwidth": 2048000,
"exit_policy": [
"reject *:*"
],
"contact":"Haruka
iampsychopath/\\\\@/\\\\live/\\\\./\\\\co/\\\\./\\\uk",
"platform": "Tor 0.2.9.11 on Linux",
"alleged_family": [],
"effective_family": [],
"indirect_family": [],
"is_relay": true,
"running": true,
"nickname": "HarukaMiddle",
"address": "46.101.62.152",
"or_addresses_and_ports": [],
"first_seen_millis": 1497031200000,
"last_seen_millis": 1497268800000,
"or_port": 9001,
"dir_port": 0,
"relay_flags": [
"Fast",
"Running",
"Valid"
],
"consensus_weight": 483,
"default_policy": "reject",
"port_list": "1-65535",
"last_changed_or_address_or_port": 1497031200000,
"recommended_version": true,
"measured": true,
"exit_addresses": {},
"consensus_weight_fraction": 1.1896908e-05,
"guard_probability": 0,
"middle_probability": 3.122049e-05,
"exit_probability": 0,
"latitude": 51.5092,
"longitude": -0.0955,
"country_code": "gb",
"country_name": "United Kingdom",
"region_name": "England",
"city_name": "London",
"as_name": "Digital Ocean, Inc.",
"as_number": "AS14061",
"host_name": "46.101.62.152"
}
}}}
I deployed the following hotfix:
{{{
diff --git a/src/main/java/org/torproject/onionoo/docs/DocumentStore.java
b/src/main/java/org/torproject/onionoo/docs/DocumentStore.java
index 34bc8ef..246c02b 100644
--- a/src/main/java/org/torproject/onionoo/docs/DocumentStore.java
+++ b/src/main/java/org/torproject/onionoo/docs/DocumentStore.java
@@ -537,8 +537,9 @@ public class DocumentStore {
Gson gson = new Gson();
try {
result = gson.fromJson(documentString, documentType);
- } catch (JsonParseException e) {
+ } catch (JsonParseException | NumberFormatException e) {
/* Handle below. */
+ log.error(documentString);
log.error(e.getMessage(), e);
}
if (result == null) {
}}}
The hourly update is still running, but I believe the result will be that
the relay publishing this descriptor will simply not show up in Onionoo
results, or at least not with recent data. Should be fine for the moment,
but deserves a better fix. The real fix is to check our logic for
escaping/unescaping JSON strings, yet once more.
--
Ticket URL: <https://trac.torproject.org/projects/tor/ticket/22594>
Tor Bug Tracker & Wiki <https://trac.torproject.org/>
The Tor Project: anonymity online
More information about the metrics-bugs
mailing list