[or-cvs] r18296: {torflow} Fix a log line and jot down some notes. (torflow/trunk/NetworkScanners)

[mikeperry at seul.org]

Author: mikeperry
Date: 2009-01-28 10:37:52 -0500 (Wed, 28 Jan 2009)
New Revision: 18296

Modified:
   torflow/trunk/NetworkScanners/soat.py
Log:

Fix a log line and jot down some notes.



Modified: torflow/trunk/NetworkScanners/soat.py
===================================================================
--- torflow/trunk/NetworkScanners/soat.py	2009-01-28 14:30:28 UTC (rev 18295)
+++ torflow/trunk/NetworkScanners/soat.py	2009-01-28 15:37:52 UTC (rev 18296)
@@ -163,6 +163,12 @@
 # Note: the more we add, the greater the potential for false positives...  
 # We also only care about the ones that work for FF2/FF3. 
 
+# TODO: If we cut down on these tags, we can cut down on false 
+# positives. The ultimate acid test would be to have two different Google 
+# queries come back with the same tag structure after filtering them.
+# Unfortunately, Google munges its javascript, so we have to do
+# some more advanced processing to reach that goal..
+# Also, I'm somewhat torn on dropping 'a' tags..
 tags_to_check = ['a', 'applet', 'area', 'base', 'body', 'embed', 'form',
                  'frame', 'iframe', 'img', 'input', 'link', 'meta', 
                  'object', 'script', 'style']
@@ -482,7 +488,7 @@
 
       self.remove_target(address)
     else:
-      plog("ERROR", self.proto+" 3-way failure at "+exit_node+". This makes "+str(err_cnt)+" node failures for "+address)
+      plog("ERROR", self.proto+" http error code failure at "+exit_node+". This makes "+str(err_cnt)+" node failures for "+address)
     
   def register_dynamic_failure(self, address, exit_node):
     if address in self.three_way_fails:
@@ -891,6 +897,13 @@
       else: self.successes[address]=1
       return TEST_SUCCESS
 
+    # TODO: Can we create some kind of diff/masking filter
+    # between the two non-Tor soups, and apply it to the
+    # Tor soup, to see if anything additional has changed?
+    # http://bramcohen.livejournal.com/37690.html
+    #  -> patiencediff.py vs difflib
+    #     "For small files difflib wins". And it's standard. Yay!
+
     # XXX: Check for existence of this file before overwriting
     exit_tag_file = open(failed_prefix+'.dyn-tags.'+exit_node[1:],'w')
     exit_tag_file.write(psoup.__str__())