Proposal: Exit Scanning

[Nick Mathewson]

On Tue, Feb 17, 2009 at 02:23:49AM -0800, Mike Perry wrote:

Good proposal.

 [...]
> While it is not realistic to expect to catch extremely targeted or
> completely passive malicious adversaries, the goal is to prevent
> malicious adversaries from deploying dragnet attacks against large
> segments of the Tor userbase.

Another goal is to discourage in would-be attackers the idea that
it's totally safe to 

> Scanning methodology:
> 
> The first scans to be implemented are HTTP, HTML, Javascript, and
> SSL scans.
> 
> The HTTP scan scrapes Google for common filetype urls such as exe, msi,
> doc, dmg, etc. It then fetches these urls through Non-Tor and Tor, and
> compares the SHA1 hases of the resulting content. 
                      ^^hashes
> The SSL scan downloads certificates for all IPs a domain will locally
> resolve to and compares these certificates to those seen over Tor. The
> scanner notes if a domain had rotated certificates locally in the
> results for each scan.
> 
> The HTML scan checks HTML, Javascript, and plugin content for
> modifications. Because of the dynamic nature of most of the web, the
> scanner has a number of mechanisms built in to filter out false
> positives that are used when a change is noticed between Tor and
> Non-Tor. 

As an eventual feature, for the above tests, it probably makes sense
to be able to imitate a few different popular browsers as the scanner
does its checks.  If an adversary can recognize the scanner, it can
MITM everything _but_ the scanner.

 [...] 
> 3. Automatic BadExit Marking:
> 
>   In the final stage, the scanner will begin marking exits depending
>   on the failure severity level in one of three different ways: by
>   node idhex, by node IP, or by node IP mask. A potential fourth, less
>   severe category of results may still be delivered via email only for
>   review.
>  
>   BadExit markings will be delivered in batches upon completion
>   of whole-network scans, so that the final false positive
>   filter has an opportunity to filter out URLs that exhibit
>   dynamic content beyond what we can filter.

These batches should include descriptions of what the node seems to
have done and when.

> Specification of Exit Marking:
> 
> Technically, BadExit could be marked via SETCONF AuthDirBadExit over
> the control port, but this would allow full access to the directory
> authority configuration and operation.

Agreed.

> The approved-routers file could also be used, but currently it only
> supports fingerprints, and it also contains other data unrelated to
> exit scanning that would be difficult to coordinate.
> 
> Instead, we propose that a new badexit-routers file that has three
> keywords:
> 
>   BadExitNet 1*[exitpattern from 2.3 in dir-spec.txt]
>   BadExitFP 1*[hexdigest from 2.3 in dir-spec.txt]
> 
> BadExitNet lines would follow the codepaths used by AuthDirBadExit to
> set authdir_badexit_policy, and BadExitFP would follow the codepaths
> from approved-router's !badexit lines.

(What's the third keyword?)

If possible, I'd like the approved-routers format to expand to also
include IPs, networks, and fingerprints.  Then I'd like a config
option to allow an authority to have more than one
approved-routers-ish file.  This way, we wouldn't need to add new
special handling for every flag we decide to set automatically via an
external tool.

Also, it would be neat to migrate the syntax something that doesn't
!have !so !many !bangs.  Maybe we should just deprecate the
approved-routers syntax, and have the syntax for the new files be
sane.

Everything in the last two paragraphs should be dead easy to hack in
during the 0.2.2.x series.

-- 
Nick