[tor-dev] When RFC 7686 and transparent proxies collide

Fri Nov 3 16:29:57 UTC 2023

On Thu, Sep 14, 2023 at 08:22:22PM +1000, Matt Jolly wrote:
> I recently inadvertently opened a much larger can of worms than I'd
> intended when fixing a bug reported downstream where cURL would,
> when configured with certain DNS backends, fail to resolve .onion
> addresses.
>
> [...]

Hi Matt,

Thanks for the detailed problem description and references, and I'm sorry to
hear about the trouble you're getting when trying to fix it.

> I've asked in the Tor Specifications issue (inspired by Silvio's
> suggestions), and in the cURL issue, but I seem to be getting nowhere
> and the impacted users are clamouring for a quick band-aid solution,
> which I feel will work out worse for everyone in the long run:
> > How can client applications (safely):
> > 
> > 1.discover that they're in a Tor-enabled environment
> > 2.resolve onion services only via Tor in that circumstance
> > 3.not leak .onion resolution attempts at all
> > 
> > Right now, not making these requests in the first place is the
> > safest (and correct) thing to do, however inconvenient it may be.
> > Rather than immediately trying to come up with a band-aid approach
> > to this problem, a sane mechanism needs to be implemented to:
> > 
> > 1.prevent each application from coming up with their own solution
> > 2.prevent inconsistency in .onion resolution (i.e. no "oh it only
> > leaks if DO_ONION_RESOLUTION is set")
> > 3.provide a standardised mechanism for applications that want to be Tor
> > aware to discover that they're in a Tor-enabled environment.
> I'm not particularly attached to that last point, but it's worth discussing.
>
> On a related note:
> -is the use of a transparent proxy recommended?

I consider it a valid use case. The only concern I'm aware is about
proxy leaks. I don't see any reason not to recommend transparent Tor
proxies if using a configuration that avoids such leakage.

> -is there a sane alternative that involves as minimal configuration
> as possible for these users?

Can't think of anything right now besides what was already covered in
length in the threads you mentioned. Instead, I'll focus in the overall
problem.

> I'm not sure what the best way forward is here, but I'm hoping that
> actual Tor developers might have a useful opinion on the matter, or
> at least be able to point me in the right direction.

The long analysis below does not necessarily represents an "official Tor
position". It may be frustrating that there is no "official" position
right now, maybe because this still need further discussion, and also
because it may be a hard topic.

# Interpreting RFC 7686

Let's start with an hermeneutical exercise, by reading carefully what
RFC 7686 says:

> 2.  Application Software: Applications (including proxies) that
>     implement the Tor protocol MUST recognize .onion names as special
>     by either accessing them directly or using a proxy (e.g., SOCKS
>     [RFC1928]) to do so.  Applications that do not implement the Tor
>     protocol SHOULD generate an error upon the use of .onion and
>     SHOULD NOT perform a DNS lookup.

Applications MUST support .onion either directly or through a proxy. The RFC
gives SOCKS as an example, but it could as well have mentioned transparent
proxies. So applications could easily allow environment variables, command line
flags, configuration parameters or "environment detection" to indicate that
they support .onion domains.

I don't see why an upstream shouldn't allow users and operating systems
toggle the behavior regarding RFC 7686. Seems like the best default
approach so far is to generate an error, but nothing is blocking
applications to expose ways users can indicate that their systems acts
as proxies for .onion.

I suggest we take a corollary from this RFC portion: applications implementing
the Tor protocol are those which either access .onion addresses directly or use
a proxy that supports Tor. If the application does not support any way to
properly resolve .onion addresses, then it must make sure to refuse any
attempts to resolve .onion addresses. Nothing is blocking an application to
operate in two modes: rejecting .onion resolution attempts by default, except
if it's told to do so.

I agree that "implement the Tor protocol" is rather vague. The "Tor protocol"
might mean a lot of stuff, but I'm assuming here the _intention_ behind this
statement is this: in order to build a Tor-enabled application, one has to
interface with Tor somehow (being a Tor client or using a proxy).

As for curl, it seems to me that upstream wants uniform behavior with all DNS
backend libraries. And c-ares is rejecting .onion resolution, so for the sake
of uniformity curl is just rejecting it for all backends.

But the RFC explicitly mentions "Applications" and not "Libraries". Shall we
assume then that the RFC is referring to curl, but not to c-ares?

Would c-ares maintainers be free do follow whatever they prefer?

Or this is not a concern for c-ares, since it's a library and not an
application? In this case they don't need to comply with the RFC, and
could just rollback this change, so curl maintainers could be more open
to accept toggling the RFC behavior.

We could say that users interact with applications, and these rely on
libraries. By the RFC interpretation above, it's in the application level that
.onion resolution should be allowed or not. It just don't make any statement
about what libraries SHOULD or SHOULD NOT do, or even how a system resolver
should behave.

I don't really think curl maintainers should accept a toggle only if c-ares is
rolled back, but maybe this could help change their minds.

Alternatively, c-ares could provide some optional initialization
parameter indicating it should actually try to resolve an .onion
address, if that's feasible.

Also note the use of SHOULD in the RFC. As of BCP 14 / RFC 8174:

> 3. SHOULD   This word, or the adjective "RECOMMENDED", mean that there
>    may exist valid reasons in particular circumstances to ignore a
>    particular item, but the full implications must be understood and
>    carefully weighed before choosing a different course.
>
> 4. SHOULD NOT   This phrase, or the phrase "NOT RECOMMENDED" mean that
>    there may exist valid reasons in particular circumstances when the
>    particular behavior is acceptable or even useful, but the full
>    implications should be understood and the case carefully weighed
>    before implementing any behavior described with this label.

From the discussion so far, there are enough "valid reasons in particular
circumstances" when things can be done differently. I haven't followed RFC
7686's process, but maybe the authors and reviewers had carefully chosen the
terminology to acknowledge that there might be special cases -- such as
transparent proxies -- where the RFC requirements can be relaxed, given that
implementors know what they're doing.

Hope this gives enough arguments to argue for implementing a tunable to account
for the transparent proxy use case in curl and in other software.

# Standardization

What seems more difficult is to standardize a way that any application from any
operating system could follow some setting, or auto-detect it's behind an
.onion-enabled proxy.

I don't see an easy way to solve this that would make everyone happy in the
short term.

But it would be possible to split proposals by timeframe (short, mid and long
term) and scope (specifications, libraries, applications, operating systems).

Short timescales and "band-aid" fixes usually happen within libraries and
applications.

Operating system changes are somehow mid-term.

Tor specs ranges from quick fixes to mid or long term.

IETF standards are long term. Could take years to have a RFC amendment,
and years more to implementors follow the it. RFC 7686 could have
improved wording and be more detailed or nuanced in the possibilities.
Right now we don't have any concrete plans or people engaged in amending
the RFC, but here and there people are discussing this. Example:
https://gitlab.torproject.org/tpo/onion-services/onion-support/-/issues/85

As for the quick fixes, I don't see nothing beyond advocating patches
implementing flags, environment variables, parameters etc for a RFC
7686-compliant tunable. Unfortunately this have to be done in a case-by-case
basis, but having RFC 7686 may actually be an advantage when requesting such
changes. I imagine that not having the RFC would make way harder to propose
anything related.

# Approaches for transproxy auto-discovery

Now let's talk a bit about long-term, idealized fixes.

What follows is just brainstorming.

It could be considered some method for transproxy auto-discovery, be it
some widespread standard or only followed by a subset of related
applications and operating systems.

An .onion auto-detection feature could be implemented on applications
by trying to detect some special-use domain like "proxy.onion"
in two ways:

1. By supporting a local hostname similar to "localhost" in many systems. If a
   name "proxy.onion" resolves locally (say to 127.0.0.1), then an
   application (or library) could assume it's behind a proxy, be it SOCKS5h or
   in a system with Tor transparent proxy.

2. By having an .onion-aware DNS resolver return a valid entry for
   "proxy.onion":
  * That won't leak the address, but may leak to the DNS that someone
    is using a given library, which may be harmless if it's widespread.

  * The query could be for a TXT record, which could give more space for
    custom replies, but could be incompatible with the option 1 above.

Upon initialization, an application could do this test to determine
whether it should try to resolve .onion addresses.

The Tor client would have to change as well, in order to support the special
"proxy.onion" address instead of triggering a malformed .onion address error.

> Thanks for your time,
> Cheers,
> Matt

Thanks for all your effort and dedication put into this issue :)

-- 
Silvio Rhatto
pronouns he/him
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 963 bytes
Desc: not available
URL: <http://lists.torproject.org/pipermail/tor-dev/attachments/20231103/91ffea70/attachment.sig>