[tor-dev] Proposal Waterfilling

Fri Mar 9 16:21:14 UTC 2018

Hi all :)

On 2018-03-08 00:31, A. Johnson wrote:
>
>> On Mar 7, 2018, at 5:12 PM, Florentin Rochet 
>> <florentin.rochet at uclouvain.be 
>> <mailto:florentin.rochet at uclouvain.be>> wrote:
>>
>> Hello,
>>
>>
>> On 2018-03-07 14:31, Aaron Johnson wrote:
>>> Hello friends,
>>>
>>>> 1) The cost of IPs vs. bandwidth is definitely a function of market
>>>> offers. Your $500/Gbps/month seems quite expensive compared to what
>>>> can be found on OVH (which is hosting a large number of relays): they
>>>> ask ~3 euros/IP/month, including unlimited 100 Mbps traffic. If we
>>>> assume that wgg = 2/3 and a water level at 10Mbps, this means that,
>>>> if you want to have 1Gbps of guard bandwidth,
>>>> - the current Tor mechanisms would cost you 3 * 10 * 3/2 = 45 
>>>> euros/month
>>>> - the waterfilling mechanism would cost you 3 * 100 = 300 euros/month
>>>
>>> The question of what the cheapest attack is can indeed be estimated by
>>> looking at market prices for the required resources. Your cost
>>> estimate of 3.72 USD/Gbps/month for bandwidth seems off by two orders
>>> of magnitude.
>>>
>>
>> Let me merge your second answer here:
>>
>>> I see that I misread your cost calculation, and that you estimated 
>>> $37.20/Gbps/month instead of $3.72/Gbps/month. This still seems low 
>>> by an order of magnitude. Thus, my argument stands: waterfilling 
>>> would appear to decrease the cost to an adversary of getting guard 
>>> probability compared to Tor’s current weighting scheme.
>>
>> There is still something wrong.
>
> What’s wrong? $37.20Gbps/month = 30 Euros/Gbps/month, which is what 
> you are claiming.

I am sorry, "wrong" was a bad chosen word. It is just that we are not 
comparing the same bandwidth. What is written above is 1Gbps of *guard* 
bandwidth, which means 1,5 Gbps of bandwidth due to the 2/3 ratio on 
vanilla Tor. Either one are fine, but since we started with 1Gbps of 
*guard* bandwidth, let's keep using this baseline not to get confused :)

> This would be the lowest price for a sustained Gbps transfer by a 
> significant margin among all of the deals that have appeared on this 
> thread. The other lowest was from Alex, who found $100/Gbps/month. I 
> somewhat doubt that you could actually achieve 1Gbps sustained for 30 
> Euros/month on a shared VPS or that OVH would actually tolerate using 
> this much bandwidth at this little cost.

Rob and s7r also raised the same argument. So, let me share my complete 
experience regarding this topic:

I decided some time ago to invest 500$ in running relays, I did some 
research to look for the cheapest offers and also to try to setup my 
relays in different AS, if possible. I did find some interesting deals 
in different countries, with different providers and I made a list to 
try them all. All of the deals were quite similar: 100 Mbits unlimited, 
at an insane low price. So insane that I was suspicious as you are all. 
I started my relays and got a few bad experiences that I can list here:

- One of the deal was 50€/year for an unlimited 100Mbits in Sweden. 
After 3 or 4 weeks, my access got simply revoked with no warning or 
message. I contacted the support and got some clumsy arguments about the 
fact that I was running an hacking tool. Needless to say, the probable 
reason was my bandwidth consumption.
- Another one was an unlimited 100 Mbits in UK for 4pounds/month. The 
first few days were nice, relaying ~70Mbits. Then I got throttled to 
8Mbits until the end of the month.
- Another one was a reseller. I managed to run 200Mbits during a few 
days of Exit bandwidth on 1 machine, for less than 8€/month. Then, my 
access were revoked due to some external complain. The funny things was 
that I did ask if I could run an Exit Tor relay before and the support 
answered that they had no problems with Tor relays.

The list can go on, I had the same kind of problems with other 
providers. All of them have something is common, they are all small 
companies using what Rob said "unlimited bandwidth as marketing term".

Hopefully I had some good experience too (all of them are exit relays):

- I run a few relays at OVH (France, Poland), 100 Mbits for 3€/month 
like the offer linked in this thread. A different datacenter for each. 
No complain from the provider and the relays are used since months.
- I run one unlimited 100Mbits relay in Moldova since months
- I run one unlimited 100Mbits relay in Canada since months

Now, If we take the /16 prefix of the IP I got from my 3 OVH European 
relays: "54.37", "137.74", "145.239", and if we do some atlas relay search:

https://metrics.torproject.org/rs.html#search/137.74
https://metrics.torproject.org/rs.html#search/%20%0954.37
https://metrics.torproject.org/rs.html#search/145.239

All relays appearing to advertise around 10~12 MiB/s are *probably* the 
offer I linked in this thread. These relays even have a huge consensus 
weight :(.

Moreover, there is some people running more than 1Gbps with this method, 
such as this relay operator: 
https://metrics.torproject.org/rs.html#details/117B99D5CE22174DEA7F1AD3BE25ECE993F486B5 
and this guy is doing it with the price I gave above :)

So why is it working? I come up the following conclusion: OVH is a big 
enough company not to lie with "unlimited, unmetered 100Mbits". I did 
not try other big providers, but that would be likely the same result.

Conclusion: we can run many Gbps of bandwidth with the price I gave 
above, for now.

> It would at least be a notable new record for the cheapest possible 
> Tor bandwidth, as far as I can tell.
>
>> With Waterfilling, we assume above a water level of 10 Mbits, so we need:
>>
>> 100 VPS SSD 1 relaying 1Gbps at the guard position, which the cost turns
>> to be 3*100 = 300 euros/month.
>
> This calculation is much too kind to waterfilling :-) Why pay for a 
> full 100Mbps with only 1 IPv4 address when you only need 10Mbps/IP to 
> achieve the same guard probability? Earlier I showed an example of a 
> cheaper VPS (https://my.hiformance.com/cart.php?a=add&pid=165) that 
> appears to provide for just $0.63/month a VPS with an IPv4 address 
> that is capped at 6Mbps sustained througput. This would be a more 
> economical way (3.5x cheaper) to attack waterfilling.

Yes, you are right. This is insane price and theoretically stronger 
against Waterfilling. But let me count the number of relays needed to 
achieve, let's say 10% of bandwidth with that provider, and let's 
suppose 10% is 15 Gbps 
(https://metrics.torproject.org/bandwidth-flags.html). Waterfilling 
reduces the bandwidth that the adversary needs by (currently) a 2/3 
ratio. So, the adversary needs 10 Gbits:

10000/6 = 1666 relays.

 From this number, I wonder the following things:

Can an adversary puts 1666 Guard relays in the network such that this 
community would not notice that something strange is happening? Given 
the fact that we don't even have 2000 Guards by now.

Does the provider have enough IPv4? Are they the same /16?

Would it be as compliant than OVH?

Given those numbers, is it a good thing to reason over security with 
money only?

> Alternatively, I bet you could get bulk IPv4 addresses for much 
> cheaper than the $3/month that OVH charges for its SSD VPS, which you 
> could then potentially apply to your 100Mbps (or larger) server to get 
> 10Mbps and more cheaply attack waterfilling. For example, OVH provides 
> 256 IP addresses for its dedicated servers at no monthly cost 
> (https://www.ovh.co.uk/dedicated_servers/details-servers-range-GAME-id-MC-64-OC.xml). 
> These servers can be had for at least 55 euros/month, which provides 
> 500Mbps unlimited. With two of those, you could achieve the above 
> attack on waterfilling for 110 euros = $136.36/month instead of 300 
> euros/month = $371.92/month.

You're right. But you're also having the same /24 for all your relays 
running on this machine. Some easy rule on the directory server can 
prevent this to happen. Limiting the number of relays over a same /24 
for example.

> Once we’re talking about trying to achieve a large fraction of the Tor 
> network, which requires many Gbps in vanilla Tor, the fixed cost of a 
> server becomes a smaller fraction of the total cost and the savings 
> from the free extra IPs become greater.
>
>> That depends on the kind of policy that the Tor network could put in
>> place. If we decide that large families become a threat in
>> end-positions, we may just aggregate all the bandwidth of the family,
>> and apply Waterfilling. That would not kick them off, but would create a
>> kind of 'quarantine'. Same kind of suggestion than the one just below.
>
> This seems to be a different argument than you were making, which was 
> that the many servers must appear to be run independently, which I 
> still disagree with.
>
>> This is what Waterfilling does: increase the cost of a well-defined
>> attacker and offer clients to choose into a more "diverse" network.
>
> Sorry, I still don’t agree. It increases the cost in terms of number 
> of IP addresses required and causes clients to spread out more across 
> guards with different IP addresses. This is a narrow notion of 
> diversity and not one that I think is useful as a design principle.

I agree that this is a narrow notion of diversity. Waterfilling is 
currently applied over IP, but this is not a *mandatory* design. What 
Tor does now, is an attacker-agnostic balance of bandwidth. Waterfilling 
should be seen as a technique that allows to take into account an 
attacker in the balance of the network. It can be applied with a wider 
notion of diversity and security, as we already outlined.

I hope it helps and many thanks for your comments :)

Best,
Florentin

>
> Best,
> Aaron
>
>
> _______________________________________________
> tor-dev mailing list
> tor-dev at lists.torproject.org
> https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.torproject.org/pipermail/tor-dev/attachments/20180309/2f2bba81/attachment-0001.html>