[tor-dev] prop224: HSDir caches question with OOM

[Yawning Angel]

On Sat, 16 Apr 2016 20:30:29 +0300
s7r <s7r at sky-ip.org> wrote:
> I agree that teor's O(Kn) is the best approach from performance (no
> additional memory allocations), simplicity and efficacy point of view.
> O(Kn) algorithm will clear the entries only based on their expiration
> time, it won't care to clean the v2 / v3 caches in equal measure which
> is good, given that we do not know how long HS operators will take /
> need to upgrade their services to prop 224.
> 
> The tap vs ntor situation was a good measure, but the threat model was
> different (we were trying to ensure new clients using ntor get
> resources from relays with priority as opposite to non-updated botnet
> zombies using tap). In the current situation we care about v2 and v3
> HS caches exactly the same, for an unknown period of time which might
> not be short, so we shouldn't penalize v2 in any way.

We do?  I can think of numerous reasons as to why v3 is objectively
better, and why we should incentive people to move towards 224 style
HSes...

What I want is possible with Teor's 2nd algorithm (which is adequate).
Make Cache A the v2 cache, and if we've freed enough ram after purging
expired entries from A in a given time period, we terminate rather than
touching cache B.

In effect this makes it slightly less likely for entries in the B cache
to be deallocated (we probably should also halt once we have reached
our target deallocation amount when attempting to compact either cache,
rather than continuing to purge the rest of the time interval, it's
the OOM handler, and an addition, a compare and a branch is cheap
enough to be done in a loop).

> This needs to be covered regardless how often a HSDir has its OOM
> triggered. I don't think we should assume it's hard to flood HSDirs
> with descriptors until the memory is full.
> 
> Now that HSDirs will need to handle two caches, is 20% of the total
> memory allocated for HS descriptors a good value? What harm would
> increasing it to let's say 25% do?

It would decrease memory available for other things, where other things
is "all HSDirs are expected to first and foremost be relays, and do
normal relay things like relaying traffic, before being HSDirs".

I don't remember if it's possible to opt out of being a HSDir or not
(likewise, with the move to make all relays be dir caches, we already
are increasing memory pressure on "things that are comically undersized
that shouldn't ever be HSDirs or DirCaches in the first place")....

Regards,

-- 
Yawning Angel
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 819 bytes
Desc: OpenPGP digital signature
URL: <http://lists.torproject.org/pipermail/tor-dev/attachments/20160416/307688e6/attachment.sig>