[tor-dev] Revisiting prop224 time periods and HS descriptor upload/downloads

Thu Apr 14 15:18:55 UTC 2016

On 13 Apr (15:34:54), George Kadianakis wrote:
> David Goulet <dgoulet at ev0ke.net> writes:
> 
> > [ text/plain ]
> > On 12 Apr (16:01:32), George Kadianakis wrote:
> >> David Goulet <dgoulet at ev0ke.net> writes:
> >> 
> >> > [ text/plain ]
> >> > On 11 Apr (14:42:02), George Kadianakis wrote:
> >> >> David Goulet <dgoulet at ev0ke.net> writes:
> >> >> 
> >> >> > [ text/plain ]
> >> >> > On 04 Apr (19:13:39), George Kadianakis wrote:
> >> >> >> Hello,
> >> >> >> 
> >> >> >> during March we discussed the cell formats of prop224:
> >> >> >>   https://lists.torproject.org/pipermail/tor-dev/2016-March/010534.html
> >> >> >> 
> >> >> >> The prop224 topic for this month has to do with the way descriptors get
> >> >> >> uploaded and downloaded, how this is scheduled using time periods and how the
> >> >> >> shared randomness subsystem interacts with all that.
> >> >> >> 
> >> >> >> <snip>
> >> >> >> 
> >> >
> >> > The logic I sketched out above makes it that we would need parameters (from
> >> > the consensus) like so (or hardcode them):
> >> >
> >> > - TIME_PERIOD_ROTATION_TIME (currently 12:00)
> >> >
> >> 
> >> [Second email with some more thoughts]
> >> 
> >> BTW, currently in prop224 the TIME_PERIOD_ROTATION_TIME is at 00:00 because of
> >> the following paragraph:
> >> 
> >>    Time periods start with the Unix epoch (Jan 1, 1970), and are
> >>    computed by taking the number of whole minutes since the epoch and
> >>    dividing by the time period. So if the current time is 2013-11-12
> >>    13:44:32 UTC, making the seconds since the epoch 1384281872, the
> >>    number of minutes since the epoch is 23071364.  If the current time
> >>    period length is 1500 (the default), then the current time period
> >>    number is 15380. It began 15380*1500*60 seconds after the epoch at
> >>    2013-11-11 20:00:00 UTC, and will end at (15380+1)*1500*60 seconds
> >>    after the epoch at 2013-11-12 21:00:00 UTC.
> >> 
> >> I wonder what's the best way to change this to start at 12:00. 
> >> 
> >> We could in theory compute the "number of whole minutes since the epoch plus 12
> >> hours" and use that in the division, but that would be a bit ugly... Is there a
> >> more elegant thing to do?
> >> 
> >> We could also in theory change the shared random value generation to happen at
> >> 12:00, and then have TIME_PERIOD_ROTATION_TIME naturally start at 00:00, but
> >> this requires changing prop250. Could it be worth it? :/
> >
> > I think it's fine we keep the start time at 12:00 here. It's just an offset
> > from the start of the epoch. Furthermore, adding a "rotation time" makes it
> > that we we can control where everything started which doesn't have to be the
> > epoch time at 00:00.
> >
> > We can find the start of the TP with those two (rotation time and lifetime)
> > and then divide that time value by the lifetime to get the nth time period.
> >
> > Also, controling the rotation time is good to have for chutney testing with
> > much more smaller timings.
> >
> 
> OK, I posted a torspec branch with some initial changes based on the
> discussions of this thread at 'prop224-timeperiods-1':
>             https://gitweb.torproject.org/user/asn/torspec.git/log/?h=prop224-timeperiods-1
> 
> Specifically, wrt time periods and the start time, I introduced a "rotation
> time offset" of 12 hours to the epoch calculation. You can see it here:
>      https://gitweb.torproject.org/user/asn/torspec.git/commit/?h=prop224-timeperiods-1&id=6499cbf1023806f75d567503ad0ab8625a3876a7

Looks good!

> 
> Please let me know if the technique can be simplified or improved or needs better wording.
> We need this mechanic to be clear and easy to understand and implement!

I find it pretty straight forward. Bottom line, we add an offset to accomodate
our time period.

> 
> >> 
> >> <snip>
> >> 
> >> 
> >> Currently the proposal says:
> >> 
> >>    Hidden service directories should accept descriptors [...] and retain them
> >>    for at least [TODO: how much?] minutes after the end of the period.
> >> 
> >> but that means that HSDirs need to keep track of when the period ends, and
> >> whether a descriptor was uploaded for the current time period or for the
> >> overlay period...
> >
> > I think this will make things much more complicated. IMO, the HSDir should
> > _only_ rely on the revision counter and an expiry time and not trying to try
> > and guess the lifetime of a descriptor from the service perspective.
> >
> > However, here is an idea. Considering teor's argument about HS fingerprinting,
> > we should make the upload happen regurlarly so having RendPostPeriod
> > customizable by an operator is probably a bad idea. We kind of need that _all_
> > HS expect to behave the same in normal circumstances with their HS desc
> > uploads. Thus, if the RendPostPeriod would become hardcoded (or consensus
> > params), the HSDir would know that the HS is "gone" or have rotated HSDirs
> > after let's say 4 * RendPostPeriod (4 here is abritrary) of not seeing a new
> > revision counter. This makes a cache entry lifetime much smaller!
> >
> > In the end, I see either we use an expiry time that is the _maximum_ lifetime
> > a descriptor can have or an expiry time that is based on the expected maximum
> > time frame we should have received a new revision counter. The latter is fun
> > because we don't need to consider client clock skew since if we happen to
> > purge the descriptor from the cache it's because we think the service is gone
> > or has rotated HSDir thus any client coming our way would fail in the first
> > place to connect to the service by getting the descriptor that should have
> > been updated X hours ago.
> >
> 
> 
> Here is another important commit that specifies the overlap period functionality:
>     https://gitweb.torproject.org/user/asn/torspec.git/commit/?h=prop224-timeperiods-1&id=737b1f2bcccce1d632e43ba6e395f24f0af8a65c
> 
> It also adds the following section:
> 
> +   HSDirs MUST retain hidden service descriptors for 33 hours before expiring
> +   them. That's 24 hours for the time period duration, plus 6 hours for the
> +   maximum overlap period span, plus 3 hours for the maximum acceptable client
> +   clock skew.
> 
> +   Hidden services should keep their old introduction circuits open for at
> +   least 3 hours after descriptor expiration, so that clients with skewed
> +   clocks can still visit them through outdated descriptors.
> 
> This implements the naive cache lifetime mechanic we discussed in this
> thread. That's an improvement over the empty TODO section of the current
> prop224 but maybe we can do better: we should think whether we want to do more
> advanced HSDir heuristics like "If I'm an HSDir and I don't receive an HS
> descriptor for N hours, consider that HS dead". Or maybe we should add
> valid-until fields to hidden service descriptors. Thoughts?

I'm good with this. Let's start with a baseline (and also almost same behavior
we have now) and we can improve as we go. In my wildest dream, we get the
HSDir side merged in 029 so we basically have until Sept. to figure that out.

> 
> Here are some things left to be done:
> 
> - Specify *when* hidden services upload descriptors. Do they do it hourly, or
>   only when a change has happened. Both approaches leak information to the
>   HSDir (the former leaks uptime, the latter leaks intro point changes).

I'm more comfortable right now with the HS uploading every RendPostPeriod
(default: 1 hour). Even if the descriptor content doesn't change, it should
increment the revision-counter.

I'm more and more convinced that making RendPostPeriod _not_ configurable is
also something we should do and thus allowing us to have the HSDir use that
value instead (maybe).

> 
> - Specify behavior of hidden services and clients with regards to time periods
>   and the use of SRVs as discussed in
>     https://lists.torproject.org/pipermail/tor-dev/2016-April/010757.html

I'm happy with your approach if you find mine a bit more complicated. They
both result in the _same_ behavior anyway. Altough, in terms of code, for each
SRV value, we need to keep the valid-after and the valid-until time in our SRV
data structure (which we don't right now with current prop250 code).

> 
> - Further specify descriptor caching behavior of HSDirs.

Can you elaborate here? Detailing expiry time? Conditions to replace a cache
entry? ...?

> 
> Did I forget anything?
> 

I think all discussions have been covered! Great work! Thanks!

David
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 603 bytes
Desc: not available
URL: <http://lists.torproject.org/pipermail/tor-dev/attachments/20160414/0490a9ee/attachment.sig>