IOCP / C10K

[Toby Douglass]

> On Wed, Mar 07, 2007 at 12:44:45PM -0000, Toby Douglass wrote:

>> They know about it.  I've removed them from the CC:.  The problem is
>> that
>> the way libiocp is written is, as I understand it, not particularly
>> compatable with the way libevent is written.  Merging the two is not an
>> obvious proposition.  How focused is the Tor code on libevent?  is it
>> only used for sockets?

> The right thing to do when you need to support multiple IO frameworks
> is usually to think of some abstraction that works with all of them,
> implement that abstraction with all of them, and then write your other
> code to use that abstraction.  To get IOCP used by Tor, the first step
> is to find some abstraction that both IOCP and libevent can provide,
> and then get Tor to use that abstraction for its network operations.

That means fitting IOCP into libevent.

> Right now, Tor uses libevent for networking stuff mainly for
> notification of which sockets are readable and writable.  It seems
> like IOCP is better at determining when reads and writes are done.
> If this is the case, it seems way easier to use the event_base-based
> functions in libevent to implement something like IOCP's interface
> than to use IOCP to implement something like the event_base functions.

This may be impossible.

I'm not familiar with libevent, but if it actually has callers waiting, in
a sleep, for events to occur, then all the event handling under Win32 will
need to be rewritten to use I/O completion events.  I think it is not
possible to use IOCP in conjunction with normal Wait() type functions,
which I suspect libevent must ultimately rest upon.

> In fact, there's already a good candidate: IOCP looks like a closer
> fit for libevent's bufferevent interface than for its regular
> event_base interface.  (Bufferevent is currently written to assume
> it's working using an existing event_base, but there's nothing
> stopping somebody from putting a parallel implementation into place.)
> If we can get better windows performance by using bufferevent, I bet
> that would be the cleanest way to go about this.

I will try to examine the libevent code and form an opinion.