Support Windows

[carllerche]

Overview

Currently, Mio currently only supports Linux and Darwin platforms (though *BSD support could happen relatively easily). It uses epoll and kqueue respectively to provide a readiness API to consumers. Windows offers a completion based API (completion ports) which is significantly different from epoll & kqueue. The goal would be to tweak Mio in order to support Windows while still maintaining low overhead that mio strives for across all platforms.

History

I have wavered a bunch on the topic of how to best support Windows. At first, I had originally planned to do whatever was needed to support windows even if the implementation was less than ideal. Then, started towards not supporting windows with Mio and instead provide a standalone IO library that supported windows only. I started investigating the IOCP APIs in more depth and thinking about what a windows library would look like and it was very similar to what mio already is.

Completion Ports

There are a number of details related to using completion ports, but what matters is that instead of being notified when an operation (read, write, accept, ...) is ready to be performed and then performing the operation, an operation is submitted and then completion is signaled by reading from a queue.

For example, when reading, a byte buffer is provided to the operating system. The operating system then takes ownership of the buffer until the operation completes. When the operation completes, the application is notified by reading off of the completion status queue

Strategy

The strategy would be to, on windows, internally manage a pool of buffers. When a socket is registered with the event loop with readable interest, a read a system read would be initiated supplying an available buffer. When the read completes, the internal buffer is now full. The the event loop would notify readiness and the user will then be able to read from the socket. The read would copy data from the internal buffer to the user's buffer and the read would be complete.

On write, the user's data would be copied to a an internal buffer immediately and then the internal buffer submitted to the OS for the system write call.

Mio API changes

In order to implement the above strategy, Mio would not be able to rely on IO types from std::net anymore. As such, I propose to bring back TcpStream and TcpListener implemented in mio::net. Since Mio will then own all IO types, there will be no more need to have the NonBlock wrapper. Also, it seems thatNonBlock` can be confusing (see #154). So, all IO types in mio will always be blocking.

I believe that this will be the only required API change.

[retep998]

So effectively Mio would read in the background whenever the buffers for a handle are empty? Would it also read when the buffers are only partially empty? Would there be a cutoff level for when it reads more?

[carllerche]

@retep998 I think there will be some experimentation around that and possibly some tunable params. I was thinking of defaulting to 4kb per socket. If oneshot is requested, then only one read will be queued. Otherwise, it will attempt to keep the buffer full. There probably should be a heuristic as to when to start a new read if the buffer was only partially consumed. For example, if the 4kb buffer is full and the user reads 1 byte, it probably doesn't make sense to attempt another system read of 1 byte. Maybe a good default would be 1KB? So, once the internal buffer has 1KB available, Mio attempts to fill it.

[jnicholls]

The idea of internal buffer management is a good one for sticking with the current Stream API. But the data copies would make Windows performance second-class unnecessarily. In truth, IOCP and RIO are actually superior models to epoll/kqueue that reduce CPU/syscalls. It would be a shame to have to do a 1-copy of the data.

I think it would be wise to keep the (Try)Read interface on top of the Windows streams, but also offer them access directly to the internal buffer and take ownership of it if desired. When they take ownership, Mio will allocate a replacement. I'd like to think that is a far more efficient model. It does diverge the API surface but that's where we are.

The only way to keep the API surface consistent while not sacrificing performance is to emulate the IOCP model on top of epoll/kqueue, which is easy to do. The reverse is not. E.g. the Handler would send buffers of data received or return buffers of data to write, rather than notifications of readiness.

I personally am not concerned about Windows support, but many others are. I think this will be an important decision for mio. If we go with the copy model, I would expect there to be a fork of mio or a second project spun up to compete.

[jnicholls]

Some more research may want to go into Windows RIO (Registered IO) and think about explicitly targeting that instead of overlapped IO. RIO supports different notification mechanisms (polled, evented, and IOCP) and thus it might fit more nicely into the current Mio readiness model. If that is the case, I honestly don't see a problem with only supporting Windows 8+. It is the future, and this is a green field lib :)

[retep998]

Perhaps a feature flag to distinguish between using an older more compatible but less efficient IOCP model, or using the fancier RIO model?

[jnicholls]

Yeah definitely, if RIO ends up making sense to support. I'll dig in and get back with some details.

[dpc]

It seems to me having Windows support (even performance-impaired) is a good thing, so that any software written on top of mio (even the one that is not really performance critical) is portable. So the plan looks OK for stuff I'm interested in.

Give the fundamental differences between asynchronous I/O handling models, it might be a job for higher level libraries and software to switch between different mio-like libraries for different platforms. Unless of course someone figures out a really neat unified model.

[jnicholls]

Yeah that's fair. I do believe it will inspire another Rust I/O library that has the unified model. If Mip maintainers are okay with that, then this plan is the easiest path forward.

There is a unified model; the question is whether Mio wants to break its current model, or not.

[dpc]

Is there really an unified model? It seems to me that it's not really possible, and eg. RIO requires registering memory beforhand which does not seem to work well with Unix-like IO.

[jnicholls]

RIO wouldn't be a part of a unified model. RIO is an optimization over traditional overlapped IO where the buffers are registered ahead of time, like you said. This reduces syscalls & allocs, and thus CPU.

A unified model wouldn't need RIO. IOCP is fine, and it's already a superior model over epoll/kqueue as it stands (zero-copy, fewer syscalls, etc.). Given that, unified model absolutely exists: the IOCP model. Emulating IOCP on top of epoll/kqueue is absolutely doable; the reverse is not. Said plainly, the model is asynchronous I/O: the caller invokes an IO action like read/write, we take ownership of the buffer and perform the action, and notify the caller when we've completed the action. This is exactly what libuv does to abstract async IO over IOCP, epoll and kqueue.

Doing the above would be a big change for Mio. It would require state management of buffers, fds, requests, and callbacks (unless a single trait-based handler is sufficient for the notifications). Mio is currently a minimal abstraction with zero-allocations and no state management. Windows support will change that one way or another.