Wasip3: Implement pipe, testing will need cooperative threads though

[cpetig]

This works, if

• you also combine it with cooperative threading
• you then don't close any of the file descriptors on either side (because there is no other process)

It was a nice dream to see this in close reach, though.

[cpetig]

Another part of #711

[alexcrichton]

Thinking more about this, I'm wondering if this makes sense for wasip3. The main disconnect I can think of with this implementation and what Linux has, for example, is that there's no buffer between the read/write ends. I think linux is something like 4k or 8k by default, and I believe this buffer is managed by the kernel such that small writes will typically immediately return. That's in contrast with using stream<u8> where small writes will block until a read rendezvous with them.

To me this might be a significantly enough of a departure from preexisting semantics to break many programs. One option perhaps is to manage a buffer internally in wasi-libc to recover similar semantics, but that's a much more involved implementation. What do you think about this though? I presume this has helped port things for you, so this may also be less of a concern than I think

[cpetig]

With multiple threads handling network traffic in C++, pipes are the most portable way to implement an event mechanism to activate worker threads. I guess this could be in scope for wasip3 and the absence of a buffer shouldn't affect the user.

I later realized that perhaps some reference counting logic could be devised that the first call to close would not destroy the descriptor, so running unmodified code could be possible. But still it feels like I need to make assumptions of use for pipe.

[alexcrichton]

Yeah I don't mean to belittle the usage/importance of pipe, I know it's used in a lot of foundational locations. One way we could in theory thread this needle would be to to have a __wasilibc_pipe(..) function which is semantically like pipe but clearly indicates "this is not a pipe because there's no buffer". That would ease the burden of porting code (in theory) while still forcing applications to acknowledge the difference. This isn't a great situation, though, since keeping stuff working as-is is generally better.

Another possible fix would be to literally implement buffering. I think that's all workable in wasi-libc given the nature of these pipes (e.g. no true multithreading, just concurrency). That'd make the implementation more complex, but would enable making these more suitable drop-in replacements for Unix pipes.