Miri stdout is buffered and stdin in always empty

[RalfJung]

Miri can now read from stdin (thanks to @samrat), but unfortunately stdin is always empty when doing cargo miri run. I think this is because of the way that we are contorting cargo into running Miri -- Miri needs all the compiler flags, so we run cargo check and set RUSTC_WRAPPER to ourselves to dispatch "compiling" the final binary to Miri. cargo thinks it is running a compiler, and does not forward any stdin to it. (Here is a test that should work.)

Similarly, stdout is always buffered, so even if the program flushes stdout, cargo miri run will only print it after the next newline (this is rust-lang/cargo#6641).

@ehuss is there any chance that we could get cargo to invoke Miri in a way that's more like cargo run and less like running a compiler?

[oli-obk]

If we want to do a hacky solution we can use the mitosis crate and have the cargo-miri wrapper forward stdin to the miri driver.

[RalfJung]

mitosis says it is "WIP", so that seems a bit too fragile. Also I don't see how that helps because we have cargo sitting between cargo-miri and miri -- without cargo support, I don't see how we could forward the true stdout/stdin to miri.

[oli-obk]

well, we don't need mitosis actually, we just need the ipc-channel crate to open a single IPC channel and spin up a thread in cargo-miri that forwards stdin to the ipc channel. We create the ipc channel before invoking cargo from cargo-miri and forward the ipc channel's id to miri via command line flags.

[RalfJung]

So IPC channels have a file name or so that can be passed around? Also, does this work on Windows? The ipc_channel crate only mentions macOS and Linux.

[bjorn3]

servo/ipc-channel#166
servo/ipc-channel#233

There have been several attempts to add Windows support. Unfortunately they seem to have stalled.

[RalfJung]

Also it doesn't look like servo's ipc-channels have filenames, so I am not sure how we could pass them down to the child process. A unix domain socket in a temporary directory would do it, but the crate only seems to support anonymous channels.

Oh and of course this seems like a horrible hack.^^ A better solution might be being able to instruct cargo to just forward stdin and stdout unchanged. That would mean it also gets forwarded to the invoked rustc processes, but that still seems better than the current situation.

[ehuss]

I don't think there is a way to do that in cargo directly. I would probably do things in two phases. Run cargo with RUSTC_WRAPPER, and capture whatever arguments you need (print them in JSON or whatever), and then run miri directly with stdin/stdout inherited. That's kinda hacky, but should work.

[bjorn3]

Also it doesn't look like servo's ipc-channels have filenames, so I am not sure how we could pass them down to the child process.

The channels implement Serialize and Deserialize.

[RalfJung]

@ehuss hm, that is also an interesting idea. I kind of like it.

@oli-obk what do you think?

[oli-obk]

Oh, that's pretty cool actually, as we don't need the separate binary anymore (except for the new RUSTC_WRAPPER, but that's just a "serialize all arguments" binary with no relation to miri) as cargo-miri will then be able to directly run the code.

Another alternative is to have cargo-miri capture all stdin, write it to a file and pass the path to the file by inserting a new argument into the argument list

[RalfJung]

I think I'd pefer to keep the separate binary. Currently cargo-miri does not link against the private rustc libs at all.

Another alternative is to have cargo-miri capture all stdin, write it to a file and pass the path to the file by inserting a new argument into the argument list

That doesn't work when stdin gets more data during execution (think: the program prints a question on stdout and expects a user answer on stdin).

[RalfJung]

One concern I have with this data-gathering scheme is concurrency -- there can be multiple binaries to run and we need to capture all their command-line arguments. Is adding a new line to a file atomic? Or should we instead use a folder and create one file per run in that folder?

Also, we might have to capture more than just the CLI arguments -- e.g., env vars or the working directory.