Make it possible to run binaries produced by cargo directly.

[matklad]

EDIT: 90% of this is solved by --message-format=json flag.

Hi! This is a followup of #1924 and #1726.

Problem

Sometimes one needs to run an executable produced by Cargo (be it example, binary or doc/unit/integration test) directly.

The most common use case is debugger integration in the IDE, but other examples include strace, perfrecord and similar tools.

Peculiarities

cargo test can produce more than one binary. We need to handle this somehow.

Both cargo run and cargo test have different flavors for specifying profiles, features, targets and arguments for the binary. Ideally, it should be easy to learn the name of output file by the cargo command.

Cargo can setup LD_LIBRARY_PATH when running a binary, it would be good to be able replicate this as well.

Solutions

--with wrapper

Add a --with option to the run family of commands to allow to specify a custom wrapper. There's a stale pull request implementation in #1763.

This would probably be the most useful option on the command line. However, it is not flexible enough: there's still an intermediate Cargo process around, and this won't be enough for IntellJ Rust: we launch debugger process first and then send it a command to launch the debugee.

Also, this can be implemented as an external subcommand on top of other options, so that you can run
cargo with-wrapper strace run --release --bin foo.

--output-file option

We can implement #1706, than clients could specify the name they want. This is a nice option, but it's not perfect: the binary may assume certain location and use something like ../../foo/bar to get resources at run time or something like that. While arguably you should not do this, ideally we should be able able to run binaries exactly as Cargo does. Also with explicit output file the client has to manage temporary directories, which is some extra work.

Stable file names

We can make the names of binaries predictable. This is perhaps the simplest option, but I don't like it for various reasons:

• We already have debug and release profiles, which affect the output path. This means that clients should implement some logic to get the current profile, which may be brittle.

• This requires stabilizing target directory layout and may make future features harder. What if one day we would like to add custom profiles and an ability to configure the custom profile via environment variables/config files?

• This does not allow to easily derive the binary name given the cargo command, which again means some logic on the client's side.

Add file names to cargo metadata

This is very similar to the previous option, and does not solve the profiles problem at all.

Print the resulting file name with --no-run

This is my favorite option :)

• It is naturally forward compatible.

• It's naturally extendable to print more information about the environment which is used to run the process (LD_LIBRARY_PATH, command line flags).

• We almost do this already: with --message-format=json we print the path to binary if it is not fresh. I suggest that we always output this info, even we don't actually rebuild the binary. Perhaps we should use a new, separate message, to make it easier to extend it to support additional and to make it easier for the clients to separate dependencies from the actual end binaries. (Emit more info on --message-format=json #3319, Always produce artifact messages #3752)

• It would be trivial to match cargo commands with binary names: just add --no-run option (I'd like to add this flag to cargo run as well).

• I like that on the client side, you have to actually build the binary before trying to use it. This solves debugging stale binaries problem by construction.

I'll be glad to implement any of these options :)

cc @bruno-medeiros @vojtechkral.

[alexcrichton]

Thanks for summarizing the lay of the land here @matklad! I agree with your analysis and I'd be totally down with extending json messages and such. To take that route it sounds like:

• We should emit messages about paths to binaries even if the target is fresh
• We may be able to avoid cargo run --no-run as that's just the equivalent of cargo build, right?
• Perhaps add messages about LD_LIBRARY_PATH with new entries that need to be added?

[vojtechkral]

Thanks for the summary!

Your proposal sounds good to me... In my usecase (ie. gdb, strace et al.) I could do something like gdb $(cargo build --no-run), which sounds allright. It wouldn't be hard to write a short script or Makefile rule for anything more involved...

[koute]

Yes please!

I recently wrote a Cargo subcommand that needs to know the name of the binaries which are produced after a build, and this was absolutely the biggest pain point I had. Right now I have this really ugly hack where I basically just list all of the files in the target directory that could match whatever the kind of a build I want to run could generate, then I try to build it, and then I see what changed.

I would love to have something like cargo rustc [other args] --print-output-path which would just tell me what the previous invocation with the exactly the same arguments has generated.

with --message-format=json we print the path to binary if it is not fresh. I suggest that we always output this info, even we don't actually rebuild the binary.

In my use case I really don't want to use --message-format=json, simply because I don't want to do the formatting of the messages myself. What I have is a (not so simple) wrapper around cargo rustc, and I just want to pipe anything that cargo rustc produces (error messages, etc.) directly into the user's terminal, but still be able to know what output the invocation has generated.

[malbarbo]

How the proposed approaches works for doc tests (they are ephemeral)? The --with approach would work.

[matklad]

How the proposed approaches works for doc tests (they are ephemeral)? The --with approach would work.

I think neither approach would work out of the box, because it's rustdoc who manages the execution of doctests. That is, any solution we implement in cargo, we need to also implement in rustdoc and then make Cargo to forward command line arguments to rustdoc.

But I think that executing doctests directly is an extremely niche use case, and perhaps we can just not do this?

[matklad]

We may be able to avoid cargo run --no-run as that's just the equivalent of cargo build, right?

I think I want run --no-run for IntelliJ Rust, but this is certainly something we can add or not add later. Let me describe my use case, perhaps it is useful elsewhere.

In IDEA UI the rule in general is "everything you can run, you can debug". So the user may launch command line like cargo run --release --bin foo -- my_program_arg, and then use Shift + F10 to run, and Shift + F9 to debug the program, without configuring debugging separately.

So I want to be able to get the command line for debugging from the command for running.

One way to do this is to add --no-run everywhere, so that Cargo would tell me: "Hey, I've compiled everything, and I would launch this binary with these arguments and in this environment, but I'll let the IDE take over from this point".

Another way would be to parse the command line, find out the equivalent build command and figure the necessary args for the final binary. This parsing of Cargo's command line I would love to avoid, but this is not a major problem.

[alexcrichton]

Oh so the user is typing in cargo run and IntelliJ wants to hijack that? I think it's definitely a good idea to avoid parsing arguments and emulating what Cargo is doing, so for that --no-run seems fine to me.

[vojtechkral]

Side note: Just seeing the literal cargo run --no-run command in the discussion, it really stands out as an obvious oxymoron, something really counter-intuitive and possibly confusing. Perhaps we could move it out of the run command entirely and introduce something like cargo path, cargo runcmd, cargo runenv or somesuch?

[matklad]

Side note: Just seeing the literal cargo run --no-run command in the discussion, it really stands out as an obvious oxymoron, something really counter-intuitive and possibly confusing. Perhaps we could move it out of the run command entirely and introduce something like cargo path or somesuch?

A similar issue is that we have cargo test --no-run in the first place, which does something similar to cargo build actually.

[alexcrichton]

Yeah I wouldn't expect a user to type cargo run --no-run, but having tools be able to insert such a command everywhere seems useful!

[vojtechkral]

Ok, let me try to summarize:

• cargo run --no-run seems to be equivalent to cargo build
• the cargo build --no-run is confusing, build doesn't run stuff anyway. If the options is meant for printing information out, why not call it cargo build --print? It could also optionally take an argument specifying what things you'd like printed, something like ps -o.
• cargo test --no-run is, if I'm getting it right, meant to build target in test configuration but not actually run tests. I'm not sure whether this option belongs in cargo test or if it rather should be something like cargo build --test, but in any case, it seems to me to be orthogonal to this issue.

Correct me if I'm wrong in any of those points...

[malbarbo]

@matklad

But I think that executing doctests directly is an extremely niche use case, and perhaps we can just not do this?

My use case for executing doctests directly is to collect code coverage. Although some says that doctest should not be used in code coverage, some functions have doctests that are good enough and not running them would decrease the coverage.

[alexcrichton]

@vojtechkral did you see @matklad's description above? It sounds like users are typing cargo run and then IntelliJ would have to intepret the command line and translate that to cargo build, whereas slapping on a --no-run would be much eaiser.

And currently cargo build --test isn't equivalent to cargo test --no-run because the former builds one test while the latter builds all tests.

[vojtechkral]

@alexcrichton I must have missed that post or misunderstood it earlier. Sorry.

I'm not convinced that's a good reasoning, though. I don't think it's a good idea to make cargo's interface tailored to IntelliJ. Picture a few months/years down the road a newbie comes along:

• Q: How do I get a Rust binary built and a path to it printed? I need that for XYZ.
• A: Just do cargo run --no-run or cargo build --no-run
• Q: Allright, but wtf?
• A: Yeah, I know, it was designed that way a while ago because it's better for debugging in IntelliJ or somesuch...

Could we instead perhaps separate "no running" and "printing"? Or maybe even better make "printing" imply "not running", where applicable? Ie. --print would make sense with cargo build and it would suppress running with cargo run and cargo test, which would make more sense to me that the other way around...

And currently cargo build --test isn't equivalent to cargo test --no-run because the former builds one test while the latter builds all tests.

cargo build --tests perhaps?

I guess my problem with the --no-run flag is that it basically promotes an interface along the lines of cargo dosomething --wait-dont-actually-do-that-do-something-else.

I hope I'm making sense. Anyway, obvisouly, that's just my $.02, if you guys come to the conclusion --no-run is ok enough, that's fine by me, it's IMO still an improvement over --with.