Improve mechanism to separate unit and integration tests

[martynp]

Proposal

Problem statement

There is no simple way to run unit tests only, without understanding of the crate content (i.e. cargo test --bins, cargo test --lib, cargo test --bins --lib).

Motivation, use-cases

For complex systems integration tests can be time consuming and do not always add value during all aspects of the development cycle, it is favorable to be able to select between only running integration tests and not running integrations tests as part of the normal test suite.

It is currently possible to select integration tests using:

cargo test --test '*'

And for a bin only crate the following works to select unit tests:

cargo test --bins

And for a lib only crate the following works:

cargo test --lib

If you use cargo test --lib on a bin only crate, you get a warning. But if you use cargo test --bins on a lib crate you get an error.

So there isn't a consistent mechanism to select unit tests only.

Solution sketches

The simplest solution would be to make cargo test --bins emit a warning when there are no bins to test. However...

I would like to add --exclude-integration and --integration to the test lib, this would keep consistency with the ignored and include-ignored commands. i.e.

cargo test -- --exclude-integration
cargo test -- --integration

Links and related work

I have a first pass at the code here:

da004f7

[clubby789]

@rustbot label +A-libtest