`cargo t --doc` does not respect `--color` when a compilation error occurs

[zacknewman]

Problem

When using cargo to run doctests, --color is not respected during a compilation failure; however cargo t --tests does respect it and rustdoc --test almost respects it. Similarly cargo t --doc does respect --color when a compilation failure does not occur.

Steps

/// ```
/// bar
/// ```
pub fn foo() {}
#[cfg(test)]
mod tests {
    #[test]
    fn foo() {
        bar
    }
}

[zack@laptop src]$ cargo t --color never --doc -- --color never
    Finished `test` profile [unoptimized + debuginfo] target(s) in 0.00s
   Doc-tests bar

running 1 test
test src/lib.rs - foo (line 1) ... FAILED

failures:

---- src/lib.rs - foo (line 1) stdout ----
error[E0423]: expected value, found crate `bar`
 --> src/lib.rs:3:1
  |
5 | bar
  | ^^^ not a value

error: aborting due to 1 previous error

For more information about this error, try `rustc --explain E0423`.
Couldn't compile the test.

failures:
    src/lib.rs - foo (line 1)

test result: FAILED. 0 passed; 1 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.02s

error: doctest failed, to rerun pass `--doc`

This portion above is always color and bold formatted:

error[E0423]: expected value, found crate `bar`
 --> src/lib.rs:3:1
  |
5 | bar
  | ^^^ not a value

error: aborting due to 1 previous error

For more information about this error, try `rustc --explain E0423`.

In contrast,

[zack@laptop src]$ cargo t --color never --tests -- --color never
   Compiling bar v0.1.0 (/home/zack/projects/bar)
error[E0425]: cannot find value `bar` in this scopear(test)
 --> src/lib.rs:9:9
  |
9 |         bar
  |         ^^^ not found in this scope

For more information about this error, try `rustc --explain E0425`.
error: could not compile `bar` (lib test) due to 1 previous error

has no color or formatting. Also notice the different error code, E0425, compared to E0423.

With the exception of the two instance of FAILED as red, the following has no color or formatting:

[zack@laptop src]$ rustdoc --test --color never lib.rs

running 1 test
test lib.rs - foo (line 1) ... FAILED

failures:

---- lib.rs - foo (line 1) stdout ----
error[E0425]: cannot find value `bar` in this scope
 --> lib.rs:2:1
  |
3 | bar
  | ^^^ not found in this scope

error: aborting due to 1 previous error

For more information about this error, try `rustc --explain E0425`.
Couldn't compile the test.

failures:
    lib.rs - foo (line 1)

test result: FAILED. 0 passed; 1 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.02s

Also notice that it has the same error code, E0425, as cargo t --tests even though I would have guessed for it to be the same as cargo t --doc.

Unsurprisingly when trying to retain color via --color always when piping stdout into a different process, the color is not retained presumably for the same reason that --color never is not being respected when writing to a TTY.

I think this might be related to the fact that rustdoc writes the compilation error to stdout; thus cargo t --doc also writes the formatted output to stdout; however cargo t --tests writes the compilation error to stderr. Normally both stderr and stdout are not formatted though, so not sure why that would matter.

Possible Solution(s)

No response

Notes

No response

Version

cargo 1.80.1 (376290515 2024-07-16)
release: 1.80.1
commit-hash: 37629051518c3df9ac2c1744589362a02ecafa99
commit-date: 2024-07-16
host: x86_64-unknown-linux-gnu
libgit2: 1.7.2 (sys:0.18.3 vendored)
libcurl: 8.6.0-DEV (sys:0.4.72+curl-8.6.0 vendored ssl:OpenSSL/1.1.1w)
ssl: OpenSSL 1.1.1w  11 Sep 2023
os: Arch Linux [64-bit]

[weihanglo]

Note that cargo t --color never --tests without -- --color never doesn't respect Cargo's color option either. We have an open issue for that #1983.

For rustdoc, unlike libtest, rustdoc seems to be the only tool for Cargo to generate docs. The question would be — should Cargo pass --color to rustdoc invocations?

cargo/src/cargo/ops/cargo_test.rs

Lines 170 to 175 in cc74523

	fn run_doc_tests(
	ws: &Workspace<'_>,
	options: &TestOptions,
	test_args: &[&str],
	compilation: &Compilation<'_>,
	) -> Result<Vec<UnitTestError>, CliError> {

[ehuss]

Yea, this is likely just an oversight, it should be passing --color to rustdoc.

[ShashiSugrim]

Hello, I am currently working on this issue.