Skip to content

Commit

Permalink
Auto merge of rust-lang#8884 - jakoschiko:accept-test-execution-time,…
Browse files Browse the repository at this point in the history
… r=alexcrichton

Relaxes expectation of `cargo test` tests to accept test execution time

rust-lang/rust#75752 changes the output of libtest.

For example, output before:

```
test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out
```

Output after:

```
test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.01s
```

This breaks some cargo tests: https://github.com/rust-lang-ci/rust/runs/1439245145

As a preparation for the merge, this PR relaxes the test expectations of `cargo test` tests to accept both outputs. I'm using the existing pattern feature of `Execs::with_stdout`:

```
test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out[..]
```

I made this change for the following tests:

- `test::can_not_mix_doc_tests_and_regular_tests`
- `test::cargo_test_quiet_with_harness`
- `test::test_filtered_excludes_compiling_examples`
- `test::cargo_test_doctest_xcompile_ignores` (didn't fail in rust-lang/rust#79317, but failed locally)
- `test::cargo_test_doctest_xcompile` (doesn't run locally, I changed it just to be safe)
- `test::cargo_test_doctest_xcompile_runner` (doesn't run locally, I changed it just to be safe)
- `test::cargo_test_doctest_xcompile_no_runner` (doesn't run locally, I changed it just to be safe)

If requested, I will open another PR later to change the expectation to:

```
test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in [..]s
```

However, I don't know how to handle WASM targets because WASM doesn't support time measuring, therefore the libtest output didn't change for WASM. Is WASM even relevant here?
  • Loading branch information
bors authored and ehuss committed Nov 30, 2020
1 parent e69b590 commit 85a746c
Showing 1 changed file with 13 additions and 13 deletions.
26 changes: 13 additions & 13 deletions tests/testsuite/test.rs
Original file line number Diff line number Diff line change
Expand Up @@ -175,7 +175,7 @@ fn cargo_test_quiet_with_harness() {
"
running 1 test
.
test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out
test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out[..]
",
)
Expand Down Expand Up @@ -1552,7 +1552,7 @@ fn test_filtered_excludes_compiling_examples() {
running 1 test
test tests::foo ... ok
test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out
test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out[..]
",
)
Expand Down Expand Up @@ -3724,13 +3724,13 @@ pub fn foo() -> u8 { 1 }
running 1 test
test tests::it_works ... ok
test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out
test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out[..]
running 1 test
test src/lib.rs - foo (line 1) ... ok
test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out
test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out[..]
\n",
)
.run();
Expand All @@ -3746,7 +3746,7 @@ test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out
running 1 test
test tests::it_works ... ok
test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out
test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out[..]
\n",
)
.run();
Expand All @@ -3763,7 +3763,7 @@ test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out
running 1 test
test src/lib.rs - foo (line 1) ... ok
test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out
test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out[..]
",
)
Expand Down Expand Up @@ -3877,30 +3877,30 @@ fn cargo_test_doctest_xcompile_ignores() {
#[cfg(not(target_arch = "x86_64"))]
p.cargo("test")
.with_stdout_contains(
"test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out",
"test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out[..]",
)
.run();
#[cfg(target_arch = "x86_64")]
p.cargo("test")
.with_status(101)
.with_stdout_contains(
"test result: FAILED. 0 passed; 1 failed; 0 ignored; 0 measured; 0 filtered out",
"test result: FAILED. 0 passed; 1 failed; 0 ignored; 0 measured; 0 filtered out[..]",
)
.run();

#[cfg(not(target_arch = "x86_64"))]
p.cargo("test -Zdoctest-xcompile")
.masquerade_as_nightly_cargo()
.with_stdout_contains(
"test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out",
"test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out[..]",
)
.run();

#[cfg(target_arch = "x86_64")]
p.cargo("test -Zdoctest-xcompile")
.masquerade_as_nightly_cargo()
.with_stdout_contains(
"test result: ok. 0 passed; 0 failed; 1 ignored; 0 measured; 0 filtered out",
"test result: ok. 0 passed; 0 failed; 1 ignored; 0 measured; 0 filtered out[..]",
)
.run();
}
Expand Down Expand Up @@ -3940,7 +3940,7 @@ fn cargo_test_doctest_xcompile() {
))
.masquerade_as_nightly_cargo()
.with_stdout_contains(
"test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out",
"test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out[..]",
)
.run();
}
Expand Down Expand Up @@ -4020,7 +4020,7 @@ fn cargo_test_doctest_xcompile_runner() {
))
.masquerade_as_nightly_cargo()
.with_stdout_contains(
"test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out",
"test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out[..]",
)
.with_stderr_contains("this is a runner")
.run();
Expand Down Expand Up @@ -4064,7 +4064,7 @@ fn cargo_test_doctest_xcompile_no_runner() {
))
.masquerade_as_nightly_cargo()
.with_stdout_contains(
"test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out",
"test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out[..]",
)
.run();
}
Expand Down

0 comments on commit 85a746c

Please sign in to comment.