Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

RFC: Add descriptive names to doctests #3311

Open
wants to merge 8 commits into
base: master
Choose a base branch
from
Open
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
142 changes: 142 additions & 0 deletions text/0000-doctest-name.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,142 @@
- Feature Name: doctest-name
- Start Date: 2010-10-07
- RFC PR: [rust-lang/rfcs#0000](https://github.com/rust-lang/rfcs/pull/0000)
- Rust Issue: [rust-lang/rust#0000](https://github.com/rust-lang/rust/issues/0000)

# Summary
[summary]: #summary

Allow giving documentation tests names for identification in test runner output
by including `name=IDENT` in doctest code block info strings.

# Motivation
[motivation]: #motivation

When doctests run, the test runner prints the name of the module that contains
them and the line number that they start on. If a module contains many
doctests, it is hard to identify which doctest is which in the test runner
output by line number alone.

By giving users the option to give names to doctests, identifying individual
doctests in test runner output will be easier. Additionally, users could re-run
specific tests using `cargo test --doc -- $TEST_NAME`.

# Guide-level explanation
[guide-level-explanation]: #guide-level-explanation

Doctests are written as fenced markdown code blocks that start and end with
lines containing three backticks.

The first set of backticks may be followed by an info string. For example, to
tell the Rust compiler to ignore the doctest:

````rust

// This doctest won't run.
```ignore
assert_eq!(2 + 2, 4);
```

````

Normally, doctests do not have names, and when run, the test runner will
identify them by their test binary, module name, and line number:

```
Doc-tests foo

running 1 test
test src/lib.rs - bar (line 37) ... ok

test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out
```

If you'd like to have the test runner print a more informative name, perhaps
because there are many doctests in a single file, you can put `name=IDENT`,
where `IDENT` is the name the test should have, which must be an identifier, in
the doctest's info string.


````rust

```name=arithmetic
assert_eq!(2 + 2, 4);
```

````

```
Doc-tests foo

running 1 test
test src/lib.rs - bar::arithmetic (line 37) ... ok

test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out
```

# Reference-level explanation
[reference-level-explanation]: #reference-level-explanation

The set of valid words that may appear in code block info strings is extended
to include patterns of the form `name=IDENT`.

This `IDENT` will be the name of the documentation test generated by the code
block, and must be a valid Rust identifier, which includes raw `r#`-prefixed
identifiers.

When the test runner runs the documentation test, the test will be identified
by this name, in addition to the test binary, module, and line number.

Multiple `name=IDENT` words may not appear in a single code block's info
string.

Non-unique `IDENT`s and `IDENTS` that conflict with other items within the
current namespace will produce a warn-by-default error that may become a hard
error in the future.

`name=IDENT` may be combined with existing annotations like `rust` or
`should_panic` by separating the annotations with commas:

````

```rust,name=foo
assert!(true)
```

````

The ability to include multiple comma-separated annotations is an existing
feature.

# Drawbacks
[drawbacks]: #drawbacks

- Adds complexity to code block parsing.

- Requires parsing multi-word info strings, which are standard markdown, but
which does increase the complexity of the markdown parser.

# Rationale and alternatives
[rationale-and-alternatives]: #rationale-and-alternatives

This design seems like the simplest design possible, although viable
alternatives may have been overlooked.

The impact of adopting this RFC is minimal. Doctests test output will remain a
little hard to make sense of.

# Prior art
[prior-art]: #prior-art

Unit and integration tests have names. This change would bring doctests closer
to parity, by allowing features like test filtering to work with doctests.
casey marked this conversation as resolved.
Show resolved Hide resolved

# Unresolved questions
[unresolved-questions]: #unresolved-questions

None.
casey marked this conversation as resolved.
Show resolved Hide resolved

# Future possibilities
[future-possibilities]: #future-possibilities

None.