add new section on the rustdoc test suite

Author: lolbinarycat

src/doc/rustc-dev-guide/src/rustdoc-internals/htmldocck.md

@@ -0,0 +1,101 @@
+# `rustdoc` tests
+
+This page is specifically about the `rustdoc` test suite, for other test suites used for testing rustdoc, see [Rustdoc § Tests](../rustdoc.md#tests).
+
+`htmldocck.py` is a custom checker script that uses [XPath] to verify the HTML output of rustdoc.
+
+[XPath]: https://en.wikipedia.org/wiki/XPath
+
+## Directives
+Directives to htmldocck are similar to those given to `compiletest` in that they take the form of `//@` comments.
+
+All `PATH`s in directives are relative to the the rustdoc output directory (`build/TARGET/test/rustdoc/TESTNAME`),
+so it is conventional to use a `#![crate_name = "foo"]` attribute to avoid writing paths.
+To avoid repetion, `-` can be used in any `PATH` argument to re-use the previous `PATH` argument.
+
+All arguments take the form of quoted strings,
+with the exception of `COUNT` and the special `-` form of `PATH`.
+
+Directives are assertions that place constraints on the generated HTML.
+
+All directives (except `files`) can be negated by putting a `!` in front of their name.
+
+Similar to shell commands,
+directives can extend across multiple lines if their last char is `\`.
+In this case, the start of the next line should be `//`, with no `@`.
+
+For example, `//@ !has 'foo/struct.Bar.html'` checks that crate `foo` does not have a page for a struct named `Bar` in the crate root.
+
+### `has`
+
+Usage 1: `//@ has PATH`
+Usage 2: `//@ has PATH XPATH PATTERN`
+
+In the first form, `has` checks that a given file exists.
+
+In the second form, `has` is an alias for `matches`,
+except `PATTERN` is a whitespace-normalized[^1] string instead of a regex.
+
+### `matches`
+
+Usage: `//@ matches PATH XPATH PATTERN`
+
+Checks that the text of each element selected by `XPATH` in `PATH` matches the python-flavored regex `PATTERN`.
+
+### `matchesraw`
+
+Usage: `//@ matchesraw PATH PATTERN`
+
+Checks that the contents of the file `PATH` matches the regex `PATTERN`.
+
+### `hasraw`
+
+Usage: `//@ hasraw PATH PATTERN`
+
+Same as `matchesraw`, except `PATTERN` is a whitespace-normalized[^1] string instead of a regex.
+
+### `count`
+
+Usage: `//@ count PATH XPATH COUNT`
+
+Checks that there are exactly `COUNT` matches for `XPATH` within the file `PATH`.
+
+### `snapshot`
+
+Usage: `//@ snapshot NAME PATH XPATH`
+
+Creates a snapshot test named NAME.
+A snapshot test captures a subtree of the DOM, at the location
+determined by the XPath, and compares it to a pre-recorded value
+in a file. The file's name is the test's name with the `.rs` extension
+replaced with `.NAME.html`, where NAME is the snapshot's name.
+
+htmldocck supports the `--bless` option to accept the current subtree
+as expected, saving it to the file determined by the snapshot's name.
+compiletest's `--bless` flag is forwarded to htmldocck.
+
+### `has-dir`
+
+Usage: `//@ has-dir PATH`
+
+Checks for the existance of directory `PATH`.
+
+### `files`
+
+Usage: `//@ files PATH ENTRIES`
+
+Checks that the directory `PATH` contains exactly `ENTRIES`.
+`ENTRIES` is a python list of strings inside a quoted string,
+as if it were to be parsed by `eval`.
+(note that the list is actually parsed by `shlex.split`,
+so it cannot contain arbitrary python expressions).
+
+Example: `//@ files "foo/bar" '["index.html", "sidebar-items.js"]'`
+
+[^1]: Whitespace normalization means that all spans of consecutive whitespace are replaced with a single space.  The files themselves are also whitespace-normalized.
+
+## Limitations
+All `XPATH` arguments must start with `//` due to a flaw in the implemention.
+
+Only well-formed HTML can be parsed (hopefully rustdoc doesn't output mismatched tags).
+

src/doc/rustc-dev-guide/src/rustdoc.md

@@ -77,27 +77,27 @@ does is call the `main()` that's in this crate's `lib.rs`, though.)
   `doctest.rs`.
 * The Markdown renderer is loaded up in `html/markdown.rs`, including functions
   for extracting doctests from a given block of Markdown.
-* The tests on the structure of rustdoc HTML output are located in `tests/rustdoc`, where
-  they're handled by the test runner of bootstrap and the supplementary script
-  `src/etc/htmldocck.py`.
 * Frontend CSS and JavaScript are stored in `html/static/`.
 
 ## Tests
 
-* All paths in this section are relative to `tests` in the rust-lang/rust repository.
-* Tests on search engine and index are located in `rustdoc-js` and `rustdoc-js-std`.
+* Tests on search engine and index are located in `tests/rustdoc-js` and `tests/rustdoc-js-std`.
   The format is specified
   [in the search guide](rustdoc-internals/search.md#testing-the-search-engine).
 * Tests on the "UI" of rustdoc (the terminal output it produces when run) are in
-  `rustdoc-ui`
+  `tests/rustdoc-ui`
 * Tests on the "GUI" of rustdoc (the HTML, JS, and CSS as rendered in a browser)
-  are in `rustdoc-gui`. These use a [NodeJS tool called
+  are in `tests/rustdoc-gui`. These use a [NodeJS tool called
   browser-UI-test](https://github.com/GuillaumeGomez/browser-UI-test/) that uses
   puppeteer to run tests in a headless browser and check rendering and
   interactivity.
 * Additionally, JavaScript type annotations are written using [TypeScript-flavored JSDoc]
   comments and an external d.ts file. The code itself is plain, valid JavaScript; we only
   use tsc as a linter.
+* The tests on the structure of rustdoc HTML output are located in `tests/rustdoc`,
+  where they're handled by the test runner of bootstrap and
+  the supplementary script `src/etc/htmldocck.py`.
+  [These tests have several extra directives available to them](./rustdoc-internals/htmldocck.md).
 
 [TypeScript-flavored JSDoc]: https://www.typescriptlang.org/docs/handbook/jsdoc-supported-types.html
 

src/doc/rustc-dev-guide/src/tests/compiletest.md

@@ -78,7 +78,7 @@ The following test suites are available, with links for more information:
 
 ### Rustdoc test suites
 
-See [Rustdoc tests](../rustdoc.md#tests) for more details.
+See [Rustdoc § Tests](../rustdoc.md#tests) for more details.
 
 | Test suite       | Purpose                                                                  |
 |------------------|--------------------------------------------------------------------------|