Merge pull request #5662 from NomicFoundation/galargh/test-reporter-r…

…eadme

docs: enhance the test reporter's README

v-next/hardhat-node-test-reporter/README.md

@@ -2,4 +2,96 @@
 
 This package includes Hardhat's `node:test` reporter.
 
-This project is heavily inspired by https://github.com/voxpelli/node-test-pretty-reporter
+This project is heavily inspired by https://github.com/voxpelli/node-test-pretty-reporter.
+
+It tries to mimic [the `Mocha`'s default `Spec` reporter](https://mochajs.org/#spec), as close as possible.
+
+It is designed to output information about the test runs as soon as possible and in test **defintion** order.
+
+Once the test run ends, it will output global information about it, based on the diagnostics emitted by `node:test`, and any custom or unrecognized diagnostics message.
+
+Finally, it will output the failure reasons for all the failed tests.
+
+It introduces a number of custom features to make it more suitable for use with Hardhat.
+
+![Demo](./demo.gif)
+
+## Installation
+
+`hardhat-node-test-reporter` comes built-in with Hardhat's `node:test` plugin, which itself comes bundled in by default with `hardhat-toolbox`. You don't need to install it separately.
+
+If you want to use the reporter in your own project, you can install it with npm (optionally, with a `--save-dev` flag):
+
+```bash
+npm install hardhat-node-test-reporter
+```
+
+## Usage
+
+If you're using `hardhat`'s `node:test` plugin, the reporter will be used by default whenever you run the `hardhat test` task.
+
+If you want to use the reporter directly with `node`, you can do so by passing the `--test-reporter` flag:
+
+```bash
+node --test --test-reporter=@ignored/hardhat-vnext-node-test-reporter
+```
+
+## Custom features
+
+### Slow Tests
+
+Slow threshold is configured to 75ms. If a test case takes longer than that, it will be highlighted in red.
+
+### Test Coverage
+
+Test coverage is currently not supported by this reporter.
+
+### GitHub Actions
+
+This reporter is designed to work well with GitHub Actions. By default, it will create error annotations for failed tests. You can disable this feature by setting the `NO_GITHUB_ACTIONS_ANNOTATIONS` environment variable to `true`.
+
+### Colour Output
+
+This reporter will colour the output by default in terminals that support it. You can forcefully disable this feature by setting the `FORCE_COLOR` environment variable to `0` (or passing a `--no-color` flag). Similarly, you can forcefully enable this feature by setting the `FORCE_COLOR` environment variable to `1` (or passing a `--color` flag).
+
+The behaviour is inherited from the [`chalk` package](https://github.com/chalk/chalk?tab=readme-ov-file#supportscolor).
+
+#### Colour Legend
+
+| Output Type | Colour       |
+| ----------- | ------------ |
+| Cancelled   | Gray         |
+| Error       | Red          |
+| Failure     | Red          |
+| Skipped     | Cyan         |
+| Success     | Green (tick) |
+| TODO        | Blue         |
+
+### Nesting
+
+This reporter will indicate nesting of test suites by indenting the output at 2-space wide intervals.
+
+### Error Formatting
+
+This reporter will format errors in a human readable way.
+
+It will try to use the `inspect` method from the `node:util` module to print the error object together with its stack trace.
+
+It will replace file URLs with relative paths. This currently does not work well on Windows.
+
+Finally, it will attempt to show a diff of the expected and actual values of the error object, if they are available.
+
+### Diagnostics
+
+This reporter will aggregate and output diagnostics emitted by `node:test` at the end of the test run. If it doesn't recognize a diagnostic message, it will output them as-is after the well-known diagnostics.
+
+Well-known diagnostics are:
+
+- `tests`
+- `suites`
+- `pass`
+- `fail`
+- `cancelled`
+- `skipped`
+- `todo`
+- `duration_ms`

v-next/hardhat-node-test-reporter/src/error-formatting.ts

@@ -5,6 +5,7 @@ import { inspect } from "node:util";
 import chalk from "chalk";
 import { diff } from "jest-diff";
 
+import { indent } from "./formatting.js";
 import {
   cleanupTestFailError,
   isCancelledByParentError,
@@ -18,7 +19,10 @@ export function formatError(error: Error): string {
       chalk.red("Test cancelled by parent error") +
       "\n" +
       chalk.gray(
-        "    This test was cancelled due to an error in its parent suite/it or test/it, or in one of its before/beforeEach",
+        indent(
+          "This test was cancelled due to an error in its parent suite/it or test/it, or in one of its before/beforeEach",
+          4,
+        ),
       )
     );
   }
@@ -27,7 +31,7 @@ export function formatError(error: Error): string {
     return (
       chalk.red(`Test file execution failed (exit code ${error.exitCode}).`) +
       "\n" +
-      chalk.gray("    Did you forget to await a promise?")
+      chalk.gray(indent("Did you forget to await a promise?", 4))
     );
   }
 

v-next/hardhat-node-test-reporter/src/formatting.ts

@@ -124,7 +124,7 @@ function nestingToIndentationLength(nesting: number): number {
   return (nesting + 1) * 2;
 }
 
-function indent(str: string, spaces: number): string {
+export function indent(str: string, spaces: number): string {
   const padding = " ".repeat(spaces);
   return str.replace(/^/gm, padding);
 }

v-next/hardhat-node-test-reporter/src/reporter.ts

@@ -53,14 +53,14 @@ export default async function* customReporter(
    * repeate it, we keep track of the last printed context element. We do this
    * by keeping track of its index in the stack.
    *
-   * We also keep track of any diagnostic message that its reported by node:test
+   * We also keep track of any diagnostic message that is reported by node:test
    * and at the end we try to parse the global diagnostics to gather information
    * about the test run. If during this parsing we don't recognize or can't
    * properly parse one of this diagnostics, we will print it at the end.
    *
-   * Whenever a test fails, we pre-format its failure reason, so that don't need
-   * to keep the failure event in memory, and we can still print the failure
-   * reason at the end.
+   * Whenever a test fails, we pre-format its failure reason, so that we don't
+   * need to keep the failure event in memory, and we can still print the
+   * failure reason at the end.
    *
    * This code is structed in the following way:
    *  - We use an async generator to process the events as they come, printing
@@ -74,7 +74,7 @@ export default async function* customReporter(
    *  - The generaor drives the high-level format of the output, and only uses
    *  the formatting functions to generate repetitive parts of it.
    *
-   * [1] As reporter by node:test, in defintion order, which may differ from
+   * [1] As reported by node:test, in defintion order, which may differ from
    * actual execution order.
    */