Skip to content

Commit e3d4330

Browse files
committed
doc,lib,src,test: add --test-skip-pattern cli option
1 parent f8e325e commit e3d4330

File tree

8 files changed

+114
-40
lines changed

8 files changed

+114
-40
lines changed

doc/api/cli.md

Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1969,6 +1969,9 @@ A regular expression that configures the test runner to only execute tests
19691969
whose name matches the provided pattern. See the documentation on
19701970
[filtering tests by name][] for more details.
19711971

1972+
If both `--test-name-pattern` and `--test-skip-pattern` are supplied,
1973+
`--test-name-pattern` will take precedence.
1974+
19721975
### `--test-only`
19731976

19741977
<!-- YAML
@@ -2037,6 +2040,20 @@ node --test --test-shard=2/3
20372040
node --test --test-shard=3/3
20382041
```
20392042

2043+
### `--test-skip-pattern`
2044+
2045+
<!-- YAML
2046+
added:
2047+
- REPLACEME
2048+
-->
2049+
2050+
A regular expression that configures the test runner to only execute tests
2051+
whose name don't match the provided pattern. See the documentation on
2052+
[filtering tests by name][] for more details.
2053+
2054+
If both `--test-name-pattern` and `--test-skip-pattern` are supplied,
2055+
`--test-name-pattern` will take precedence.
2056+
20402057
### `--test-timeout`
20412058

20422059
<!-- YAML

doc/api/test.md

Lines changed: 40 additions & 24 deletions
Original file line numberDiff line numberDiff line change
@@ -298,20 +298,20 @@ describe.only('a suite', () => {
298298

299299
## Filtering tests by name
300300

301-
The [`--test-name-pattern`][] command-line option can be used to only run tests
302-
whose name matches the provided pattern. Test name patterns are interpreted as
303-
JavaScript regular expressions. The `--test-name-pattern` option can be
304-
specified multiple times in order to run nested tests. For each test that is
305-
executed, any corresponding test hooks, such as `beforeEach()`, are also
306-
run. Tests that are not executed are omitted from the test runner output.
307-
308-
Given the following test file, starting Node.js with the
309-
`--test-name-pattern="test [1-3]"` option would cause the test runner to execute
310-
`test 1`, `test 2`, and `test 3`. If `test 1` did not match the test name
311-
pattern, then its subtests would not execute, despite matching the pattern. The
312-
same set of tests could also be executed by passing `--test-name-pattern`
313-
multiple times (e.g. `--test-name-pattern="test 1"`,
314-
`--test-name-pattern="test 2"`, etc.).
301+
The [`--test-name-pattern`][] and [`--test-skip-pattern`][] command-line
302+
options provide flexibility in selecting which tests to run and which to
303+
skip based on their names.
304+
305+
### Using `--test-name-pattern`
306+
307+
The `--test-name-pattern` option filters tests based on their names by
308+
matching them against the provided pattern, which is interpreted as a
309+
JavaScript regular expression. This option can be specified multiple times
310+
to include nested tests. When a test matches the pattern and is executed,
311+
any corresponding test hooks, such as `beforeEach()`, are also run. Tests
312+
that do not match the pattern are omitted from the test runner output.
313+
314+
Consider the following test file:
315315

316316
```js
317317
test('test 1', async (t) => {
@@ -325,14 +325,19 @@ test('Test 4', async (t) => {
325325
});
326326
```
327327

328-
Test name patterns can also be specified using regular expression literals. This
329-
allows regular expression flags to be used. In the previous example, starting
330-
Node.js with `--test-name-pattern="/test [4-5]/i"` would match `Test 4` and
331-
`Test 5` because the pattern is case-insensitive.
328+
Using Node.js with the `--test-name-pattern="test [1-3]"` option would
329+
execute `test 1`, `test 2`, and `test 3`. If `test 1` did not match the
330+
pattern, its subtests would not execute, even if they match the pattern.
331+
Alternatively, specifying the pattern multiple times (e.g.,
332+
`--test-name-pattern="test 1"`, `--test-name-pattern="test 2"`, etc.)
333+
achieves the same result.
334+
335+
Regular expression literals can also be used, allowing regular expression flags.
336+
For instance, using `--test-name-pattern="/test [4-5]/i"` would match `Test 4`
337+
and `Test 5` due to the case-insensitive flag.
332338

333-
To match a single test with a pattern, you can prefix it with all its ancestor
334-
test names separated by space, to ensure it is unique.
335-
For example, given the following test file:
339+
To match a single test uniquely, prefix its name with all its ancestor test
340+
names separated by spaces. For example:
336341

337342
```js
338343
describe('test 1', (t) => {
@@ -344,10 +349,20 @@ describe('test 2', (t) => {
344349
});
345350
```
346351

347-
Starting Node.js with `--test-name-pattern="test 1 some test"` would match
348-
only `some test` in `test 1`.
352+
Using `--test-name-pattern="test 1 some test"` would match only `some test` in `test 1`.
353+
354+
### Using `--test-skip-pattern`
355+
356+
Similarly, the `--test-skip-pattern` option allows skipping tests based on their
357+
names by matching them against the provided pattern, interpreted as JavaScript
358+
regular expressions. If a test's name matches the skip pattern, it will be
359+
excluded from execution.
360+
361+
If both `--test-name-pattern` and `--test-skip-pattern` are
362+
supplied, `--test-name-pattern` will take precedence.
349363

350-
Test name patterns do not change the set of files that the test runner executes.
364+
Test name patterns and skip patterns do not alter the set of files executed by
365+
the test runner.
351366

352367
## Extraneous asynchronous activity
353368

@@ -3157,6 +3172,7 @@ Can be used to abort test subtasks when the test has been aborted.
31573172
[`--test-only`]: cli.md#--test-only
31583173
[`--test-reporter-destination`]: cli.md#--test-reporter-destination
31593174
[`--test-reporter`]: cli.md#--test-reporter
3175+
[`--test-skip-pattern`]: cli.md#--test-name-pattern
31603176
[`--test`]: cli.md#--test
31613177
[`MockFunctionContext`]: #class-mockfunctioncontext
31623178
[`MockTimers`]: #class-mocktimers

lib/internal/test_runner/runner.js

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -113,7 +113,7 @@ function filterExecArgv(arg, i, arr) {
113113
!ArrayPrototypeSome(kFilterArgValues, (p) => arg === p || (i > 0 && arr[i - 1] === p) || StringPrototypeStartsWith(arg, `${p}=`));
114114
}
115115

116-
function getRunArgs(path, { forceExit, inspectPort, testNamePatterns, only }) {
116+
function getRunArgs(path, { forceExit, inspectPort, testNamePatterns, testSkipPatterns, only }) {
117117
const argv = ArrayPrototypeFilter(process.execArgv, filterExecArgv);
118118
if (forceExit === true) {
119119
ArrayPrototypePush(argv, '--test-force-exit');
@@ -124,6 +124,9 @@ function getRunArgs(path, { forceExit, inspectPort, testNamePatterns, only }) {
124124
if (testNamePatterns != null) {
125125
ArrayPrototypeForEach(testNamePatterns, (pattern) => ArrayPrototypePush(argv, `--test-name-pattern=${pattern}`));
126126
}
127+
if (testSkipPatterns != null) {
128+
ArrayPrototypeForEach(testSkipPatterns, (pattern) => ArrayPrototypePush(argv, `--test-skip-pattern=${pattern}`));
129+
}
127130
if (only === true) {
128131
ArrayPrototypePush(argv, '--test-only');
129132
}

lib/internal/test_runner/test.js

Lines changed: 24 additions & 15 deletions
Original file line numberDiff line numberDiff line change
@@ -84,6 +84,7 @@ const {
8484
forceExit,
8585
sourceMaps,
8686
testNamePatterns,
87+
testSkipPatterns,
8788
testOnlyFlag,
8889
} = parseCommandLine();
8990
let kResistStopPropagation;
@@ -300,8 +301,7 @@ class Test extends AsyncResource {
300301
ownAfterEachCount: 0,
301302
};
302303

303-
if ((testNamePatterns !== null && !this.matchesTestNamePatterns()) ||
304-
(testOnlyFlag && !this.only)) {
304+
if (!this.testMatchesCriteria() || (testOnlyFlag && !this.only)) {
305305
this.filtered = true;
306306
this.parent.filteredSubtestCount++;
307307
}
@@ -408,18 +408,22 @@ class Test extends AsyncResource {
408408
}
409409
}
410410

411-
matchesTestNamePatterns() {
412-
const matchesByNameOrParent = ArrayPrototypeSome(testNamePatterns, (re) =>
413-
RegExpPrototypeExec(re, this.name) !== null,
414-
) ||
415-
this.parent?.matchesTestNamePatterns();
411+
testMatchesCriteria() {
412+
const patterns = testNamePatterns ?? testSkipPatterns;
413+
if (patterns === null) return true;
416414

417-
if (matchesByNameOrParent) return true;
415+
const isSkipping = !testNamePatterns; // Name takes precedence over skip
416+
const matchesByNameOrParent = ArrayPrototypeSome(patterns, (re) =>
417+
RegExpPrototypeExec(re, this.name) !== null,
418+
) || this.parent?.testMatchesCriteria();
419+
if (matchesByNameOrParent) {
420+
return !isSkipping;
421+
}
418422

419423
const testNameWithAncestors = StringPrototypeTrim(this.getTestNameWithAncestors());
420-
if (!testNameWithAncestors) return false;
424+
if (!testNameWithAncestors) return !isSkipping;
421425

422-
return ArrayPrototypeSome(testNamePatterns, (re) => RegExpPrototypeExec(re, testNameWithAncestors) !== null);
426+
return ArrayPrototypeSome(patterns, (re) => RegExpPrototypeExec(re, testNameWithAncestors) !== null) === isSkipping;
423427
}
424428

425429
/**
@@ -897,7 +901,7 @@ class Test extends AsyncResource {
897901
this.finished = true;
898902

899903
if (this.parent === this.root &&
900-
this.root.waitingOn > this.root.subtests.length) {
904+
this.root.waitingOn > this.root.subtests.length) {
901905
// At this point all of the tests have finished running. However, there
902906
// might be ref'ed handles keeping the event loop alive. This gives the
903907
// global after() hook a chance to clean them up. The user may also
@@ -987,15 +991,15 @@ class TestHook extends Test {
987991
getRunArgs() {
988992
return this.#args;
989993
}
990-
matchesTestNamePatterns() {
994+
testMatchesCriteria() {
991995
return true;
992996
}
993997
postRun() {
994998
const { error, loc, parentTest: parent } = this;
995999

9961000
// Report failures in the root test's after() hook.
9971001
if (error && parent !== null &&
998-
parent === parent.root && this.hookType === 'after') {
1002+
parent === parent.root && this.hookType === 'after') {
9991003

10001004
if (isTestFailureError(error)) {
10011005
error.failureType = kHookFailure;
@@ -1016,7 +1020,7 @@ class Suite extends Test {
10161020
constructor(options) {
10171021
super(options);
10181022

1019-
if (testNamePatterns !== null && !options.skip) {
1023+
if ((testNamePatterns !== null || testSkipPatterns !== null) && !options.skip) {
10201024
this.fn = options.fn || this.fn;
10211025
this.skipped = false;
10221026
}
@@ -1050,7 +1054,12 @@ class Suite extends Test {
10501054
// tests that it contains - in case of children matching patterns.
10511055
this.filtered = false;
10521056
this.parent.filteredSubtestCount--;
1053-
} else if (testOnlyFlag && testNamePatterns == null && this.filteredSubtestCount === this.subtests.length) {
1057+
} else if (
1058+
testOnlyFlag &&
1059+
testNamePatterns == null &&
1060+
testSkipPatterns == null &&
1061+
this.filteredSubtestCount === this.subtests.length
1062+
) {
10541063
// If no subtests are marked as "only", run them all
10551064
this.filteredSubtestCount = 0;
10561065
}

lib/internal/test_runner/utils.js

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -200,6 +200,7 @@ function parseCommandLine() {
200200
let destinations;
201201
let reporters;
202202
let testNamePatterns;
203+
let testSkipPatterns;
203204
let testOnlyFlag;
204205

205206
if (isChildProcessV8) {
@@ -240,6 +241,9 @@ function parseCommandLine() {
240241
testNamePatternFlag,
241242
(re) => convertStringToRegExp(re, '--test-name-pattern'),
242243
) : null;
244+
const testSkipPatternFlag = getOptionValue('--test-skip-pattern');
245+
testSkipPatterns = testSkipPatternFlag?.length > 0 ?
246+
ArrayPrototypeMap(testSkipPatternFlag, (re) => convertStringToRegExp(re, '--test-skip-pattern')) : null;
243247
}
244248

245249
globalTestOptions = {
@@ -250,6 +254,7 @@ function parseCommandLine() {
250254
sourceMaps,
251255
testOnlyFlag,
252256
testNamePatterns,
257+
testSkipPatterns,
253258
reporters,
254259
destinations,
255260
};

src/node_options.cc

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -661,6 +661,9 @@ EnvironmentOptionsParser::EnvironmentOptionsParser() {
661661
"run test at specific shard",
662662
&EnvironmentOptions::test_shard,
663663
kAllowedInEnvvar);
664+
AddOption("--test-skip-pattern",
665+
"run tests whose name don't match this regular expression",
666+
&EnvironmentOptions::test_skip_pattern);
664667
AddOption("--test-udp-no-try-send", "", // For testing only.
665668
&EnvironmentOptions::test_udp_no_try_send);
666669
AddOption("--throw-deprecation",

src/node_options.h

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -176,6 +176,7 @@ class EnvironmentOptions : public Options {
176176
bool test_only = false;
177177
bool test_udp_no_try_send = false;
178178
std::string test_shard;
179+
std::vector<std::string> test_skip_pattern;
179180
bool throw_deprecation = false;
180181
bool trace_atomics_wait = false;
181182
bool trace_deprecation = false;
Lines changed: 20 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,20 @@
1+
// Flags: --test-skip-pattern=disabled --test-skip-pattern=/no/i
2+
'use strict';
3+
const common = require('../../../common');
4+
const {
5+
describe,
6+
it,
7+
test,
8+
} = require('node:test');
9+
10+
test('top level test disabled', common.mustNotCall());
11+
test('top level skipped test disabled', { skip: true }, common.mustNotCall());
12+
test('top level skipped test enabled', { skip: true }, common.mustNotCall());
13+
it('top level it enabled', common.mustCall());
14+
it('top level it disabled', common.mustNotCall());
15+
it.skip('top level skipped it disabled', common.mustNotCall());
16+
it.skip('top level skipped it enabled', common.mustNotCall());
17+
describe('top level describe', common.mustCall());
18+
describe.skip('top level skipped describe disabled', common.mustNotCall());
19+
describe.skip('top level skipped describe enabled', common.mustNotCall());
20+
test('this will NOt call', common.mustNotCall());

0 commit comments

Comments
 (0)