Node 22 ignores `--test-name-pattern`

[ai]

Version

v22.3.0

Platform

Linux savoia 6.9.5-200.fc40.x86_64 #1 SMP PREEMPT_DYNAMIC Sun Jun 16 15:47:09 UTC 2024 x86_64 GNU/Linux

Subsystem

test

What steps will reproduce the bug?

one.test.js:

import { equal } from 'node:assert'
import { test } from 'node:test'

test('one', () => {
  equal(1, 1)
})

$ node --test-name-pattern="wehadnosuchtextintest" --test one.test.js

How often does it reproduce? Is there a required condition?

Always

What is the expected behavior? Why is that the expected behavior?

To have skipped 1 in the output and one test should not be run

What do you see instead?

$ node --test-name-pattern="wehadnosuchtextintest" --test one.test.js
✔ one (0.66693ms)
ℹ tests 1
ℹ suites 0
ℹ pass 1
ℹ fail 0
ℹ cancelled 0
ℹ skipped 0
ℹ todo 0
ℹ duration_ms 39.554192

Additional information

Here is a simple project where I found that issue

ai/better-node-test#3

[cjihrig]

@redyetidev

[avivkeller]

I'm not seeing the issue. --test-skip-pattern will skip any tests matching the pattern. None of your tests are matching the pattern.

If you want to only run tests with a pattern, use --test-name-pattern.

If I'm missing something, let me know

[avivkeller]

Even with --test-name-pattern, the test is skipped.

[ai]

Sorry, I fixed the issue.

$ node --test-name-pattern="wehadnosuchtextintest" --test one.test.js
✔ one (0.66693ms)
ℹ tests 1
ℹ suites 0
ℹ pass 1
ℹ fail 0
ℹ cancelled 0
ℹ skipped 0
ℹ todo 0
ℹ duration_ms 39.554192

With --test-name-pattern I have the same result (all tests are running).

[avivkeller]

The test isn't running, but I do see it's marked as pass and not skipped.

You can tell it's not running:

import { equal } from 'node:assert'
import { test } from 'node:test'

test('one', () => {
  console.log("TEST")
  equal(1, 1)
})

TEST won't be logged.

[ai]

Yes, the problem is with output

[avivkeller]

@cjihrig You worked on --test-name-pattern, right? If so, is it supposed to be logged under pass or skip?

FWIW --test-skip-pattern has the same behavior, so It doesn't seem like a bug right away.

[cjihrig]

Oh, I see what is going on here. There is no bug. There is one test file being run, which is passing. When there are no tests executed in a file, the test runner reports that the file ran (pass/fail is determined by that file's exit code).

[ai]

Weird. Why don’t we count inactive test() calls?

[cjihrig]

This behavior was changed in v22 at the request of users. When filtering is used (--test-name-pattern, --test-skip-pattern, --test-only) tests that are not run are not included in the output (it is like they don't exist at all). The reasoning is that when you have a really large test suite, and you apply filtering, all of the skip output clutters things up. And, if we suppress test output but still include them in the counts, that is even more confusing.

I would not be opposed to adding another line in the summary for filtered test count (and test file count for that matter), but it's probably not something I'll work on personally any time soon.

[avivkeller]

I would not be opposed to adding another line in the summary for filtered test count (and test file count for that matter), but it's probably not something I'll work on personally any time soon.

I also wouldn't be opposed to it (not that that really matters), but that (IMO) should be discussed in a seperate issue

---

As for this issue, I'm closing this as it appears to be resolved (working as intended).

If you (@ai) would like to work on a change that modifies the code, feel free to open a PR, and if you want a new feature, feel free to open an issue describing it.

If someone disagrees, please re-open the issue