-
Notifications
You must be signed in to change notification settings - Fork 29.6k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
test_runner: flatten TAP output when running using --test
#46440
Conversation
Review requested:
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Left some comments and have some concerns about the edge cases (which I left comments on), but looking nice.
lib/internal/test_runner/test.js
Outdated
if (subtest.skipReporting || subtest.counted) { | ||
return; | ||
} | ||
subtest.counted = true; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I really dislike this block of code for a few reasons:
skipReporting
is defined on a subclass ofTest
. It seems like that logic should be contained to theFileTest
class.counted
changes the V8 shape of theTest
objects. If this were going to be added, it should happen in the constructor, but...- The test runner is getting more and more of these checks added to determine if something already ran (via
once()
and other boolean checks). To me, this kind of implies that we don't actually know when code is executing, plus it slows things down. Shouldn't this only be called once per subtest?
602e6bb
to
0c377ea
Compare
@cjihrig I have pushed some changes addressing your feedback |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM. Only one comment that I think we should address before landing (regarding testTimeoutFailure
s).
a64dbbc
to
bd1e3ba
Compare
Commit Queue failed- Loading data for nodejs/node/pull/46440 ✔ Done loading data for nodejs/node/pull/46440 ----------------------------------- PR info ------------------------------------ Title test_runner: flatten TAP output when running using `--test` (#46440) Author Moshe Atlow (@MoLow) Branch MoLow:flatten-tap-output -> nodejs:main Labels author ready, needs-ci, dont-land-on-v14.x, commit-queue-squash, test_runner Commits 2 - test_runner: flatten TAP output when running using `--test` - CR Committers 1 - Moshe Atlow PR-URL: https://github.com/nodejs/node/pull/46440 Fixes: https://github.com/nodejs/node/issues/45833 Refs: https://github.com/nodejs/node/issues/45833 Reviewed-By: Colin Ihrig Reviewed-By: Benjamin Gruenbaum ------------------------------ Generated metadata ------------------------------ PR-URL: https://github.com/nodejs/node/pull/46440 Fixes: https://github.com/nodejs/node/issues/45833 Refs: https://github.com/nodejs/node/issues/45833 Reviewed-By: Colin Ihrig Reviewed-By: Benjamin Gruenbaum -------------------------------------------------------------------------------- ⚠ Commits were pushed since the last review: ⚠ - CR ℹ This PR was created on Tue, 31 Jan 2023 09:45:05 GMT ✔ Approvals: 2 ✔ - Colin Ihrig (@cjihrig) (TSC): https://github.com/nodejs/node/pull/46440#pullrequestreview-1294610234 ✔ - Benjamin Gruenbaum (@benjamingr): https://github.com/nodejs/node/pull/46440#pullrequestreview-1294768331 ✔ Last GitHub CI successful ℹ Last Full PR CI on 2023-02-17T11:14:48Z: https://ci.nodejs.org/job/node-test-pull-request/49624/ - Querying data for job/node-test-pull-request/49624/ ✔ Last Jenkins CI successful -------------------------------------------------------------------------------- ✔ Aborted `git node land` session in /home/runner/work/node/node/.ncuhttps://github.com/nodejs/node/actions/runs/4205387741 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Fresh LGTM to appease the commit queue bot.
Landed in 3354f89 |
This doesn't land cleanly on v19.x, could it please be backported |
@MylesBorins |
PR-URL: nodejs#46440 Fixes: nodejs#45833 Refs: nodejs#45833 Reviewed-By: Colin Ihrig <cjihrig@gmail.com> Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
PR-URL: nodejs#46440 Fixes: nodejs#45833 Refs: nodejs#45833 Reviewed-By: Colin Ihrig <cjihrig@gmail.com> Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
PR-URL: nodejs#46440 Fixes: nodejs#45833 Refs: nodejs#45833 Reviewed-By: Colin Ihrig <cjihrig@gmail.com> Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
PR-URL: nodejs#46440 Fixes: nodejs#45833 Refs: nodejs#45833 Reviewed-By: Colin Ihrig <cjihrig@gmail.com> Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
PR-URL: nodejs#46440 Fixes: nodejs#45833 Refs: nodejs#45833 Reviewed-By: Colin Ihrig <cjihrig@gmail.com> Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
PR-URL: nodejs#46440 Fixes: nodejs#45833 Refs: nodejs#45833 Reviewed-By: Colin Ihrig <cjihrig@gmail.com> Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Refs: #45833 (comment)
Fixes: #45833
this removes the reporting of a test file and reports its children's tests directly
A few notable things/edge cases:
also diff of
test_runner_output_cli.out
is 10X more readable when diff is set to ignore whitespaces