Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

test_runner: display skipped subtests in spec reporter output #46651

Conversation

richiemccoll
Copy link
Contributor

@richiemccoll richiemccoll commented Feb 14, 2023

This PR highlights skipped tests when running with --test-reporter=spec.

At the moment this is using the same approach as the tap reporter which is to display the # SKIP message. I've also updated the icon/color in the spec style to visually indicate what tests are skipped.

Given the following block of tests, here are some before/after snippets to highlight the minor change.

const { test, describe, it } = require('node:test');

describe('parent block', () => {
  it('child subtest 1 should run', () => {
    console.log('running first');
  });

  it('child subtest 2 should not run', { skip: true }, () => {
    console.log('running second');
  });
});

test('parent block', async (t) => {
  await t.test('child subtest 1 should run', () => {
    console.log('running first');
  });

  await t.test('child subtest 2 should not run', { skip: true }, () => {
    console.log('running second');
  });
});

test('parent block three', (t) => {
  t.skip('this is skipped');
});

Before

▶ parent block
  ✔ child subtest 1 should run (1.365111ms)
  ✔ child subtest 2 should not run (0.08218ms)
▶ parent block (4.580764ms)

▶ parent block
  ✔ child subtest 1 should run (0.157236ms)
  ✔ child subtest 2 should not run (0.081149ms)
▶ parent block (0.634555ms)

✔ parent block three (0.119197ms)
ℹ tests 3
ℹ pass 2
ℹ fail 0
ℹ cancelled 0
ℹ skipped 1
ℹ todo 0

After

▶ parent block
  ✔ child subtest 1 should run (1.403468ms)
  ✔ child subtest 2 should not run (0.090423ms) # SKIP
▶ parent block (5.349161ms)

▶ parent block
  ✔ child subtest 1 should run (0.169557ms)
  ✔ child subtest 2 should not run (0.119462ms) # SKIP
▶ parent block (0.780943ms)

✔ parent block three (0.130612ms) # SKIP
ℹ tests 3
ℹ pass 2
ℹ fail 0
ℹ cancelled 0
ℹ skipped 1
ℹ todo 0
ℹ duration_ms 11.546297

@nodejs-github-bot
Copy link
Collaborator

Review requested:

  • @nodejs/test_runner

@nodejs-github-bot nodejs-github-bot added dont-land-on-v14.x needs-ci PRs that need a full CI run. test_runner Issues and PRs related to the test runner subsystem. labels Feb 14, 2023
@richiemccoll richiemccoll marked this pull request as ready for review February 14, 2023 15:41
@MoLow
Copy link
Member

MoLow commented Feb 14, 2023

I'd prefer a different color and symbol for skipped tests instead of adding more text, WDYT? I'v used an arrow, but also - can be okay.

image

@richiemccoll richiemccoll force-pushed the test-runner/skip-tests-spec-reporter branch from 188f7bb to b523233 Compare February 15, 2023 10:17
@richiemccoll
Copy link
Contributor Author

@MoLow Agree, although I do think having the text does help. I pushed a small adjustment. WDYT?

Screenshot 2023-02-15 at 09 50 08

Copy link
Contributor

@cjihrig cjihrig left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM if @MoLow is ok with it.

@aduh95 aduh95 added author ready PRs that have at least one approval, no pending requests for changes, and a CI started. request-ci Add this label to start a Jenkins CI on a PR. labels Feb 16, 2023
@github-actions github-actions bot removed the request-ci Add this label to start a Jenkins CI on a PR. label Feb 16, 2023
@nodejs-github-bot
Copy link
Collaborator

@richiemccoll richiemccoll force-pushed the test-runner/skip-tests-spec-reporter branch from b523233 to 63cc30e Compare February 17, 2023 11:48
@richiemccoll richiemccoll force-pushed the test-runner/skip-tests-spec-reporter branch from 63cc30e to a65bbaa Compare February 17, 2023 11:53
@MoLow MoLow added the request-ci Add this label to start a Jenkins CI on a PR. label Feb 18, 2023
@github-actions github-actions bot removed the request-ci Add this label to start a Jenkins CI on a PR. label Feb 18, 2023
@nodejs-github-bot
Copy link
Collaborator

@nodejs-github-bot
Copy link
Collaborator

@nodejs-github-bot
Copy link
Collaborator

@MoLow MoLow added the commit-queue Add this label to land a pull request using GitHub Actions. label Feb 18, 2023
@nodejs-github-bot nodejs-github-bot removed the commit-queue Add this label to land a pull request using GitHub Actions. label Feb 18, 2023
@nodejs-github-bot
Copy link
Collaborator

Commit Queue failed
- Loading data for nodejs/node/pull/46651
✔  Done loading data for nodejs/node/pull/46651
----------------------------------- PR info ------------------------------------
Title      test_runner: display skipped subtests in spec reporter output (#46651)
   ⚠  Could not retrieve the email or name of the PR author's from user's GitHub profile!
Branch     richiemccoll:test-runner/skip-tests-spec-reporter -> nodejs:main
Labels     author ready, needs-ci, dont-land-on-v14.x, test_runner
Commits    1
 - test_runner: display skipped tests in spec reporter output
Committers 1
 - richiemccoll 
PR-URL: https://github.com/nodejs/node/pull/46651
Reviewed-By: Colin Ihrig 
Reviewed-By: Moshe Atlow 
------------------------------ Generated metadata ------------------------------
PR-URL: https://github.com/nodejs/node/pull/46651
Reviewed-By: Colin Ihrig 
Reviewed-By: Moshe Atlow 
--------------------------------------------------------------------------------
   ⚠  Commits were pushed since the last review:
   ⚠  - test_runner: display skipped tests in spec reporter output
   ℹ  This PR was created on Tue, 14 Feb 2023 15:37:17 GMT
   ✔  Approvals: 2
   ✔  - Colin Ihrig (@cjihrig) (TSC): https://github.com/nodejs/node/pull/46651#pullrequestreview-1299611688
   ✔  - Moshe Atlow (@MoLow): https://github.com/nodejs/node/pull/46651#pullrequestreview-1300108084
   ✔  Last GitHub CI successful
   ℹ  Last Full PR CI on 2023-02-18T20:42:36Z: https://ci.nodejs.org/job/node-test-pull-request/49708/
- Querying data for job/node-test-pull-request/49708/
   ✔  Last Jenkins CI successful
--------------------------------------------------------------------------------
   ✔  Aborted `git node land` session in /home/runner/work/node/node/.ncu
https://github.com/nodejs/node/actions/runs/4213293471

@nodejs-github-bot nodejs-github-bot added the commit-queue-failed An error occurred while landing this pull request using GitHub Actions. label Feb 18, 2023
@MoLow MoLow added commit-queue Add this label to land a pull request using GitHub Actions. and removed commit-queue-failed An error occurred while landing this pull request using GitHub Actions. labels Feb 18, 2023
@nodejs-github-bot nodejs-github-bot removed the commit-queue Add this label to land a pull request using GitHub Actions. label Feb 18, 2023
@nodejs-github-bot nodejs-github-bot merged commit 261bf96 into nodejs:main Feb 18, 2023
@nodejs-github-bot
Copy link
Collaborator

Landed in 261bf96

MoLow pushed a commit to MoLow/node that referenced this pull request Feb 25, 2023
PR-URL: nodejs#46651
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Moshe Atlow <moshe@atlow.co.il>
MoLow pushed a commit to MoLow/node that referenced this pull request Feb 25, 2023
PR-URL: nodejs#46651
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Moshe Atlow <moshe@atlow.co.il>
@MoLow MoLow added the backport-open-v18.x Indicate that the PR has an open backport. label Feb 25, 2023
MoLow pushed a commit to MoLow/node that referenced this pull request Feb 25, 2023
PR-URL: nodejs#46651
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Moshe Atlow <moshe@atlow.co.il>
juanarbol pushed a commit that referenced this pull request Mar 3, 2023
PR-URL: #46651
Backport-PR-URL: #46839
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Moshe Atlow <moshe@atlow.co.il>
@juanarbol juanarbol mentioned this pull request Mar 3, 2023
juanarbol pushed a commit that referenced this pull request Mar 5, 2023
PR-URL: #46651
Backport-PR-URL: #46839
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Moshe Atlow <moshe@atlow.co.il>
targos pushed a commit that referenced this pull request Mar 13, 2023
PR-URL: #46651
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Moshe Atlow <moshe@atlow.co.il>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
author ready PRs that have at least one approval, no pending requests for changes, and a CI started. backport-open-v18.x Indicate that the PR has an open backport. needs-ci PRs that need a full CI run. test_runner Issues and PRs related to the test runner subsystem.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants