Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

test: fix test-watch-mode #45585

Merged
merged 1 commit into from
Dec 1, 2022

Conversation

StefanStojanovic
Copy link
Contributor

This PR fixes flakiness in watch mode tests in test/sequential/test-watch-mode.mjs. The changes made introduce a basic synchronization between restarting the process (rewriting the file) and its execution. Previously, it was possible to restart the process while it was still executing which caused logs to appear in an unexpected order, thus failing the assertions in various test cases.

With these changes, one problem still persists. Rarely, in less than 1% of runs, logs from the start of the process, until the first restart are missing. Eg. instead of getting

Completed running 'C:\\_work\\node\\test\\.tmp.0\\12.js'
Restarting 'C:\\_work\\node\\test\\.tmp.0\\12.js'
Completed running 'C:\\_work\\node\\test\\.tmp.0\\12.js'

test gets

Restarting 'C:\\_work\\node\\test\\.tmp.0\\12.js'
Completed running 'C:\\_work\\node\\test\\.tmp.0\\12.js'

This behavior seems to be random, and by the looks of it, it's related to watch mode in a more general way, not directly related to these tests. It should probably be addressed in a separate issue and PR.

Refs: #44898

@nodejs-github-bot nodejs-github-bot added needs-ci PRs that need a full CI run. test Issues and PRs related to the tests. labels Nov 22, 2022
@MoLow MoLow added test_runner Issues and PRs related to the test runner subsystem. watch-mode Issues and PRs related to watch mode and removed test_runner Issues and PRs related to the test runner subsystem. labels Nov 23, 2022
@StefanStojanovic
Copy link
Contributor Author

Hi, is there something I can do to help move this forward? I've seen Test ASan failing, but it seems unrelated to my changes.

@Trott Trott added the request-ci Add this label to start a Jenkins CI on a PR. label Dec 1, 2022
@github-actions github-actions bot removed the request-ci Add this label to start a Jenkins CI on a PR. label Dec 1, 2022
@nodejs-github-bot

This comment was marked as outdated.

@nodejs-github-bot
Copy link
Collaborator

@Trott Trott added the author ready PRs that have at least one approval, no pending requests for changes, and a CI started. label Dec 1, 2022
@Trott Trott added the commit-queue Add this label to land a pull request using GitHub Actions. label Dec 1, 2022
@nodejs-github-bot nodejs-github-bot removed the commit-queue Add this label to land a pull request using GitHub Actions. label Dec 1, 2022
@nodejs-github-bot nodejs-github-bot merged commit 8950240 into nodejs:main Dec 1, 2022
@nodejs-github-bot
Copy link
Collaborator

Landed in 8950240

targos pushed a commit that referenced this pull request Dec 12, 2022
Refs: #44898
PR-URL: #45585
Reviewed-By: Moshe Atlow <moshe@atlow.co.il>
Reviewed-By: Erick Wendel <erick.workspace@gmail.com>
Reviewed-By: Rich Trott <rtrott@gmail.com>
danielleadams pushed a commit that referenced this pull request Dec 30, 2022
Refs: #44898
PR-URL: #45585
Reviewed-By: Moshe Atlow <moshe@atlow.co.il>
Reviewed-By: Erick Wendel <erick.workspace@gmail.com>
Reviewed-By: Rich Trott <rtrott@gmail.com>
danielleadams pushed a commit that referenced this pull request Dec 30, 2022
Refs: #44898
PR-URL: #45585
Reviewed-By: Moshe Atlow <moshe@atlow.co.il>
Reviewed-By: Erick Wendel <erick.workspace@gmail.com>
Reviewed-By: Rich Trott <rtrott@gmail.com>
danielleadams pushed a commit that referenced this pull request Jan 3, 2023
Refs: #44898
PR-URL: #45585
Reviewed-By: Moshe Atlow <moshe@atlow.co.il>
Reviewed-By: Erick Wendel <erick.workspace@gmail.com>
Reviewed-By: Rich Trott <rtrott@gmail.com>
danielleadams pushed a commit that referenced this pull request Jan 4, 2023
Refs: #44898
PR-URL: #45585
Reviewed-By: Moshe Atlow <moshe@atlow.co.il>
Reviewed-By: Erick Wendel <erick.workspace@gmail.com>
Reviewed-By: Rich Trott <rtrott@gmail.com>
danielleadams pushed a commit that referenced this pull request Jan 5, 2023
Refs: #44898
PR-URL: #45585
Reviewed-By: Moshe Atlow <moshe@atlow.co.il>
Reviewed-By: Erick Wendel <erick.workspace@gmail.com>
Reviewed-By: Rich Trott <rtrott@gmail.com>
@tniessen
Copy link
Member

@MoLow Did this indeed fix the flakiness of the test? sequential.status still allows this test to be flaky:

# https://github.com/nodejs/node/issues/44898
test-watch-mode: PASS, FLAKY
test-watch-mode-inspect: PASS, FLAKY

But the referenced issue #44898 has been closed following this PR.

@MoLow
Copy link
Member

MoLow commented Mar 18, 2023

I am not sure what is the way to verify that, it does not appear in the latest reports in https://github.com/nodejs/reliability

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
author ready PRs that have at least one approval, no pending requests for changes, and a CI started. needs-ci PRs that need a full CI run. test Issues and PRs related to the tests. watch-mode Issues and PRs related to watch mode
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants