-
-
Notifications
You must be signed in to change notification settings - Fork 31.9k
test: add tests for REPL custom evals (second attempt) #57850
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
test: add tests for REPL custom evals (second attempt) #57850
Conversation
8c394e5
to
9413135
Compare
9413135
to
1e33c67
Compare
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## main #57850 +/- ##
==========================================
- Coverage 90.17% 90.16% -0.01%
==========================================
Files 636 637 +1
Lines 188028 188122 +94
Branches 36895 36908 +13
==========================================
+ Hits 169547 169622 +75
- Misses 11233 11237 +4
- Partials 7248 7263 +15
🚀 New features to boost your workflow:
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
There are still failures. Here is one in the latest CI run: https://ci.nodejs.org/job/node-test-commit-linux-containered/50117/nodes=ubuntu2204_sharedlibs_withoutssl_x64/testReport/(root)/parallel/test_repl_custom_eval/
|
oof... 😓 sorry about that, thanks @lpinca, I'll have a look 🙏 |
1e33c67
to
584b6d9
Compare
@lpinca I couldn't really reproduce the issue locally but I refactored the code to make it more robust regarding timing, I hope that this will fix the CI check, could you re-run the tests? 🙏 |
Failed to start CI⚠ Commits were pushed since the last approving review: ⚠ - test: add tests for REPL custom evals ✘ Refusing to run CI on potentially unsafe PRhttps://github.com/nodejs/node/actions/runs/14532544242 |
Failed to start CI⚠ Commits were pushed since the last approving review: ⚠ - test: add tests for REPL custom evals ✘ Refusing to run CI on potentially unsafe PRhttps://github.com/nodejs/node/actions/runs/14833744376 |
584b6d9
to
8196294
Compare
5ad76ef
to
9bd4725
Compare
this commit reintroduces the REPL custom eval tests that have been introduced in nodejs#57691 but reverted in nodejs#57793 the tests turned out problematic before because `getReplOutput`, the function used to return the repl output wasn't taking into account that input processing and output emitting are asynchronous operation can resolve with a small delay the new implementation here replaces `getReplOutput` with `getReplRunOutput` that resolves repl inputs by running them and using the repl prompt as an indicator to when the input processing has completed
9bd4725
to
a5ef49b
Compare
@jasnell, @BridgeAR and @gurgunday I've significantly changed the code since your last approval (for the better, the tests now are very much faster and more reliable than my previous iterations), could you please give this one a re-review when you get the chance? 🙏 |
This comment was marked as outdated.
This comment was marked as outdated.
This comment was marked as outdated.
This comment was marked as outdated.
This comment was marked as outdated.
This comment was marked as outdated.
This comment was marked as outdated.
This comment was marked as outdated.
Landed in d6dade5 |
this commit reintroduces the REPL custom eval tests that have been introduced in #57691 but reverted in #57793 the tests turned out problematic before because `getReplOutput`, the function used to return the repl output wasn't taking into account that input processing and output emitting are asynchronous operation can resolve with a small delay the new implementation here replaces `getReplOutput` with `getReplRunOutput` that resolves repl inputs by running them and using the repl prompt as an indicator to when the input processing has completed PR-URL: #57850 Reviewed-By: James M Snell <jasnell@gmail.com> Reviewed-By: Ruben Bridgewater <ruben@bridgewater.de>
this PR reintroduces the REPL custom eval tests that have been introduced in #57691 but reverted in #57793
the tests were working fine locally but caused failures in CI (like this one) the problem, I believe, being that the
getReplOutput
would return theoutput
value right away even though the repl processing is asynchronous. Some small delay there is, I imagine, present in the CI runs and that's what I think was causing the issue.I addressed this problem by returning a promise that resolves to the repl's output using the repl prompt as the signal to understand when the input processing has concluded, instead of returning the output right away.