Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adding --trace-sigint for tests that are timing out in tools/test.py? #53739

Open
joyeecheung opened this issue Jul 5, 2024 · 2 comments
Open
Labels
test Issues and PRs related to the tests. tools Issues and PRs related to the tools directory.

Comments

@joyeecheung
Copy link
Member

joyeecheung commented Jul 5, 2024

It seems recently many flakes are tests that are timing out. In this case, in the CI we usually see nothing but a hint that it is failing because of a timeout. It seems useful to either always run the tests with --trace-sigint, or support re-running the test immediately with --trace-sigint once it times out. (another option would be to use a stress test job separately, but not all configs are supported in stress tests)

Some caveats:

  1. This may work well enough at least for tests that are not timing out because a child process is timing out. For the child process case, maybe it's useful to implement a dedicated environment variable for this (I think simply using NODE_OPTIONS might not be enough since the tests might not be concatenating what's from the environment).
  2. --trace-sigint is implemented in JS and is only printing the main thread's JS stack trace. In this use case I think we are very likely to need the native stack traces too, and for all threads.
  3. Maybe an option built into Node.js isn't the most robust approach and we should implement something external for this?

cc @nodejs/build @nodejs/testing @legendecas (who implemented --trace-int)

@joyeecheung joyeecheung added test Issues and PRs related to the tests. tools Issues and PRs related to the tools directory. labels Jul 5, 2024
@legendecas
Copy link
Member

Always run the tests with --trace-sigint sounds good to me. I will make it print native stack as well.

However, it will not be helpful if the tests are timed out for non busy-loops, like the child process case, and I believe waiting on external resources will be most of the cases (no exact figures at the moment, could be a false assumption).

@joyeecheung
Copy link
Member Author

joyeecheung commented Jul 6, 2024

Actually, on Linux there's eu-stack from elfutils that can dump the native stack traces of all threads in a given process. Not sure about other platforms, or whether it's feasible to install them in our CI infra, though. In addition I think having JS stack traces are still useful..

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
test Issues and PRs related to the tests. tools Issues and PRs related to the tools directory.
Projects
None yet
Development

No branches or pull requests

2 participants