test: reduce CI flakiness via span-race filters - #8087
Conversation
Overall package sizeSelf size: 5.56 MB Dependency sizes| name | version | self size | total size | |------|---------|-----------|------------| | import-in-the-middle | 3.0.1 | 82.56 kB | 817.39 kB | | dc-polyfill | 0.1.10 | 26.73 kB | 26.73 kB |🤖 This report was automatically generated by heaviest-objects-in-the-universe |
|
✨ Fix all issues with BitsAI or with Cursor
|
BenchmarksBenchmark execution time: 2026-04-27 17:04:46 Comparing candidate commit 0b826e7 in PR branch Found 0 performance improvements and 0 performance regressions! Performance is the same for 1339 metrics, 105 unstable metrics. |
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## master #8087 +/- ##
=======================================
Coverage 73.79% 73.79%
=======================================
Files 783 783
Lines 36369 36369
=======================================
Hits 26840 26840
Misses 9529 9529 Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 0e40bad879
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
0e40bad to
215aea4
Compare
Caching `checkAgentStatus()` in the test helper removes the dominant cause of the "Timeout of 5000ms exceeded" hook flakes: without it a stressed test agent that answers `/info` in a few hundred ms makes every `beforeEach(agent.load(...))` that slow and the 5 s mocha hook budget is quickly exhausted. Probing once per worker, gating the probe on `DD_TEST_AGENT_URL` (set only by `.github/actions/testagent/ start`), and deferring the await to the trace-forwarding handler unblocks `agent.load()` even when the test agent is slow or unreachable. Under the same conditions a 20-cycle plugin suite goes from ~6.4 s to ~0.6 s, and workflows and local runs that don't start a test agent skip the probe entirely. Plus real race fixes that stop assuming `traces[0][0]` is the span we're looking for across kafkajs, confluentinc-kafka-javascript, redis, ioredis, mariadb, mysql, mysql2, graphql (parse + validate), aws-sdk s3 span-pointers, elasticsearch, opensearch, mongodb-core, pg, prisma, iovalkey, and the IAST `testOutsideRequestHasVulnerability` and `checkVulnerabilityInRequest` helpers. Moleculer's `startBroker` now retries on EADDRINUSE to cover the documented port race, and the rhea `before` hook finally returns the `agent.load` promise so mocha actually waits for plugin registration.
542f0be to
e8d8308
Compare
BridgeAR
changed the title
|
I limited this to span race filters and I move the agent changes to a separate PR |
| agent.assertSomeTraces(traces => { | ||
| try { | ||
| const span = traces[0][0] | ||
| const span = traces[0].find(s => s.meta?.['aws.operation'] === 'putObject') |
There was a problem hiding this comment.
all changes look good to me, tho im wondering if we should consider a helper function for matching the correct span assertion?
There was a problem hiding this comment.
I think that makes sense, yes.
We have something like that already and improving our current mechanism is good!
That way the tests would generally run a tad faster because we generate less intermediate errors.
Right now, I would land it as is as stop gap.
CarlesDD
left a comment
CarlesDD
left a comment
There was a problem hiding this comment.
LGTM from IAST prespective
This addresses real span race conditions. Assuming `traces[0][0]` is the span we're looking for across kafkajs, confluentinc-kafka-javascript, redis, ioredis, mariadb, mysql, mysql2, graphql (parse + validate), aws-sdk s3 span-pointers, elasticsearch, opensearch, mongodb-core, pg, prisma, iovalkey, and the IAST `testOutsideRequestHasVulnerability` and `checkVulnerabilityInRequest` helpers is not always correct. Moleculer's `startBroker` now prevents the port race by using the the zero port, and the rhea `before` hook finally returns the `agent.load` promise so mocha actually waits for plugin registration.
This addresses real span race conditions. Assuming `traces[0][0]` is the span we're looking for across kafkajs, confluentinc-kafka-javascript, redis, ioredis, mariadb, mysql, mysql2, graphql (parse + validate), aws-sdk s3 span-pointers, elasticsearch, opensearch, mongodb-core, pg, prisma, iovalkey, and the IAST `testOutsideRequestHasVulnerability` and `checkVulnerabilityInRequest` helpers is not always correct. Moleculer's `startBroker` now prevents the port race by using the the zero port, and the rhea `before` hook finally returns the `agent.load` promise so mocha actually waits for plugin registration.
4 participants
A real race fixes that stop assuming
traces[0][0]is the span we're looking for across kafkajs, confluentinc-kafka-javascript, redis, ioredis, mariadb, mysql, mysql2, graphql (parse + validate), aws-sdk s3 span-pointers, elasticsearch, opensearch, mongodb-core, pg, prisma, iovalkey, and the IASTtestOutsideRequestHasVulnerabilityandcheckVulnerabilityInRequesthelpers. Moleculer'sstartBrokernow retries on EADDRINUSE to cover the documented port race, and the rheabeforehook finally returns theagent.loadpromise so mocha actually waits for plugin registration.