-
Notifications
You must be signed in to change notification settings - Fork 112
fix: flaky metric test #2472
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
fix: flaky metric test #2472
Conversation
Summary of ChangesHello @alkatrivedi, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed! This pull request addresses a flakiness issue in a metric test by refining an assertion logic. The change ensures that the test accurately reflects the expected relationship between attempt and operation latencies, improving the reliability of the test suite and preventing intermittent failures. Highlights
Using Gemini Code AssistThe full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips. Invoking Gemini You can request assistance from Gemini at any point by creating a comment using either
Customization To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a Limitations & Feedback Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here. You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension. Footnotes
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Code Review
This pull request addresses a flaky test by correctly asserting that attemptLatency should be approximately equal to operationLatency when there is only a single attempt. The change is correct and improves test stability.
I've identified a similar issue in another test within the same file. The test 'should have correct latency values in metrics except AFE when AFE Server timing is disabled' in test/metrics/metrics.ts has the same logic on line 713 that was just fixed. It still compares attemptLatency with MIN_LATENCY. To prevent future flakiness, this should also be updated to compare against operationLatency.
59941d3 to
24a2d5b
Compare
| ); | ||
| // Since we only have one attempt, the attempt latency should be fairly close to the operation latency | ||
| assertApprox(MIN_LATENCY, attemptLatency, 30); | ||
| assertApprox(operationLatency, attemptLatency, 30); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There are 2 test cases which uses this MIN_LATENCY, both needs to be corrected
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
done
test/metrics/metrics.ts
Outdated
|
|
||
| // check that the latency matches up with the measured elapsed time within 10ms | ||
| assertApprox(elapsedTime, totalOperationLatency, 10); | ||
| assertApprox(elapsedTime, totalOperationLatency, 50); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This doesn't seem right to me, Unit test should not take this long, ideally elapsed time should always be closer to totalOperationLatency.
In your testing what behavior do you see, totalOperationLatency is bigger or elapsed time ?
Can you add logs and see what all operationLatency value is coming for different methods vs the total latency.
We need to see for this test case if createSession is called when database.run is called or before that ? Maybe that is why we see this discrepancy now with mux session.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Check what all "resourceMetrics" contains, is it having createSession or not and how it differs from regular sessio
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
In your testing what behavior do you see, totalOperationLatency is bigger or elapsed time ?
elapsedTime is more than the totalOperationLatency. Ref - https://github.com/googleapis/nodejs-spanner/actions/runs/19628416394/job/56202180201
24a2d5b to
dde6277
Compare
dde6277 to
b02dc21
Compare
b02dc21 to
2548c4d
Compare
No description provided.