-
Notifications
You must be signed in to change notification settings - Fork 8.5k
[Uptime] Increase timeout in attempt to fix skipped a11y test #73078
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Uptime] Increase timeout in attempt to fix skipped a11y test #73078
Conversation
3541ae4 to
526f3d3
Compare
|
Passed a round of flaky test runner: https://kibana-ci.elastic.co/job/kibana+flaky-test-suite-runner/653/ |
526f3d3 to
933cda2
Compare
933cda2 to
9d77f97
Compare
|
Second round of flaky test runner passed: https://kibana-ci.elastic.co/job/kibana+flaky-test-suite-runner/659/ |
|
Pinging @elastic/uptime (Team:uptime) |
|
Third round of flaky test runner passed x100 executions: https://kibana-ci.elastic.co/job/kibana+flaky-test-suite-runner/662/ |
|
@elasticmachine merge upstream |
💚 Build SucceededBuild metrics
History
To update your PR or re-run it, just comment with: |
shahzad31
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM !!
spalger
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I don't see any problems with this, though my instincts say that overriding the default timeouts isn't a good idea. I have no idea how those timeouts are chosen, but it seems like a tricky game to play...
|
@spalger do you recommend removing the custom timeout altogether? This test was just failing because 2kms was too short, but I'm fine with removing that and simply relying on the default. I don't think it will impact anything. I only added the greater value as a quick fix, I didn't really give it much thought. |
In general, yes. I think trying to manage timeouts manually is tricky business and will lead to a great deal of flakiness unless folks are managing them pretty carefully and with the considerations of the potentially very surprising delays we see in CI caused by us running so much at one time in VMs. I'm not necessarily recommending doing it right now, but I think in general it's an anti-pattern to override the defaults unless there's a clear reason for it. |
|
Ok thanks. I can bring this up in my team's next technical sync that we should aim to remove these unless we can provide a justification for each one. |
…c#73078) * Increase timeout in attempt to fix skipped a11y test. * Temporarily only run uptime tests for faster flaky testing. * Uncomment other test suites. * Unskip test and delete comment. Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
…c#73078) * Increase timeout in attempt to fix skipped a11y test. * Temporarily only run uptime tests for faster flaky testing. * Uncomment other test suites. * Unskip test and delete comment. Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
#73346) * Increase timeout in attempt to fix skipped a11y test. * Temporarily only run uptime tests for faster flaky testing. * Uncomment other test suites. * Unskip test and delete comment. Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com> Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
#73345) * Increase timeout in attempt to fix skipped a11y test. * Temporarily only run uptime tests for faster flaky testing. * Uncomment other test suites. * Unskip test and delete comment. Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com> Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
Summary
Code is WIP.Fixes #72994.
We have a few skipped a11y tests in master. At least one of these is resulting of an underlying Uptime helper function, rather than actual availability-related problems. This patch should fix that.
Checklist
Delete any items that are not applicable to this PR.
For maintainers