Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Reserve more CPU cores for integration tests with high rate of flakes #27257

Merged
merged 2 commits into from
May 9, 2023

Conversation

yanavlasov
Copy link
Contributor

Additional Description:
This should reduce the rate of timing related flakes (which it does in local runs). However to take full advantage of CPU reservation all integration tests need to be configured with more CPU cores.

Risk Level: Low
Testing: Unit Tests
Docs Changes: N/A
Release Notes: N/A
Platform Specific Features: N/A

Signed-off-by: Yan Avlasov <yavlasov@google.com>
@@ -40,6 +40,9 @@ envoy_extension_cc_test(
size = "large",
srcs = ["local_ratelimit_integration_test.cc"],
extension_names = ["envoy.filters.http.local_ratelimit"],
tags = [
"cpu:3",
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i experimentally added something similar here https://github.com/envoyproxy/envoy/pull/27243/files#diff-3ca343c3f15cbe6a9354d5c44647e85b92d118a4a01123b470c7866cc5c088adR62

i wasnt 100% clear how this works but iiuc its indicative to bazel - bazel schedules less local jobs according to their cpu reservation

i think in the RBE case it doesnt help - we get mostly 2-core workers (ive seen sometimes 16-core also but infrequently) - but i think this flag doesnt come into to play in that case

also ~related #27243 (comment)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It does help in RBE case as well. It will prevent scheduling two integration test processes from running on the same two core VM. Reduce CPU over subscription by half.

Signed-off-by: Yan Avlasov <yavlasov@google.com>
@RyanTheOptimist
Copy link
Contributor

/assign @phlax

Copy link
Member

@phlax phlax left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm, thanks @yanavlasov

@phlax phlax merged commit 20ca3ad into envoyproxy:main May 9, 2023
@yanavlasov yanavlasov deleted the more-cpu-for-integration-tests branch May 9, 2023 16:58
wbpcode pushed a commit to wbpcode/envoy that referenced this pull request Jun 12, 2023
phlax pushed a commit that referenced this pull request Jun 12, 2023
reskin89 pushed a commit to reskin89/envoy that referenced this pull request Jul 11, 2023
…envoyproxy#27257)

Signed-off-by: Yan Avlasov <yavlasov@google.com>
Signed-off-by: Ryan Eskin <ryan.eskin89@protonmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants