-
Notifications
You must be signed in to change notification settings - Fork 4.8k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
cds_integration_test
is flakey
#30107
Comments
cc @KBaichoo |
cds_integration_test
is flakey (tsan)cds_integration_test
is flakey
seems its not just tsan - failing here in c-t-o [ RUN ] IpVersionsClientTypeDelta/DeferredCreationClusterStatsTest.DeferredCreationTrafficStatsWithClusterCreateUpdateDelete/3
external/envoy/test/integration/cds_integration_test.cc:334: Failure
Expected equality of these values:
test_server_->gauge("cluster.cluster_1.ClusterTrafficStats.initialized")
Which is: 1
nullptr
Which is: (nullptr) not sure if its a different issue - same test failing |
Thanks for raising phlax, I'll assign myself and take a look |
@KBaichoo ive optimistically added to 1.28 milestone - would be good to resolve before release sets sail |
cc @alyssawilk |
I've looked into this a bit, and imo doesn't seem related to #28702 as it fails in cases where that added feature is off. I think the issue is we can race whether the counter exists when the cluster is remove or not and there's no "counter does not exist mechanism" |
yeah its possible it goes back further - i think it does - there was just limited info to track it with, so looked for related activity |
Fail can be seen here https://dev.azure.com/cncf/envoy/_build/results?buildId=151916&view=logs&j=d1f76054-8f79-554b-6f4a-11d6a63b8e00&t=a193292f-96b1-53c3-0505-a923ddcc3f84&l=304
Its happened a couple of times on
main
in the last couple of daysthe first occurrence i can see at least is early september
the most recent changes to related/files to then is #28702
The text was updated successfully, but these errors were encountered: