Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

xds: Fix XDS control plane client retry timer backoff duration when connection closes after results are received #11766

Merged
merged 2 commits into from
Dec 19, 2024

Conversation

larry-safran
Copy link
Contributor

@larry-safran larry-safran commented Dec 19, 2024

The backoffPolicy was calculating time since last attempt, but if the stream had been successfully connected and received responses it should delay from the current time (i.e. reset the stopwatch). Changed test so that stopwatch wasn't always 0, which masked the problem.

@larry-safran larry-safran requested a review from ejona86 December 19, 2024 19:38
@ejona86
Copy link
Member

ejona86 commented Dec 19, 2024

It appears this is using the connection-style backoff, which does indeed start the timer at the start of the previous attempt. https://github.com/grpc/grpc/blob/master/doc/connection-backoff.md#proposed-backoff-algorithm

@ejona86
Copy link
Member

ejona86 commented Dec 19, 2024

Nit: include xds: in commit/PR titles when appropriate

@ejona86
Copy link
Member

ejona86 commented Dec 19, 2024

And in this case the title should also say "xds client" in some fashion. The title is not clear as-is.

@larry-safran larry-safran changed the title Fix retry timer backoff duration. xds: Fix retry timer backoff duration. Dec 19, 2024
@ejona86
Copy link
Member

ejona86 commented Dec 19, 2024

@danielzhaotongliu

Copy link
Collaborator

@danielzhaotongliu danielzhaotongliu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.

@@ -446,6 +446,7 @@ private void handleRpcStreamClosed(Status status) {
// Reset the backoff sequence if had received a response, or backoff sequence
// has never been initialized.
retryBackoffPolicy = backoffPolicyProvider.get();
stopwatch.reset();
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Quick question for my understanding, StopWatch itself is not a thread-safe class and requires external synchronization, but in the control plane client it is thread safe because we run it in the synchronization context?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes. We rely heavily on synchronization context.

@larry-safran larry-safran changed the title xds: Fix retry timer backoff duration. xds: Fix XDS control plane client retry timer backoff duration when connection closes after results are received Dec 19, 2024
@larry-safran larry-safran merged commit ef7c2d5 into grpc:master Dec 19, 2024
16 checks passed
@larry-safran larry-safran deleted the fallback branch December 20, 2024 18:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants