Skip to content

Conversation

mdumandag
Copy link
Contributor

In split-brain scenarios, the cluster id stays the same for different halves of the split.

When the client disconnects from the first half it was connected to and reconnects to the other half, from the point of view of the client, it connects to the same cluster.

However, it might very well be the case that, the client has sent some state to the first half of the cluster, and that state must be replicated to the other half when the client reconnects to it.

This is especially needed for Compact schemas.

In this PR, we are sending the client state to the cluster after reconnections, regardless it is connected back to possibly the same cluster with the same id or not.

We couldn't add proper split brain tests, as the remote controller does not have that ability yet. However, I have added a simple test that verifies the current behavior.

closes #575

In split-brain scenarios, the cluster id stays the same for
different halves of the split.

When the client disconnects from the first half it was connected
to and reconnects to the other half, from the point of view of
the client, it connects to the same cluster.

However, it might very well be the case that, the client has sent
some state to the first half of the cluster, and that state must
be replicated to the other half when the client reconnects to it.

This is especially needed for Compact schemas.

In this PR, we are sending the client state to the cluster after
reconnections, regardless it is connected back to possibly the same
cluster with the same id or not.

We couldn't add proper split brain tests, as the remote controller
does not have that ability yet. However, I have added a simple
test that verifies the current behavior.
@mdumandag mdumandag added this to the 5.2.0 milestone Mar 9, 2023
@mdumandag mdumandag requested a review from yuce March 9, 2023 15:34
@mdumandag mdumandag self-assigned this Mar 9, 2023
@codecov-commenter
Copy link

Codecov Report

Merging #613 (82a7470) into master (b2196d4) will increase coverage by 0.00%.
The diff coverage is 100.00%.

@@           Coverage Diff           @@
##           master     #613   +/-   ##
=======================================
  Coverage   96.47%   96.47%           
=======================================
  Files         357      357           
  Lines       20515    20519    +4     
=======================================
+ Hits        19791    19796    +5     
+ Misses        724      723    -1     
Impacted Files Coverage Δ
hazelcast/connection.py 92.24% <100.00%> (-0.28%) ⬇️
hazelcast/invocation.py 92.75% <0.00%> (-0.73%) ⬇️
hazelcast/statistics.py 88.20% <0.00%> (+0.56%) ⬆️
hazelcast/listener.py 92.39% <0.00%> (+2.17%) ⬆️

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

Copy link
Contributor

@yuce yuce left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good!

@mdumandag mdumandag merged commit b728a54 into hazelcast:master Mar 10, 2023
@mdumandag mdumandag deleted the split-brain-client-state branch March 10, 2023 15:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[TRACKING ISSUE] [API-1531] Send client state to cluster in re-connections
3 participants