Repository navigation
[#949] Report a ReplicaOfflineMsg the broker refused as not sent - #976
Merged
vharseko merged 1 commit intoSep 11, 2026
Conversation
…sed as not sent ReplicationDomain.publish() discarded the outcome of broker.publish(), so an announcement the broker never wrote to a session - it has no usable session, the recovery still has to republish the changes which come before it, or it was stopped in between - was still recorded as sent, and the shutdown of a collocated replication server then spent its whole grace period waiting for it to be forwarded. ReplicationBroker.publish() now reports whether the message was written rather than true when its loop ends on the shutdown, ReplicationDomain.publish() passes that answer on, and PendingChanges reports the CSN of the offline message the replication service accepted, so publishReplicaOfflineMsg() announces only what really was sent.
vharseko
force-pushed
the
issues/949-report-a-refused-replica-offline-msg
branch
from
September 9, 2026 08:51
1288ead to
a2dd8e4
Compare
Member
Author
|
Rebased on master, where #946 has landed - the "whichever lands first" of the overlap this PR
Checked on the rebased branch with The description is updated to record the result rather than the plan. One stale reference went |
maximthomas
approved these changes
Sep 11, 2026
vharseko
deleted the
issues/949-report-a-refused-replica-offline-msg
branch
September 11, 2026 13:19
This was referenced Sep 11, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #949
ReplicationDomain.publish()discarded the outcome ofbroker.publish(), so aReplicaOfflineMsgthe broker refused was still recorded as sent:The path
PendingChanges.pushCommittedChanges()publishes the announcement through the domain, and thevoid overload of
ReplicationBroker.publish()returns without touching any session when thereplica has no usable session (
connectionError) or when the changes which come before this onestill have to be republished by the recovery (
connectRequiresRecovery). Its publish loop alsoends when the broker is stopped in between, and it reported that message as published too.
The
connectionErrorpath is the reachable one: a replica whose replication server went awayfirst announces itself offline into a broken session.
What it costs
The same as #918:
DSRSShutdownSyncholds an announcement which is not on the wire, andReplicationServer.shutdown()spends the whole grace period inawaitReplicaOfflineMsgsForwarded()waiting for it to be forwarded. The announcement itself islost as well, so
notifyReplicaOffline()never reaches the changelog of the peer RSs and theirmedium consistency point keeps waiting for changes from a replica which is gone - which is what
OPENDJ-1453 added
DSRSShutdownSyncfor.The change
ReplicationBroker.publish()returnsdonerather thantrue: its loop also ends when thebroker is stopped, and then nothing was written to any session. Four callers already act on that
answer - the export of a total update, and the two initialization requests #861 made check it -
and they now fail on a broker stopped mid-flight instead of waiting for an answer which cannot
come.
ReplicationDomain.publish()reports whether the broker wrote the message. Its other effects stayunconditional on purpose: the domain state says which changes the replica has done, and it is by
finding it ahead of the state its replication server reports that the next session knows which
changes to republish from the historical information of their entries. A change the broker refuses
is recovered that way - the offline announcement is stored nowhere, so it is the one message whose
delivery the caller must know about.
PendingChanges.pushCommittedChanges()therefore reports the CSN of the offline message thereplication service accepted, and
putReplicaOfflineMsg()returns it only when it is the messageit has just queued - the message carries the newest CSN of the replica, so anything else means a
change before it held it back. Either way the message leaves the queue, which is what #946 made
putReplicaOfflineMsg()do: there is nobody left to publish it afterwards, and it must notsurface on the session which follows.
publishReplicaOfflineMsg()records only what was reallysent, and traces the case where the replica could not announce itself: an operator looking at a
peer RS which never got the offline CSN otherwise has nothing to go on.
Rebased on #946
#946 has landed (
bb6e7c5816), and this branch is rebased on it - the "whichever lands first"of the overlap this PR described. The two verdicts became one: the answer now comes from
pushCommittedChanges(), so the message the broker refused is reported as not sent as well asthe message a change in flight held back, and the unconditional
pendingChanges.remove()of #946still drops what was not published. A message which could not be published is both dropped and
reported as not sent, and the trace of
publishReplicaOfflineMsg()names both reasons. The testclass is the union of the two.
Not fixed here
finds nothing to clear - A ReplicaOfflineMsg forwarded before it is recorded leaves a pending announcement nothing will clear, and the shutdown waits out its grace period #950.
ReplicaOfflineMsgbranch ofpushCommittedChanges()still has no recovery guard, unlikethe branch above it. The broker refuses the message while a recovery is pending, and that
refusal is now reported rather than swallowed, but whether the announcement should wait for the
recovery to finish deserves its own decision.
Tests
PendingChangesTest, the class #946 added, grows to five cases: the announcement the brokerpublished is reported as sent, the one it refused is not, the one a change in flight holds back is
not even attempted, a message which could not be sent is not published later on the session which
follows, and a change the broker refused still leaves the pending changes - it is the recovery,
not the queue, which publishes it again.