Repository navigation
Conversation
…tion delayed close (#471) In replicateOverWS, when the wire subscription went 0-length, scheduleClose fired close(1008, ..., intentional=true) unconditionally. That marks the NodeReplicationConnection isFinished/intentionallyUnsubscribed, emits 'finished' (removing it from the worker connections map), and never reconnects. But after a base-copy resync a still-desired peer (its nodeSubscriptions still populated) can briefly map to a 0-length wire subscription, which permanently wedged it connected:false with no reconnect (live 4-node preprod). New pure predicate shouldFinishEmptySubscriptionClose(connection) gates intentional on whether the peer is genuinely no longer desired: a genuine unsubscribe sets nodeSubscriptions to [] (replicator.assignReplicationSource subscribe([],false) and unsubscribeFromNode->unsubscribe()), so empty/absent => finish (idle cleanup unchanged); populated => not finished, so the close handler reconnects and the connection self-heals. Sibling to harper-pro#466/#467 (outbound reconnect-recovery) and #420 (open-but-idle wedge). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
|
Warning You have reached your daily quota limit. Please wait up to 24 hours and I will start processing your requests again! |
Contributor
|
Reviewed; no blockers found. |
…resence, not nodeSubscriptions Codex cross-model review: the wire subscription is a 1:1 map of connection.nodeSubscriptions, so at close time that array is empty in BOTH the genuine-unsubscribe and spurious-empty cases — gating on it was a no-op for the real path. The only two empty-subscribe([]) sources are database removal (assignReplicationSource — database gone, genuinely terminal) and subscribeToNode's nodes.filter(shouldReplicateFromNode) collapsing to [] while the database is still present (spurious, e.g. the #470 self-gate misread for a still-desired peer). So discriminate on database presence: finish only when getDatabases()[databaseName] is gone; an empty subscription while the database is still present stays retryable and self-heals. Preserves the prior behavior exactly for the database-absent case. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Member
Author
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #471
Summary
In
replicateOverWS, when a connection's wire subscription went 0-length,scheduleClosefiredclose(1008, ..., intentional=true)unconditionally.intentional=truemakes theNodeReplicationConnectionclose handler setisFinished/intentionallyUnsubscribed, emit'finished'(removing it from the worker
connectionsmap), and never reconnect. That finishes a connection whosesubscription emptied spuriously — notably when the harper-pro#470 self-record gate misread collapses
subscribeToNode'snodes.filter(shouldReplicateFromNode)to[]for a still-desired peer whosedatabase is still present — permanently stranding the peer at
connected:false(the trigger half of thelive 4-node preprod wedge).
Where to look — the discriminator (revised after Codex review)
My first attempt keyed the decision on
connection.nodeSubscriptions. Codex correctly flagged that asa no-op: the wire subscription is a 1:1
.map()ofconnection.nodeSubscriptions, so at close timethat array is empty in BOTH the genuine and spurious cases (a repopulating
subscribe()would havecleared
delayedClosebefore the timer fired). The only two empty-subscribe([])sources are:replicator.assignReplicationSourceon database removal — the local database is gone. Terminal.subscribeToNode'snodes.filter(shouldReplicateFromNode)collapsing to[]while the database isstill present — spurious. Must stay retryable.
So the fix discriminates on database presence:
shouldFinishEmptySubscriptionClose(databasePresent)=
!databasePresent, withdatabasePresent = getDatabases()?.[databaseName] != nullread at close time.Please sanity-check this discriminator — in particular that there is no genuine-unsubscribe path that
keeps the local database present (the
unsubscribe()user path setsintentionallyUnsubscribeddirectlyand never reaches this delayed close, so it is unaffected). The database-absent (cleanup) behavior is
strictly unchanged from before.
Relationships (open context for the reviewer)
spurious empty subscription. With replication: hdb_nodes point lookups assume sync get() but system is RocksDB (MaybePromise) — self-gate/auth read a Promise, disabling recovery #470 fixed, the filter no longer empties for still-desired peers; this
PR is defense-in-depth against any spurious empty while the database is present.
Tests:
unitTests/replication/emptySubscriptionClose.test.mjs(database-present ⇒ retryable;database-gone ⇒ finish); proven red-without/green-with. Full replication unit suite green (181 passing).
Generated by Claude Code (Claude Opus 4.8).