Repository navigation
fix(replication): close copy-cursor crash + recover wedged followers - #326
Conversation
…(harper-pro#321) Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
There was a problem hiding this comment.
Code Review
This pull request addresses replication issues related to malformed cursors and undefined tables. It introduces optional chaining when checking the copy-resume table to prevent runtime errors, and discards malformed copy-resume cursors on disk to trigger a clean full copy. The review feedback points out that while optional-chaining tables prevents a crash at line 1716, a similar TypeError could still occur later at line 1855 if tables is undefined, suggesting that this reference should also be guarded.
Important
The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.
|
Reviewed; no blockers found. |
heskew
left a comment
There was a problem hiding this comment.
Nice work running both manifestations down — I traced the logic and it holds up: tableToTableEntry(undefined) is falsy so the guard recopies instead of throwing, for…in over an undefined tables is a safe no-op, and copyStartTime is captured before copyResume is reset so the resume anchor survives. The disk-cursor cleanup matches the existing un-awaited remove, too.
Two small things, neither blocking:
- Include the repro test. You mention
badCursorRepro.test.mjsalready exists and exercises both branches deterministically (no dependence on the flaky natural #321 trigger). Since our standard is coverage for changed functions, that's exactly what belongs in this PR rather than left in the working tree. - Trim a couple comments so they don't rot (suggestions inline) — a few pin moment-in-time details that'll drift.
Thanks for the detailed write-up; made this easy to follow.
🤖 Posted by Claude on Nathan's behalf
… trims, unit test) Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
@heskew thanks for the careful trace — both items addressed in 1ab03b1: Comments tightened with your exact suggested wording (no "silent" overclaim at L1712, no L1701 anchor in the comment after L1716). Regression test added at Also picked up Gemini's L1855 finding in the same commit ( |
Summary
Two surgical changes in
replication/replicationConnection.tsthat close the remaining two manifestations of harper-pro#321 not covered by #322's write-site guard.#322 stops new malformed cursors from being persisted. This PR closes the silent crash variant and recovers clusters already wedged on a corrupt cursor.
What this fixes
(b) Silent channel death at L1712 —
tablesundefined throwThe resume-validity check at L1712 was
tables[copyResume.currentTable]. On a freshly-joined peer,tablesitself can beundefined, andtables[…]then throwsTypeError: Cannot read properties of undefined (reading 'undefined')(char 55 of the original line). The throw is caught by the outer.catchat L1874-L1877 which callsclose(1008, ...)— a silent subscription teardown: no warn log on the leader, no restart-loop visible, just a closed channel that takes the data direction with it. The colleague's bench captured exactly this stack signature.Change: optional-chain
tablestotables?.[copyResume.currentTable]. Whentablesis undefined the index returnsundefined,tableToTableEntry(undefined)is falsy, and we fall into the existing warn-and-recopy branch (already correct:copyStartTimeis preserved fromcopyResume?.copyStartTimeat L1701, so the post-copy resume point doesn't lose its anchor). Same branch also picks up the malformed-currentTablecase (a no-op cleanup behavior change sincetables[undefined]was already falsy in that case).(c) Recovery for already-wedged followers at L2420
A cluster that's already looping with a corrupt cursor on disk does not recover from #322's fix alone — that fix only prevents new poisoning. The persisted entry stays, gets sent verbatim in every subscription request, and the leader's warn-and-recopy branch runs every reconnect. In the production failure surface the colleague reported (spectrum-et, system DB 800M+, "67 MB SSTs landing each cycle but never finalizing") COPY_COMPLETE may never arrive while the loop is active, so the cursor never gets cleared.
Change: right after
dbisDB.get([Symbol.for('copyCursor'), nodeId]), ifcopyCursor && !copyCursor.currentTable, warn,dbisDB.remove(...), and treatcopyCursor = undefinedfor the rest of the flow. Next reconnect requests a clean full copy instead of forwarding the malformed cursor. One-shot recovery, no protocol bump.What this does NOT touch
event.tablebecomingundefinedat the write site (still unknown per replication: malformed copy-resume cursor (currentTable undefined) causes permanent full-copy restart loop #321 and fix(replication): guard copy cursor write against undefined table name #322's note).badCursorRepro.test.mjs(kept in working tree, not in this PR) exercises both branches manually.Verification
npm run build: dist emitted with both changes; no new TS errors. Pre-existing_socket/Numeric/code/isHandlederrors unchanged.npm run lint: pre-existing lint errors unchanged; onlyreplicationConnection.tsmention is an unrelatedsubscriptionRequestnever-assigned warning at L693.Test plan
Cannot read properties of undefined (reading 'undefined')no longer appears in error logs for peers with undefinedtables🤖 Generated with Claude Code