Repository navigation
deploy_component: fail the deploy on peer replication failures (non-zero exit) - #1334
Merged
Merged
Conversation
deploy_component recorded per-node replication status in peer_results but
never let a peer failure change the overall outcome: the deploy finished
'success', returned 2xx / an SSE done event, and the CLI exited 0 even when
the package failed to replicate to the cluster.
Treat any failed peer as an overall deploy failure (non-2xx → non-zero CLI
exit) by default, with an ignore_replication_errors opt-out for best-effort
deploys. The component is still deployed and (if requested) restarted on the
origin; restart runs before the failure is reported. The failure response
carries failed_peers + deployment_id on both the SSE error event and the
non-SSE structured body.
Also fix normalizePeerResult to read the replicator's { status: 'failed',
reason, node } shape — failure detail lives on `reason`, not `error`, so
without this the failure message and failed_peers reported "unknown error".
Closes #1333
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This comment has been minimized.
This comment has been minimized.
Contributor
|
Reviewed; no blockers found. |
If recorder.finish('failed') rejects (full disk, lock, dropped system
table), the original outErr — carrying phase, install output, and
failed_peers — was swallowed and replaced by the DB write error. Wrap
finish() so outErr always propagates; log the record failure.
Surfaced by the Gemini cross-model review leg.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
kriszyp
requested review from
cb1kenobi and
hdbjeff
and removed request for
cb1kenobi
June 17, 2026 01:31
kriszyp
marked this pull request as ready for review
June 17, 2026 01:35
This comment has been minimized.
This comment has been minimized.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #1333
Summary
deploy_componentrecords per-node replication status inpeer_resultsbut never let a peer failure change the overall outcome — the deploy finishedsuccess, returned 2xx (SSEdone), and the CLI exited 0 even when the package failed to replicate to the cluster.This treats any failed peer as an overall deploy failure (non-2xx → non-zero CLI exit) by default, with an
ignore_replication_errors: trueopt-out for best-effort deploys to partially-available clusters. The component is still deployed and (if requested) restarted on the origin node; the restart runs before the failure is reported. The failure response carriesfailed_peers+deployment_idon both the SSEerrorevent and the non-SSE structured body, so callers see which nodes failed and why without a secondget_deploymentround-trip.Changes
components/operations.js— after the replicate (and restart) phase, throw whengetFailedPeers()is non-empty andignore_replication_errorsis not set; the existing catch wraps it as a 500, finishes the rowfailed, and surfacesfailed_peers/deployment_idon both transports. Also wraprecorder.finish('failed')so a row-write error can't mask the real deploy error.components/deploymentRecorder.ts— newgetFailedPeers()helper;normalizePeerResultnow reads the replicator's{ status: 'failed', reason, node }shape (failure detail is onreason, noterror).components/operationsValidation.js—ignore_replication_errorsboolean added to the deploy validator.getFailedPeers, thereason-shaped peer result, and a bare{ status: 'failed' }.Where to look / open questions for the reviewer
normalizePeerResultchange is the highest-leverage bit: without it the failure message andfailed_peersalways read "unknown error", becausereplicateOperation(harper-proreplicator.ts) reports per-peer failures viareason, noterror. Worth confirming the shape against the replicator you have in mind.hdb_deploymentrowstatus = 'failed'. Per-peer granularity is preserved inpeer_results, but the aggregatestatusalone can't distinguish "origin failed" from "origin OK, one peer offline". Raised in deploy_component: treat replication failures as an overall deploy failure (non-zero exit code) #1333 as an open question — kept asfailedhere to match the requested "overall failure" semantics; happy to add areplication_failed/partialstatus if preferred.validateBySchemadiscards Joi's coerced value, so a raw-HTTP JSONignore_replication_errors: "false"(string) would be truthy and suppress errors. Affects all boolean deploy flags (force,install_allow_scripts), not just this one; the CLI is safe (itJSON.parses flag values). Fixing the wrapper is broader than this change.replicateOperationis a stub that returns no peers, so the multi-node assertion belongs in harper-pro (alongside the existing deploy-tracking peer test). Covered here by unit tests of the detection logic.Review
Cross-model review (thorough): Harper-domain (
deep-review) + Gemini (agy). The Codex CLI was unavailable. Domain-pass findings (thereason/errorshape mismatch and the SSE-vs-non-SSEfailed_peersasymmetry) and the Gemini finding (afinish()write error could mask the informative deploy error) are all fixed in this PR. Remaining open considerations are the items above.🤖 Generated with Claude Code (Opus 4.8)