Skip to content

Two COPY fault-reporting follow-ups from #3110: three sibling sites keep the unclosed start-phase residual, and the PostgreSQL-target timeout sentence can be applied to a store write #3111

Closed

Description

@erikdarlingdata

Two follow-ups surfaced by #3110, which added a StoreCopyPhase axis to the collector's binary COPY into the store. Both were reported as out of scope by that change's author and left untouched, correctly.

1. Three sibling COPY sites carry the identical unclosed residual

#3110 gave DarlingCollectorRunner's COPY a distinguishable start phase, because BeginBinaryImportAsync awaits CopyInResponse under the connection's CommandTimeout — unclosable at the call site — while importer.Timeout bounds only the data phase, and both surface as the same message.

The same shape exists, unaddressed, at:

  • RdsPlanIngestor.cs:178
  • RdsCpuIngestor.cs:259
  • RdsDeadlockIngestor.cs:186

Mechanical to extend now that CollectorFaultCopyPhase exists and is pinned. Worth doing together rather than one at a time, so "a fresh phase" has one body across every COPY the service performs.

2. A PostgreSQL-target timeout message may be applied to a STORE-write timeout — narrative verified, database-naming unverified

DarlingWorker.PostgresTimeoutExplanation produces the operator-facing sentence for a PostgreSQL command timeout. Its own comment states the gate plainly:

"This arm fires for EVERY PostgreSQL collector classified as CommandTimeout"

What I verified. The client-side branch of that message is target-read-specific in its remediation:

"…Npgsql cancelled the read mid-stream… The statement was still running when it was cut off, so the work asked for does not fit the deadline; raising the deadline is the wrong half of that and shrinking the work is the right one."

A store-write timeout is also an NpgsqlException wrapping a TimeoutException with no SQLSTATE, so it classifies as the same CommandTimeout and reaches the same arm — and would receive advice to shrink a read that was not the failing operation. "Shrink the work" is not the remedy for a COPY write blocked by store-side contention.

What I could NOT verify, and it cuts against the stronger version of this report. The call site deliberately takes the database off the exception rather than from the runtime, with the reasoning stated inline:

"pg_index_bloat fans out per database and opens its own connection for each, so the runtime's initial-probe database is not the one that ran out of time. Naming the wrong database confidently is worse than naming none."

So the author already guarded the wrong-database failure mode. Whether a store-write exception carries a database that resolves to the store (accurate, if unhelpfully terse) or to an unrelated target (the reported defect) is not something I established. A report that this "names an unrelated database" is therefore unconfirmed, and the guard above is evidence against it.

The narrative mis-attribution stands regardless of how the database resolves, and it is the part that misleads: a confident, specific, wrong remediation instruction is worse than a vague one, because an operator acts on it.

Suggested direction

Now that #3110 exists, the phase axis is the discriminator this arm lacked. A fault carrying StoreCopyPhase.Start or .Data is a store-write fault by construction, so the arm can route it to a store-side sentence instead of the target-read one. That is a smaller change than it would have been before #3110 and is the natural second consumer of that work — the first being #3099.

Acceptance criterion: a store-write timeout on a server whose target is PostgreSQL produces a message describing a store write, and does not advise shrinking a target read.

Before implementing, settle the database question by observation rather than by reading — the unverified half above is exactly the kind of claim that should be measured on a real fault rather than argued from the call site.

Not included: an analyzer-warning backlog

The same review flagged 190 repo-wide xUnit2013 warnings, with PostgresFaultOutcomeTests at 24 and ControlPlaneReloadDurabilityTests carrying both forms. That is pre-existing house drift rather than anything either change introduced, and it does not belong to this issue.

Activity

  1. added a commit that references this issue on Sep 7, 2026
    376b904
  2. erikdarlingdata commented on Sep 7, 2026

    @erikdarlingdata
    OwnerAuthor

    Claude posting for Erik Darling

    Fixed by #3117, merged to dev. Closed by hand — closing keywords do not fire here, since merges land on dev and the default branch is main.

    Both items landed, and the two prose-only notes this issue was carrying are discharged: IsSafeToReattempt's transport leg now has a test that exercises it on the real path (the existing one raised an InvalidOperationException, a start-phase fault that is not a transport fault, so it covered the phase leg only), and the finally justification now states the actual reason — the fault is returned rather than thrown, so nothing is in flight for a throwing dispose to replace.

    One thing worth recording, because it came out of review rather than the brief. The pin guarding "a stamped fault is provably a store write" covered three of the four routes by which that can become false. The uncovered one was CollectorFaultCopyPhase.Stamp( being called from a non-COPY site — nothing constrained the set of stamp call sites to be the set of COPY sites, so a stamped target read would have been excluded from the PostgreSQL-target arm and labelled store-write, which is the confident-wrong direction IsProvenStoreWrite's own remarks identify as the expensive one.

    Closed with set equality, and the lane then found a further route past its own new assertion: the detector matches a qualified call, so a using static on the axis would make Stamp( legal unqualified and blind the scan. Both are now closed and mutation-verified red in three directions.

  3. added 2 commits that reference this issue on Sep 23, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions