fix(sync): say which check returned 13 - #911
Merged
Merged
Conversation
Exit 13 is the sqlite sync driver's "a check failed", and it was the only thing
a caller received. 113 sites across 13 functions returned it with nothing on
stderr, so
storage sync reprocess failed for team 'agmsg' (exit 13):
driver returned a non-zero exit without diagnostics
named no check and no line. Working out which of the 113 had fired meant reading
the file against the failing machine's state.
Nothing above the driver needed changing. remote-sync.mjs already prefers the
driver's stderr and falls back to that sentence only when it is empty, and it
forwards stderr to the terminal as it arrives. So the message quoted above was
never "we cannot tell you" -- it was an accurate report that the driver had said
nothing. It now means only that.
The location is derived, not written. A hand-authored reason at each site is one
more sentence that can drift from the condition beside it, which is the drift
#781 removed by folding an authority-file fault and its wording into a single
function. FUNCNAME and BASH_LINENO cannot disagree with where the return is,
because the shell computes them from it.
One transform, two shapes, no exceptions:
76 `|| return 13` -> `|| { _sqlite_sync_why; return 13; }`
37 already inside a brace group, an if body, or a case body -> prefixed
---
113 sites, and 113 `return 13` still in the file afterwards
The brace on the first shape is load-bearing. `cmd || _sqlite_sync_why; return
13` ends the `||` list at the semicolon and returns unconditionally -- every
check in the file would stop guarding anything. That is also why the helper only
reports: a function cannot return on behalf of its caller, so each site keeps
its own `return`.
Behaviour is unchanged. The exit code is still 13 on every path; what is added
is a line on stderr.
Measured against main, same call, same argument:
main (silent) exit 13
this branch agmsg: sqlite-sync: storage_sync_reprocess failed
at sqlite-sync.sh:1151 exit 13
1151 is the `_sqlite_sync_valid_binding` gate, which is the check that call
actually fails.
Refs #908
A diagnostic that has never been emitted is indistinguishable from one that does not work. 113 sites were added; this drives one of them for real and asserts the string arrives. The test already existed and already went red on this change: it asserted $output was empty for an absent binding, and bats merges stderr into $output, so it had been pinning 'says nothing about why' -- the exact defect being fixed. Rather than loosen it, it now runs with --separate-stderr and asserts BOTH halves: stdout stays empty, because 'fail closed' is about not emitting a status record, AND stderr names the function and line. Neither can be lost by the other changing. --separate-stderr is new to this suite; bats has had it since 1.5.0 and CI installs bats unpinned via npm.
This was referenced Aug 21, 2026
joelmitz
added a commit
to joelmitz/agmsg
that referenced
this pull request
Aug 23, 2026
`_sqlite_sync_apply_fail` closed fd 3 with `exec 3<&- 2>/dev/null || true`. `exec` with no command word runs nothing: it applies its redirections to the shell itself, and they outlive the statement. Every later write to stderr in that shell therefore went to /dev/null. The first thing written after it is `_sqlite_sync_why` -- the line fujibee#911 added so a 13 says which check produced it. On a malformed pull page the operator saw agmsg: sqlite-sync: jq: error (at <stdin>:1): record 1: not one JSON value on its line agmsg: sqlite-sync: record 1 ended mid-frame at field type and nothing else. The two messages the helper emits before the close still arrived, so the output looked almost right; only the line naming the check was missing. With the brace group the same input also prints agmsg: sqlite-sync: storage_sync_apply_pull failed at sqlite-sync.sh:1062 This is the single cleanup path for every failure after the stream opens, so it covered every apply failure on the pull path, not one unlucky site. The regression test drives a malformed page through `storage_sync_apply_pull` and asserts both the 13 and the presence of the "failed at" line. It fails on the previous line and passes on this one. Found while mirroring this function on the read-state path, and only because a reviewer insisted stderr be part of an equivalence check.
fujibee
added a commit
that referenced
this pull request
Aug 25, 2026
…derr `_sqlite_sync_apply_fail` closed fd 3 as `exec 3<&- 2>/dev/null`. With no command word, bash does not scope that redirection to anything — it applies it to the shell, permanently. Every `>&2` after the first apply failure went to /dev/null. It sits on the failure path, so what it silences is the output a failing apply is about to produce. The first thing lost is the message #911 added this morning, naming which check returned 13 — the diagnostic that made today's work possible, disabled a few hundred lines away in the same file. `{ exec 3<&-; } 2>/dev/null` scopes it to the block. Two tests, because either alone passes while the other's failure ships: one lifts the driver's own function body out of the file by line range and calls it (a copy of the shape would keep passing while the driver regressed), one greps the file for a bare `exec` carrying a redirection. Both go red on the old form. Reported by @joelmitz, who also identified #911 as the first casualty.
This was referenced Aug 25, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #835.
The problem
Exit 13 is the sqlite sync driver's "a check failed". It was also the only thing a caller received:
That names no check and no line. There are 113
return 13sites across 13 functions inscripts/drivers/storage/sqlite-sync.sh, so working out which one fired meant reading the file against the failing machine's state.Nothing above the driver needed changing
scripts/internal/remote-sync.mjs:1973already does the right thing:It prefers the driver's stderr and falls back to that sentence only when stderr is empty;
:1936-1942also forwards stderr to the terminal as it arrives. So the message above was never "we cannot tell you" — it was an accurate report that the driver said nothing. After this change it means only that, which is what it was always trying to say.I checked this before designing anything, because a diagnostic written to a channel that gets swallowed is worth nothing.
The shape
The location is derived, not written. A hand-authored reason at each of 113 sites is 113 more sentences that can drift from the condition beside them — the drift #781 removed by folding an authority-file fault and its wording into one function.
FUNCNAMEandBASH_LINENOcannot disagree with where thereturnis, because the shell computes them from it.The transform is mechanical — two shapes, zero exceptions
|| return 13→|| { _sqlite_sync_why; return 13; }ifbody, or acasebody → prefixed in placeDerived, not enumerated, so the claim is checkable in one command each:
Reviewing one instance of A and one of B reviews the whole diff. If any site had a third shape it would appear in that last count as non-zero.
The brace on shape A is load-bearing.
cmd || _sqlite_sync_why; return 13ends the||list at the semicolon and returns unconditionally — every check in the file would stop guarding anything. This is also why the helper only reports: a function cannot return on behalf of its caller, so each site keeps its ownreturn. A naivesed 's/return 13/_sqlite_sync_why/'would have silently done exactly that.Behaviour is unchanged
The exit code is still 13 on every path. What is added is a line on stderr.
Measured against
main, same call, same arguments:mainagmsg: sqlite-sync: storage_sync_reprocess failed at sqlite-sync.sh:11511151is the_sqlite_sync_valid_bindinggate, which is the check that call actually fails. Themainrow is the control — without it, "the diagnostic appears" would not distinguish a new line from one that was always there.A diagnostic that has never fired is indistinguishable from one that does not work
So one test drives a real site and asserts the string arrives.
tests/test_remote_sync.bats— "resync status is read-only and absent bindings fail closed" — already existed and went red on this change: it asserted$outputwas empty, and bats merges stderr into$output, so it had been pinning "says nothing about why", which is the defect being fixed.Rather than loosen it, it now runs with
--separate-stderrand asserts both halves:$output(stdout) stays empty — "fail closed" is about not emitting a status record$stderrnames the function and a line numberNeither can be lost by the other changing.
--separate-stderris new to this suite; bats has had it since 1.5.0 and CI installs bats unpinned via npm.Mutations
|| _sqlite_sync_why; return 13)storage_sync_resync_statusA is the row that matters most: it is the failure mode unique to this transform, and it shows the existing suite catches it rather than only my own grep.
B strips the whole function on purpose. Removing the helper from a single site would have been an inert mutation — the test fails at whichever of that function's 9 gates trips first, so a one-site edit could leave the test green for a reason unrelated to the code.
Ran
Every suite that exercises the changed file, derived by grepping the test tree for
storage_sync_/sqlite-syncrather than picked by hand:Not run locally: the full matrix. CI's job.
Scope, and what this does not do
scripts/drivers/storage/{jsonl,sqlite}.shalso return 13. They are not on the reported path and are not touched here._sqlite_litspawning 32 of the 34 processes per applied message — a throughput problem in the same functions, not a diagnosability one. An earlier revision of this body cited it by mistake; the commit message on0d63275still carries that wrong reference and cannot be rewritten without discarding a review already in flight against this head. This is the correction of record.