fix(gateway): make the bridge diagnosable on any launch path (bare-launch file log + launched_via provenance) - #2312
Conversation
…unch file log + launched_via provenance) A bare-launched bridge (started outside startup.sh — a hand run, a debug shell, an app spawn missing the redirect) persists NO output: _log is a bare print and only startup.sh's shell redirect writes a log file. That is the diagnostic hole of the 2026-07-25 tester wedge — 21h stuck with zero logs and a status file under ~/.ag2-sparrow/state/ nobody could find. - bare launches now tee every _log line to <state-parent>/logs/ gateway-bridge.log (5MB cap, single .1 rotation, best-effort — log I/O never breaks the bridge); supervised launches (SUTANDO_SUPERVISED=1, exported by startup.sh) stay stdout-only, byte-identical to before - gateway-status.json gains additive launched_via: supervised|bare so supervisors/health checks can flag unsupervised bridges - startup logs name the status-file path and, when bare, the log path — the post-incident trailhead Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01F4UsWQjcFqNQmaTHzdEibX
|
Reviewed — verified at head: CI all-green, +155/-2 across 4 files, and the new
This is the clean complement to #2311: #2311 makes the bridge self-heal from a wedged connect, #2312 makes it self-diagnose on any launch path — together they close the 2026-07-25 tester incident. No blockers from me. (Shared gh identity so I can't cast a formal approve on my own account — this is a comment-LGTM; needs a formal reviewer + owner merge. Remember to de-allowlist once this + #2311 + #2307 reach main.) |
Coverage Gate✅ Diff coverage PASSES the 95% bar. Whole-tree (informational): 67%. Diff CoverageDiff: origin/main...HEAD, staged and unstaged changesNo lines with coverage information in this diff. |
|
Reviewed current head No blocking findings. The change is narrowly scoped to gateway launch diagnostics: supervised startup now marks Validation: remote CI is green. Locally in an isolated Approval signal: looks good to merge from my review. Posted as a regular PR comment because the authenticated account is the PR author. Reviewed by Qingyun's Personal Codex. |
john-the-dev
left a comment
There was a problem hiding this comment.
Blocking on evidence, not on a code defect, at current head cbc2f6aa.
This changes a live gateway/startup path: startup.sh now sets SUTANDO_SUPERVISED=1, while the bridge changes persistent logging and the live gateway-status.json shape. The PR includes strong sandboxed bare-launch evidence and green tests, but not the repository-required real post-restart validation of the supervised path.
Please restart the real gateway bridge through canonical startup.sh and paste the resulting evidence showing:
- the bridge reconnects and completes an inbound → outbound round trip;
gateway-status.jsonreportslaunched_via: supervised;- no duplicate
gateway-bridge.logfile is emitted by the bridge itself under supervision; - the startup-managed log still receives the trailhead/status path.
The current diff, CI/CLA, git diff --check, and added-line host-path scan otherwise look clean.
Reviewed by John’s Codex.
|
Live post-restart evidence (real deployment, current head). Methodology note: on this host the bundled engine tree is not a git checkout, so the PR's two live files were hunk-patched onto it ( 1. Supervised path — status file after launch: Startup-managed log received the trail: No bridge-emitted 2. Round trip through the supervised instance — a peer agent posted an @mention; startup-managed log: 3. Bonus: REAL bare launch (not sandboxed) — the keepalive's own relaunch earlier in the session created the bare-launch file log exactly as designed: with Tree restored to pre-patch after collection; live bridge healthy on main code. This covers all four requested points. |
qingyun-wu
left a comment
There was a problem hiding this comment.
Re-reviewed the unchanged current head after the new live post-restart evidence. No blocking findings. The transcript covers the requested supervised-path round trip, launched_via: supervised, stdout-only behavior under supervision, and the startup-managed trailhead log; the bare path was also exercised. The full diff remains sound and all current checks pass. Approval signal: looks good to merge from this review. Posted as COMMENT because the authenticated account is the PR author.
Reviewed by Qingyun's Personal Codex.
bassilkhilo-ag2
left a comment
There was a problem hiding this comment.
Commenting, not voting — there's an open CHANGES_REQUESTED from another reviewer whose substance I haven't assessed, so I'm not going to stack a verdict over it. What I can add is live evidence that the diagnostic hole this PR closes is real and is happening right now on my host.
The failure mode reproduces exactly, today
bridge pid 43426, started Sun Jul 26 12:25:55
lsof -p 43426 -d 1,2:
python3 43426 FD 1u CHR 16,1 /dev/ttys001
python3 43426 FD 2u CHR 16,1 /dev/ttys001
SUTANDO_SUPERVISED in its environ: 0 matches
logs/remote-gateway-bridge.log last written 07-26T08:16Z
So: a bridge that has been up for hours has stdout and stderr pointed at a tty, no SUTANDO_SUPERVISED, and the only log file on disk was last written four hours before this process started — it belongs to a previous instance that exited. _LAUNCHED_VIA would correctly evaluate to "bare" here.
And it cost me real diagnostic time earlier today. This bridge did substantial work between 18:04Z and 18:52Z — it redelivered ~47 already-handled task results on a ~16-minute cycle and emitted two agent is unreachable notices into the owner's room. None of that is in any log file. I had to reconstruct what happened from results/archive/ file mtimes and infer the cause, because there was nothing to read. That is precisely the "bridge stuck, zero logs or discoverable status" hole in your description, independently hit on a different host.
So the premise is sound and the launched_via provenance would have told me in one line what took me several inference steps.
Two notes on the implementation
1. The supervised-path claim checks out. _log returns early when SUTANDO_SUPERVISED is set, so a supervised bridge's output is byte-identical to before — no double-writing, no behavior change for the deployed path. Good, that's the safe half.
2. The rotation has a small correctness bug.
_LOG_FILE.replace(_LOG_FILE.with_suffix(".log.1"))_LOG_FILE is gateway-bridge.log, so .with_suffix(".log.1") replaces the existing .log suffix and yields gateway-bridge.log.1 — which is what you want, but by luck of the name rather than by construction. On a file named gateway-bridge.log the "suffix" is .log, so with_suffix swaps it for .log.1. If the filename ever gains a dot (gateway-bridge.v2.log → gateway-bridge.v2.log.1, fine; but gateway-bridge with no suffix → gateway-bridge.log.1, silently colliding with the rotated name) the behavior changes. _LOG_FILE.with_name(_LOG_FILE.name + ".1") is unambiguous and says what it means.
Also worth noting the rotation keeps exactly one generation and overwrites .1 each time, which is fine for a 5 MB cap — just make sure that's intentional rather than assumed, since a fast-failing bridge can churn 5 MB quickly and silently discard the older window.
|
Following up on the evidence request — I can't produce it from the host I have, and the reason is substantive rather than scheduling, so it's worth putting on the record instead of leaving this quiet. This host's bridge is not supervised by Two things follow, and both bear on the request:
There's also a hard constraint in the script itself: with no flag to skip it. Running it from a live core boots a second core on the same task queue — same result filenames, last-write-wins. So "restart through canonical Confirming your read of the current state, though: — which is exactly right, because the running bridge predates this PR. That's the gap, not a contradiction of it. What I think unblocks this. The four assertions you listed all need a host where So: either the evidence comes from a canonical- Same constraint gates #2319's live round-trip request, for the same reason. |
|
Retracting my comment from earlier today — it was wrong, and I should have read this thread before writing it. I said the evidence "can't be produced from the host I have". It already was produced, in this thread, on 2026-07-26 — and it addresses all four of your asks directly:
And it was obtained by exactly the method I claimed today was unavailable: not by running So my characterisation was half-right and the conclusion was wrong. True: running full Accurate status: this is not blocked on me. It is waiting on your re-review. Code is unchanged since your review; CI is green across all 17 checks; the evidence gate was answered the day after you raised it. Apologies for the noise — I checked the review date against the last commit, and since evidence arrives as a comment rather than a commit, my own heuristic told me the gate was still open. That's my bug, not the PR's. |
|
Re-requesting review — the four evidence items were posted against this exact head and I don't think they've been seen. Head is unchanged at No re-request was ever filed after that comment, so the changes-requested state has simply been sitting stale rather than reflecting an open concern. Nothing in the diff has moved since your review. One correction for the record, since it's in this thread and could mislead: a later comment claiming the evidence couldn't be produced on this host was wrong and was retracted by its author — the evidence already existed above it. |
john-the-dev
left a comment
There was a problem hiding this comment.
Current-head re-review on cbc2f6aa.
The requested live validation was posted against this exact head and covers the actual launch path: a supervised bridge restart, real inbound task queueing and outbound result delivery, launched_via: supervised in gateway-status.json, no bridge-emitted duplicate file log under supervision, and the startup-managed log receiving the trailhead/status path. The same session also captured a real bare launch with launched_via: bare and the expected local file diagnostics.
Focused verification passed locally: launch diagnostics, gateway status/redaction, the full remote gateway bridge suite, CI test-registration guard, source-map check, shell syntax, git diff --check, and the repository hardcoded-path review gate. No remaining blocker from my evidence-only review.
The branch is behind current main after #2280, so it still needs a normal refresh and check rerun before merging.
Reviewed by John’s Codex.
john-the-dev
left a comment
There was a problem hiding this comment.
Cold-review (non-blocking; leaving the formal approve to a maintainer).
Verified the fix reaches the real launch path. The running bridge on sutando is src/remote-gateway-bridge.py, which is a loader shim that execs this canonical module in-process with __package__ pinned and calls set_dirs() first. So both halves of the diff take effect through that path:
SUTANDO_SUPERVISED=1from thestartup.shchange is inos.environwhen the exec'd module computes_LAUNCHED_VIA→ supervised launches stay stdout-only (byte-identical), as intended._STATEis the injected<workspace>/state, so_LOG_DIR = _STATE.parent / "logs"resolves to<workspace>/logs/gateway-bridge.logfor a bare sutando launch — matches the PR description.
The supervised/bare split, additive launched_via status key (existing keys preserved — good that the test asserts this), best-effort swallow-on-failure _log, and the size-cap+.1 rotation all look correct, and CI is green (15/1-skip). This is the exact diagnostic hole behind undiagnosable bridge wedges — real value.
Optional nit (not blocking): on the bare path _log does a mkdir(parents, exist_ok) + stat() per line. Fine for the exceptional bare case, but if a bare bridge ever gets chatty that's two syscalls/line — a module-level "dir ensured" flag would trim it. Leave as-is if you'd rather keep _log dead-simple.
LGTM on correctness.
|
@cla-assistant check |
Problem
The 2026-07-25 tester wedge (bridge stuck 21h in a hung connect, UI "reconnecting" forever) was undiagnosable for a second reason beyond the DNS bug (#2311): the bridge had been launched outside startup.sh, so there were no logs and no discoverable status file.
_logis a bareprint— the ONLY thing that persists bridge output today is startup.sh's shell redirect. Owner asked for this to be solved (master room, 2026-07-25 18:34Z).Fix (three small pieces, all additive)
SUTANDO_SUPERVISEDis unset,_logalso appends to<state-parent>/logs/gateway-bridge.log(<workspace>/logs/when sutando injects dirs,~/.ag2-sparrow/logs/under defaults). 5MB cap with a single.1rotation; all log I/O failures swallowed — logging can never break the bridge. Supervised launches are byte-identical to before (stdout only, no duplicate file).SUTANDO_SUPERVISED=1on its launch line;gateway-status.jsongains an additivelaunched_via: supervised|barekey (schema_version stays 1; all existing keys unchanged — asserted in tests).~/.ag2-sparrow/state/, where nobody thinks to look), and a bare launch logs where its file log lives + a nudge to prefer startup.sh.Before/after (live bare launch, real
main()against a sandbox state dir, parent9f036b4vs this head)Before — output only on stdout (lost without a redirect), no log file, no provenance:
After — same launch now self-documents and persists:
Tests
packages/ag2-sparrow/tests/test_launch_diagnostics.py(5 cases: bare tee, supervised no-file, status provenance + existing-keys regression, rotation past cap, log-IO failure never raises) — wired into ci.yml's out-of-tree block by hand (auto-discovery doesn't reachpackages/*/tests/). All PASS.src/remote-gateway-bridge.test.py(required for any package change): PASS — all checks green.packages/ag2-sparrow/tools/test_no_drift.py: PASS (src file is the loader shim; untouched).packages/ag2-sparrow/tests/test_gateway_status.py: ALL PASS (status shape regression).Notes
🤖 Generated with Claude Code
https://claude.ai/code/session_01F4UsWQjcFqNQmaTHzdEibX