feat: dispatch-modes compares planner dispatch with a dispatcher thread (#44)

[lukemaj]

What: agent-observer dispatch-modes compares planner dispatch with a dispatcher thread per job: jobs, succeeded, success rate, median time to first worker and to an approving review, and tokens and cost per job, each with its sample size.
Why: The comparison was to come from the runner ledger that retires with toolboxmd/chromeria#92; T3 thread trees plus the Observer ledger answer it without that ledger.
So what: Rebased onto main after #46 and re-bumped to 0.11.0, with the review finding fixed. It needs an independent re-review of head dce7624; then the planner merges it.

Closes #44

How it works

• A job is a planner thread plus the PR its direct children link, or the Issue when they link no PR. --job accepts either. A job's threads are the planner's direct children whose subtree links it.
• The mode comes from the child names: a dispatcher- child makes it a dispatcher job; worker-, reviewer-, retry- and escalation- children make it a planner job. Older children without a role name leave the mode unknown.
• Times start at the planner turn that created the job's first child. The approving review is the first review/independent success status or GitHub approving review on any PR commit (gh). A job succeeds when that approval exists or the PR merged. If GitHub is unreachable, success is unknown.
• Tokens and cost cover every thread in the job subtrees plus the planner turns that coordinated it: turns started by a child's report, and turns in which the planner created or messaged a child. A thread or planner turn that serves several jobs is split evenly between them.
• If any job thread, including the planner, has no ledger session (a Grok child, or a thread not yet synced), that job's tokens and cost are unknown, and the job leaves the mode's mean and sample size (unmapped_threads in --json). This fixes the review finding on 7758821.
• T3 state.sqlite is read at report time with mode=ro. Only ids and timestamps are kept; the reporter id is parsed from the first line of a report message. No message text is stored or printed. Usage comes from the ledger, which is opened read-only.

Proof

• On head dce7624 (rebased on 86a503a): python3 -m unittest discover -s tests gives 495 tests OK, including 20 in tests/test_dispatch.py. The new unmapped-thread tests failed before the fix (AssertionError: 4100 is not None, 2500 is not None) and pass after it. The Grok case goes through the real t3._import_threads, which maps no Grok cursor. The fixture tree covers both modes plus an unknown-mode job. The expected figures were worked out by hand: shared spawn turn split, report-started turns, nested dispatcher worker, unknown tokens printed as unknown (n=0), GitHub offline giving unknown success, --job by Issue, window and planner filters, and the CLI end to end.
• python3 -m compileall -q agent_observer tests scripts: OK. bin/agent-observer --version: 0.11.0.
• versionctl release-check: release candidate OK, v0.11.0 at dce7624.
• Live report over the #96 jobs (feat: report deep Chromeria spawn chains and cross-thread messages (#42) #46 planner mode, feat: dispatch-modes compares planner dispatch with a dispatcher thread (#44) #47 dispatcher mode). Re-run on head dce7624 (0.11.0) at 2026-10-01T15:13:43Z against the same scratch ledger. feat: dispatch-modes compares planner dispatch with a dispatcher thread (#44) #47's tokens and cost now print unknown because its reviewer's Codex session started during the scratch sync and is missing from the ledger; the first run on 7758821 undercounted them: Compare planner dispatch with a dispatcher thread on Chromeria jobs #44

Limits

• A planner-mode reviewer that never links the PR is not counted as a job thread. Planner turns that touch no linked child count toward no job.
• Even splitting of shared threads and turns is an estimate. The output shows how many threads and planner turns were shared.
• Tokens add up each harness's own total across harnesses. Cost uses the existing list-price schedule, so it is not subscription spend.
• feat: report deep Chromeria spawn chains and cross-thread messages (#42) #46 (Report deep spawn chains and cross-thread messages in Chromeria #42) merged first. This branch is rebased on it: cli.py, README and SKILL.md keep both commands, and the version moves from 0.10.0 to 0.11.0. adapters/t3.py is untouched.

Elon record

• Requirements and who asked: the user, via Compare planner dispatch with a dispatcher thread on Chromeria jobs #44 and Routing rules for every coordinating thread, and one live task per dispatch mode chromeria#96: one command comparing the modes per job with sample sizes, unknown for missing data, and planner coordination turns included in cost.
• Deleted: no ledger schema change or new sync step. T3 is read at report time, so adapters/t3.py (also changed by Report deep spawn chains and cross-thread messages in Chromeria #42) stays untouched. compare_dispatch.py's dependence on the runner ledger is gone.
• Bottleneck: without per-job evidence the default dispatch mode is chosen by opinion.
• Checked myself: I read the live T3 state read-only. Report-back messages read [Subagent … (thread <id>) finished a turn], children are named sub.<parent>.<role>-<id>, and PR snapshots carry no review time, which is why approval comes from GitHub. I ran the command against the #96 planner thread.

🤖 Generated with Claude Code

[lukemaj]

Independent review findings (7758821)

• Blocking: usage can be reported as complete while omitting a job thread. In dispatch.py lines 329–336, job totals include only session keys returned by thread_sessions (lines 208–232). The T3 adapter maps only claudeAgent, codex, and opencode cursors (t3.py lines 356–366); a Grok child therefore contributes no responses. If another supported thread has usage, the report emits that partial token/cost total as known, contrary to the all-job-threads and unknown-on-missing-data requirements.
• Required live evidence is unverified and remains pending. Issue Compare planner dispatch with a dispatcher thread on Chromeria jobs #44 requires a live report for the Chromeria #96 jobs recorded on that Issue. The PR description says it was posted, but Issue #44 currently has no comments, and the local read-only T3 state has no #96 PR or Issue link. The scratch ledger may still be syncing, as reported; I could not verify the report in the available records.

Cleared: job/mode derivation from the thread tree, requested summary metrics with per-metric sample sizes, unknown formatting for empty/unpriced aggregate values, fixtures covering planner and dispatcher modes, and the exact-SHA proof CI check. The complete repository tree at this head has no AGENTS.md, so there was no repository-specific file to read.

[lukemaj]

Review update: live evidence verified

I verified the current Issue #44 report. It records both Chromeria #96 jobs, the exact reviewed SHA, the command and output, 81/86 responses with no unknown-token or unpriced responses in that snapshot, and that it used the freshly synced /tmp/ao44/observer.db; it explicitly says the canonical ~/.local/state/agent-observer/observer.db was untouched. This clears my earlier pending-evidence finding. The report is a valid required snapshot, with its stated n=1 per mode and pre-review timing limitations.

The accounting finding remains blocking: the changed totals path is dispatch.compare -> thread_sessions (agent_observer/dispatch.py:208-232) -> subtree aggregation (:329-336). It relies on t3_threads session mappings, but the T3 adapter only maps claudeAgent, codex, and opencode cursors (agent_observer/adapters/t3.py:356-366). For a job containing a Grok child plus supported threads, that child's responses are absent from the returned rows; because no missing-thread marker is propagated, token and cost totals are emitted as known partial totals. Under #44's requirement to include every job thread and show missing data as unknown, the affected per-job token/cost values must become unknown when any job thread lacks usage mapping, and those unknown jobs must not contribute to known-value means/sample sizes. Do not treat the live #96 snapshot as proof of Grok coverage; its report does not show per-thread harness coverage.

[lukemaj]

Independent review findings for dce7624550af30463e1db073ae6167b4c0b69307

• Fixed prior accounting finding: dispatch.compare now detects child-subtree threads without ledger sessions (agent_observer/dispatch.py:329-342), marks token and cost totals unknown (:409-418), and _mean excludes unknown values and their jobs from each sample size (:435-438). The new fixtures exercise an unmapped Grok child and an unmapped planner; they assert unknown tokens/cost and zero known-value sample size when the only job in that mode is unmapped. No remaining supported code finding from the prior review.
• Required live proof is stale for this candidate. The Compare planner dispatch with a dispatcher thread on Chromeria jobs #44 report exists and is useful evidence, but it ran on 7758821, before this accounting fix. The recorded output does not include the new unmapped_threads field or per-thread mapping evidence. The PR description says those jobs were fully mapped, but that claim is not demonstrated by the report. Since the fix changes whether #96 token and cost figures should be known, please refresh the report using this candidate (or provide equivalent exact-candidate evidence that all job threads, including the planner, are mapped and the totals remain unchanged) before final approval.

Cleared: #44 report exists; thread-tree job/mode derivation; all requested metrics and sample sizes; unknown formatting; planner coordination usage; fixtures for both modes plus the unmapped-session cases; version 0.11.0. Current proof CI is green on this SHA. The PR base and commit parent are both current main at 86a503a04e85f2ac2e79ad9621e0b15e928db15b, the #46 merge.'

[lukemaj]

Independent review update: exact-head live report verified

The exact-head limitation is cleared by the refreshed Issue #44 report. It records head dce7624550af30463e1db073ae6167b4c0b69307 and version 0.11.0. For #47 it reports unmapped_threads: 1, tokens unknown (n=0), and cost unknown (n=0) because the reviewer thread was recorded in T3 after the scratch-ledger sync had already imported Codex usage. #46 has unmapped_threads: 0 and known token/cost values. This directly verifies the prior accounting fix on the reviewed candidate and confirms the unmapped job is excluded from those means and sample sizes.

The #47 usage remains incomplete until the scratch ledger is synced again and the command rerun, as the report says. That is an honest unknown under #44's acceptance criterion, rather than a blocker: the report includes the live #96 jobs and does not present an undercount as a known total.

Re-review verdict: no blocking findings remain. The code and fixtures cover the unmapped child and planner cases, both modes, required metrics and sample sizes, unknown propagation, and planner coordination usage. Base main and this commit's parent remain 86a503a04e85f2ac2e79ad9621e0b15e928db15b (the #46 merge); proof CI is green on this exact head.'