Repository navigation
Cost Codex agents from their rollouts; price GPT Sol models - #45
Merged
Merged
Conversation
…ices priceFor knew only the Claude families, so any Codex model id fell through to the Sonnet row ($3 in / $0.30 cached / $15 out per million tokens). That puts GPT-5.6 Sol at under half its list price and GPT-6.1 Sol's cached input at three times its own, for any cost estimated from a Codex session's token counts. Rows from OpenAI's API pricing page (https://developers.openai.com/api/docs/pricing), standard tier, prompts up to 272K input tokens: gpt-5.6-sol $4.00 in / $0.40 cached / $20.00 out gpt-6.1-sol $2.00 in / $0.10 cached / $10.00 out Codex reports no separate cache-write charge, so that rate is 0. The long-context tier (over 272K input tokens) costs more and is not modelled; these are fallback estimates, like the rest of the table. Matched on the exact id, so no other model's price moves. Red before the rows existed (both ids came back as Sonnet).
🚫 This PR is missing its before/after evidenceEvery pull request here has to show its work. Screenshots or a short screen recording, before the change and after it.
How to fix it: edit the description, keep the A bug fix with no visible surface still needs it: show the failing behaviour, then the same steps passing. A terminal recording is fine. Genuinely nothing to show — a CI tweak, a typo, a dependency bump? A maintainer can apply the |
A Codex agent never reached the cost ledger and read as $0.00 / 0 tok
in the fleet snapshot, however much it ran. The ledger takes a row only
for a sample with a session id, and the usage provider had three
sources: Claude Code's live OTel, Grok's usage.json, and Claude
transcripts. The telemetry env is injected for Claude Code alone and
Codex writes no Claude transcript, so every source came back empty.
Codex does record the numbers. After every model call it appends a
`token_count` event to the session's rollout under its CODEX_HOME.
That rollout is named with the session id the agent's hooks report.
- codexActivity.ts: rolloutFileFor(home, sessionId) finds the rollout,
using the same bounded walk as the activity readers, with live
sessions first and then archived ones.
- codexUsage.ts: CodexUsageReader adds up a rollout's totals. The
running totals are per Codex PROCESS, not per session: a resumed
session starts them again from zero in the same rollout, and a status
update can repeat the previous totals unchanged. So the session's
usage is every rise in the totals, and a restart's new totals are all
new. A restart shows as any counter dropping, or as an event whose
per-call usage equals its totals: the first call of a fresh counter,
which can be larger than the previous process's whole run, so no
counter drops. Neither "the last totals" nor "the sum of every
per-call figure" is right; on one real rollout they were 77% under
and 4% over. input_tokens includes cached input, so uncached input is
the difference. Each rise is priced at the model of the turn it was
billed on, so a mid-session /model switch does not reprice the
history (which the lifetime fold would then read as a counter reset
and count twice). Each file is read once and then only from where the
last read stopped: one real rollout is 136 MB, its first read took
0.35 s, and later beats read only the appended lines.
- telemetry.ts: codexFallback, between the Grok and transcript
fallbacks. Cost is the list-price estimate, because Codex reports
tokens, not dollars. The sample is dated by the rollout's last change,
the same reading the Codex activity readers take, and carries the
real session id so the ledger takes the row.
- usage.ts / index.ts:
- usesCumulativeGate() names the providers whose samples always carry
a session id (Grok, Codex). The ledger write gates both through the
existing cumulative-sample gate, so an idle Codex agent does not
append the same row every beat.
- The breaker is fed what those samples added since this app run first
saw them (RunBaseline); the ledger row stays cumulative. A rollout
holds a session's whole history across every resume, while live
telemetry counts from app start, so a per-agent token cap means "this
run" for a Claude agent. Fed the history, a month-old session with
15M work tokens would trip a 2.5M cap on the first beat, be
constrained on the second, and again after every app restart.
- The fleet snapshot shows these agents the same sample, so their
tokens, session cost and last-active time are no longer zero or
unknown. Agents with live telemetry are unchanged.
- A Codex agent's home is resolved through CodexHomes, as the
activity readers already do.
Behaviour change: a Codex worker started by a spawn request now has a
usage sample, so it counts against the default worker token cap and is
stopped when over it, as Claude workers already are. Before, it had no
sample and was never measured.
Tests use a fixture rollout and cover: costing; the session id; a
restart adding rather than replacing, including one with no counter
drop and one where only output drops; a repeated event counted once;
pricing across a model switch, and the lifetime fold over it; totals
3 MB from the end; archived sessions; another session in the same home
ignored; no home or no totals meaning no data; the idle gate; the
incremental read; the run baseline; and the real runBreakerBeat and
writeFleetSnapshot bodies run against stubs. Each of these was red
before its change, and each of these mutations turns at least one test
red: not adding a restart, missing the first-call marker, checking only
input for a drop, dropping the repeat guard, pricing at the wrong
model, summing per event, re-reading from the top, counting cached
input as input, leaving the fallback out of the chain, leaving Codex
ungated, feeding the breaker the cumulative sample, and leaving the
snapshot on live telemetry only.
Checked against two real rollouts (40 MB and 136 MB). Over a 14.7-hour
window the reader's token totals match a hand sum of that window's
per-call usage exactly. Over the whole sessions they match an
independent rise-sum exactly; on the 40 MB rollout the first-call
marker finds one more restart than an input drop alone does.
aaroncoville
force-pushed
the
fix/codex-ledger
branch
from
October 9, 2026 18:29
2941d9e to
4c74dcb
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
usd: 0/ 0 tokens in the fleet view. Adds a Codex rollout reader (codexFallback) in the usage chain after Grok and before the Claude transcript fallback.total_token_usageis cumulative per process and starts again on resume, so the reader sums each rise and treats a restart's totals as all new. A restart is any counter dropping, or an event whose per-calllast_token_usageequals its totals (the first call of a fresh counter, which can be bigger than the previous process's whole run). A repeated event is counted once. The reader reads a rollout once, then only the appended bytes./modelswitch does not reprice the history (the lifetime fold would otherwise read the lower usd as a reset and count the history twice).RunBaseline); the ledger row stays cumulative. Without this, a month-old Codex session (15M work tokens) trips a 2.5M per-agent cap on the first beat, gets constrained on the second, and again after every app restart.tokens,sessionUsdand last-active are real. Agents with live telemetry are unchanged.Behaviour change (intended)
defaultWorkerTokenCapand is stopped when over it, exactly as Claude workers already are. Before this it had no sample and was never measured. With cached input counted (as for Claude), a Codex worker with ~200k context per call crosses a 2M cap around its tenth call.Test plan
test/codex-usage-ledger.test.cjs(20),test/pricing-codex.test.cjs(1), plus one new fleet-snapshot test intest/codex-activity.test.cjs; all red first.runBreakerBeatandwriteFleetSnapshotbodies fromindex.tsrun against stubs, with the real breaker, gate and baseline: a 15M-token history does not steer under a 2.5M cap, later work over it does, and the ledger rows stay cumulative.