feat: models.dev price sync + CodeWhale usage ledger - #25
Open
purrfecto114-lgtm wants to merge 3 commits into
Open
feat: models.dev price sync + CodeWhale usage ledger#25purrfecto114-lgtm wants to merge 3 commits into
purrfecto114-lgtm wants to merge 3 commits into
Conversation
- renderer windows: sandbox: true (preload only uses contextBridge/ipcRenderer, renderer JS has zero node API usage) + deny-by-default CSP meta on pet/panel/archive (connect-src none — all comms go over IPC) - command-safety: fail-closed single read-only command recognizer replacing the old prefix-only check; closes env-exec, git branch/remote mutation, git diff --output/--ext-diff, rg --pre, fd -x/-X, date -s and shell-grammar chaining/substitution/redirect bypasses - pretool-hook: Claude Code PreToolUse gate emitting the current hookSpecificOutput.permissionDecision structure (flat top-level fields are silently ignored by CC >= v1.0.59); no-op output leaves the normal permission flow in charge. stdin is buffer-concatenated once so CJK payloads split across chunk boundaries no longer corrupt the decoded JSON - hookinstall: registers the Bash-matcher PreToolUse gate alongside the state hooks; uninstall/hooksCurrent account for it - server: runtime.json ownership is now first-live-wins with stale-record healing — a probe-gated guard replaces the unconditional 15s re-assert that made two live instances flip-flop the file (each overwrite invalidating the other's token mid-flight); same-port stale tokens are rewritten immediately; guard interval injectable via runtimeGuardMs for tests - tests: command-safety bypass matrix (31 reject / 39 allow cases) + runtime-ownership four-scenario suite (live peer untouched, stale record taken over, same-port stale token healed, stop() never deletes a live peer's record) npm test exit 0.
…aller
Verified against CodeWhale's actual hook contract (docs/HOOKS.md + executor
source, commit 7d942bd) — the half-finished refresh had invented event names
that do not exist upstream:
- hook/codewhale-hook.js: registers the 10 real lifecycle events
(session_start/session_end/message_submit/tool_call_before/tool_call_after/
turn_end/subagent_spawn/subagent_complete/on_error/mode_change); upstream's
turn_start/error DO NOT exist. tool_call_before reads its payload from the
DEEPSEEK_* env vars (there is no CODEWHALE_TOOL_NAME upstream); observer
events merge their stdin JSON. Fails closed to {decision:"ask"} whenever
LLMPET is unreachable/untrusted, because an unanswered CodeWhale hook
defaults to ALLOW on timeout. Observer hooks install with background=true
(state sync never rides the turn's critical path); the readStdin guard
timer is cleared+unref'd so hook processes exit ~30ms after their work
instead of idling 300ms; turn_end usage metadata (turn_id/status/provider)
is forwarded for the ledger. The permission gate auto-allows CodeWhale's
real shell tool name exec_shell (the Claude-era 'Bash' spelling never
arrives on this route and made the auto-allow dead code -> permission
fatigue). mode_change maps to a synthetic ModeChange event + 'attention'
oneshot — the adapter turns Notification into a needs-input card, wrong
for a user-initiated Plan/Work/Operate switch.
- backend/codewhale-provider.js: managed TOML block installer. The permission
gate is a strict gate (continue_on_error = false, timeout 600s) so an
unresponsive pet DENIES instead of CodeWhale's default allow-on-timeout;
observer hooks stay best-effort. User TOML outside BEGIN/END markers is
preserved; atomic 0600 writes.
- backend/codewhale-permission.js: bounded permission holder — 64 pending cap
(overflow -> ask), 8-minute auto-DENY (never ask: the un-answered upstream
timeout ALLOWS; honors the documented autoDenyMs option, invalid values
fall back to the 8-minute window), connection-close cleanup, session-end
sweep.
- server: /codewhale-permission endpoint behind the same loopback/Host/Origin/
per-boot token boundary as /state; agent_id routing tags sessions; turn_end
usage (incl. turn_id/provider/status metadata) is handed to a ledger
callback (wired in the pricing PR).
- main: cw-* permission ids route to the CodeWhale holder from the pet bubble;
pending lists merge on the main pet (no dedicated CodeWhale pet yet);
installer runs only when ~/.codewhale exists or LLMPET_ENABLE_CODEWHALE=1.
Permission cards identify the asking agent (CodeWhale · exec_shell) and
state the auto-deny window; the hint derives from the pending entry's own
createdAt/expiresAt so the first push and every stats-snapshot rebuild of
the same card agree; askPermission reports 'timed out' distinctly from
'not running'.
- adapter: agentOf/AGENT_NAME know 'codewhale' (first-class like dsh);
humanizeTool understands exec_shell/write_file/edit_file/apply_patch/
fim_edit/read_file (field names verified against CodeWhale's permission-rule
docs: exec_shell->command, file tools->path); onPermissionAdded falls back
to the entry's own agent identity when the session is not in the store yet
(a tool_call_before can beat session_start).
- transport: postState reuses the already-read runtime port (one readFileSync
per hook event instead of two).
npm test exit 0 — incl. the e2e suite driving the real HTTP server and both
hook subprocesses (env->/state routing, permission trust boundary, timeout
branch against a never-responding server, stdin->stdout protocol shapes).
Windows CI fix (follow-up to the initial red run): the test-suite HOME
isolation only overrode env.HOME, but Node's os.homedir() on Windows reads
USERPROFILE — so on windows-latest the hook subprocess resolved the REAL
user home, found the runtime.json the test server itself had just written
there, and the "LLMPET unreachable -> ask" case deterministically hung on
the permission bridge until the 10s SIGKILL (exit != 0). macOS/Linux stayed
green because HOME is honored there. Both test files now override
USERPROFILE alongside HOME — in the parent before the first backend require
(RUNTIME_PATH is frozen at module load) and in the unreachable-child env.
The observer-hook timing check also moved to a dual-clock scheme:
spawn->exit (<280ms) is only asserted off-Windows (cold-start noise has no
discriminative power there), while the new ack->exit measure (<150ms from
usage delivery to process exit) strips spawn cost and catches the 300ms
stdin-guard leak on every platform. Verified by simulating Windows homedir
semantics via an os.homedir shim reading USERPROFILE: old tree fails at the
unreachable case exactly as CI did; fixed tree passes both clocks.
Second Windows-only follow-up, found on the re-run: every assertion group
passed on windows-latest but the suite still exited 1 at the final sandbox
cleanup — fs.rmSync hit ENOTEMPTY on `.octopus` because the runtime.json it
had just unlinked was not yet gone from the directory listing (POSIX shows
deletions immediately, Windows does not; the file was written microseconds
earlier by the last runtime-ownership assertion). Node ships the exact
remedy for this race: rmSync's maxRetries/retryDelay options retry
ENOTEMPTY/EBUSY/EPERM with linear backoff. All sandbox cleanups in both
test files now pass maxRetries=10, retryDelay=50.
Third Windows-only follow-up: the re-run still died at sandbox cleanup with
ENOTEMPTY even with maxRetries=10 (all 11 attempts failed across the full
2.78s backoff window — timestamps prove the retries ran). Root cause: the
global logger (backend/log.js) keeps ONE ever-open append stream on
~/.octopus/octopus.log, and server.stop() does not close it. On POSIX an
unlink is just an inode refcount drop, so Linux/macOS never notice; on
Windows the open write handle pins the file's directory entry in a
delete-pending state, so rmdir reports ENOTEMPTY for as long as the handle
lives — no amount of rmSync retries can win. Fix: log.js now exports
shutdown() (ends the stream, resolves on close with a 500ms fallback, never
rejects; a later log() transparently re-opens). Both test suites await
shutdown() before deleting their fake HOME, and still keep maxRetries as a
belt for the residual post-unlink visibility race.
Fourth Windows-only fix, this one a genuine flake that pre-exists upstream
and can hit any CI run: the re-run crashed smoke.js itself with an
uncaught ECONNRESET. Timestamps in the job log show a 4-second stall
between two pure-function assertions (only possible on a contended
runner), putting the last HTTP request 6.4s in the past — past the Node
http server's default keepAliveTimeout of 5s. The server closed the idle
keep-alive socket; Node 19+'s globalAgent (keepAlive now defaults to
true) then reused the dead socket for the next post() and read a TCP RST.
Linux runners are fast enough to never open the window, which is why this
never fired on ubuntu. Fix: both HTTP-speaking tests (smoke.js and
e2e-codewhale.js) now route every request through an explicit
new http.Agent({ keepAlive: false }) — one fresh connection per request,
nothing pooled, nothing to go stale. (Runtime-ownership and pr3-smoke do
no direct HTTP.)
- models-dev-sync: independent price cache from models.dev/catalog.json
(verified shape: {models, providers} — only providers carries prices; the
362-row top-level models object is provider-independent metadata). Catalog
legitimately contains keys like __proto__/constructor as provider/model ids:
the cache is built with null-prototype dictionaries and unsafe keys are
rejected outright, all numbers bounds-checked (NaN/Inf/negative/absurd ->
null), 64 MiB body cap, 15s timeout, atomic 0600 write, 24h TTL. Network
failure never blocks startup. Unknown paths on models.dev return 200 + SPA
HTML, so non-JSON parse failures are handled as refresh errors. The cache
path follows LLMPET_CODEWHALE_HOME like the ledger. On-demand refresh only
(no timers beyond a TTL check at read time).
- codewhale-metering: consumes the verified turn_end usage payload
(input_tokens/output_tokens/prompt_cache_hit/miss/write, reasoning_tokens)
keyed by turn_id (retried hook deliveries cannot double-bill; bounded dedup
window). DeepSeek semantics honored: input_tokens already includes the
cache-hit portion, so cached tokens are billed at the cache-read rate and
never twice. reasoning_replay_tokens have ambiguous upstream pricing —
counted in diagnostics, deliberately not billed. Unknown models count tokens
at honest $0 and land in diagnostics instead of being guessed. seenTurns
pruning is O(1) on the common path (size counter + batch drop to half the
cap); a 20k-turn bulk load records in <5s.
- adapter/growth/main: the CodeWhale ledger joins the machine total and the
main pet's billing view (tagged 'codewhale' in byModel/todayByProvider);
machineGrowth totals include it. Panel split line becomes a three-way join
when CodeWhale has spend (Claude $x · Codex $y · CodeWhale $z); machines
without CodeWhale spend keep the upstream two-way string; totals never
contradict the split sum again.
- test/pr3-smoke: protocol-shaped assertions (special keys, bounds, turn_id
dedup, cache-aware pricing math, TOML idempotence + old-format upgrade with
user content preserved, env override actually overriding, event-name
validity against the real CodeWhale vocabulary).
npm test exit 0.
This was referenced Aug 30, 2026
Closed
purrfecto114-lgtm
added a commit
to purrfecto114-lgtm/LLMPET
that referenced
this pull request
Aug 30, 2026
…rity hardening + CodeWhale provider + models.dev pricing + build/docs Merges the four stacked PR branches (pr/1..pr/4) into fork main: - myunwang#21 pr/1-security-hardening (11ff1ba): renderer sandbox+CSP, command-safety recognizer, PreToolUse gate, runtime first-live-wins - myunwang#24 pr/2-codewhale-provider (990b964): CodeWhale provider, permission bridge, TOML installer (+ Windows CI hardening) - myunwang#25 pr/3-models-dev-pricing (16987ce): models.dev price sync + CodeWhale usage ledger - myunwang#26 pr/4-build-docs-gui (da8fb38): Linux release job, lockfile guard, CodeWhale docs, GUI fixes All 4 PRs: CI green, mergeable_state=clean, stacked on upstream main 81d18d4. Conflict policy: theirs (PR side) — the series is the verified final state.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
这是什么（写给普通用户）
桌宠现在能算出 CodeWhale 花了多少钱：按 models.dev 的公开价格表给每个回合计价，日/周/终身花费一目了然，面板上的拆分行也变成三家的了。
你会在意的变化：
Claude $x · Codex $y · CodeWhale $z（没有 CodeWhale 花费的机器保持原样，不多噪音）测试：
npm testexit 0（新增 pr3-smoke：特殊 key、数值边界、去重、缓存计价数学、TOML 幂等+旧格式升级）。变更规模：9 个文件，+848/−27。堆叠说明： 本 PR 是 4 个堆叠 PR 的第 3 个（
pr/1-security-hardening→pr/2-codewhale-provider→pr/3-models-dev-pricing→pr/4-build-docs-gui）。依赖第 2 个（台账消费 provider PR 转发的 turn_end usage）。📐 技术附录（写给审阅者 / agent，点击展开）
1. models.dev 目录的形状（实测验证）
models.dev/catalog.json={models, providers}顶层两对象——只有providers带价格；362 行的顶层models是与 provider 无关的元数据。两个易错点：__proto__/constructor这类 key 作为 provider/model id → 缓存全部用 null-prototype 字典构建，危险 key 直接拒收（原型污染防护，变异测试验证过Object.create(null)被删掉时测试会红）2. 数值与网络边界
LLMPET_CODEWHALE_HOME（与台账一致；测试里 env 指向不同目录才是真覆盖——同目录是假覆盖，曾因此修过测试）3. 计量语义（DeepSeek 特有的坑）
input_tokens已包含 cache 命中部分 → 缓存读按 cache-read 价计、绝不双重计费reasoning_replay_tokens上游定价语义不明 → 计入诊断、故意不计费turn_id去重：钩子重试投递不会双计（有界去重窗口）seenTurns修剪 O(1) 常态路径 + 半量批量丢弃；20k 回合灌入 < 5s（曾是 O(n²) 全排序，20k 要 20s）4. 面板三向拆分
combineUsage增加 CodeWhale 泳道（machine 总额 + 主宠计费视图，byModel/todayByProvider 标记codewhale；machineGrowth总额含它）。拆分行规则：三家有量三向拼、两量两拼、单量不显示（单 agent 机器不该看到Codex $0.000噪音——上游原规则保留）。Claude/Codex/CodeWhale 是专有名词，三语言不翻译。5. pr3-smoke 覆盖清单
特殊 key 拒收、数值边界、turn_id 去重、缓存感知计价数学、TOML 幂等 + 旧格式升级（块重写无重复、用户内容保留）、env 真覆盖、事件名对真实 CodeWhale 词表校验、20k 灌入 < 5s。