Skip to content

Temp-worker tokenCap counts cache reads, so workers are reaped far below their real work budget #653

Description

@ayaan47-1

What happened?

Temp workers are reaped long before they reach their tokenCap in real work. The reaper (ephemeralWorkerTick in src/main/index.ts) compares workerTokensUsed(), which is input + output + cacheRead + cacheCreation, against tokenCap.

cacheRead grows with the number of requests against a large fixed context, not with what the worker does. The per-agent breaker budget already stopped counting it for this reason (#189, workTokensOf in src/main/breaker.ts), but the worker reaper was never moved over, so the two budgets measure different things.

Expected: a worker is reaped when its work tokens (input + output + cacheCreation) exceed its cap, the same as the breaker.

Steps to reproduce

  1. Write a spawn request with "tokenCap": 3000000 and a multi-step objective, for example research or a test-fix loop.
  2. Let it run. Every turn re-reads the cached context.
  3. The worker is reaped: "used 3,121,475 tokens (> its cap of 3,000,000)".
  4. The usage ledger's last row before the reap was 1,049 input / 16,474 output / 2,314,181 cache read / 128,694 cache creation. That is 94% cache reads and about 146k work tokens, under 5% of the cap.

Evidence

Last cost-ledger row of four reaped temps (all Opus):

Cap input / output / cacheRead / cacheCreation Work tokens Cache-read share
1.5M 109,377 / 18,219 / 976,611 / 211,237 338,833 74%
2.5M 95,680 / 19,206 / 2,223,519 / 79,083 193,969 92%
1.8M 48,548 / 15,546 / 1,756,594 / 76,801 140,895 93%
3.0M 1,049 / 16,474 / 2,314,181 / 128,694 146,217 94%

Logs

[worker reaped — token cap] worker-tt-oiliness-check: Worker worker-tt-oiliness-check used 3,121,475 tokens (> its cap of 3,000,000) and was reaped.

Environment

  • Munder Difflin 0.5.4 (packaged); also present on main at 2a5a585
  • macOS (Darwin 27.0)

A fix with tests is in a PR that follows.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions