Skip to content

Token audit: verify context triggers use active request size, not lifetime usage #202

Description

@Hmbown

Problem

A token-management system can make a strong model behave dumb if it triggers context rewrites from the wrong token basis.

In the v0.7.5 hard-cycle review, we found one concrete foot-gun: lifetime API usage can count the same stable prefix repeatedly across turns. Cycle/seam decisions should be based on the active request input estimate, with future assistant-output headroom considered against the model window, not cumulative billed/usage totals.

Audit scope

Read first:

Questions to answer

  • Are any context-management decisions still using lifetime summed usage where they should use active request size?
  • Are UI/status/cost labels clear about the difference between current active context, cumulative billed prompt tokens, output tokens, and cache hit/miss tokens?
  • Do hard cycle, optional seam production, emergency overflow recovery, manual compaction, and capacity-controller checkpoints all use the correct token basis for their job?
  • Are tests explicit enough that future agents will not reintroduce cumulative-usage based triggers?

Acceptance checks

Non-goals

  • No provider expansion.
  • No Whalescale rename/migration.
  • No Cargo/npm publish.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    contextContext management / contextenhancementNew feature or request

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions