Skip to content

Record what each autonomous run consumes, privately #8

Description

@drevendev

Goal

Every author and acceptor run leaves a durable record of what it consumed, in a place
that is not public.

Evidence

  • anthropics/claude-code-action exposes execution_file, a JSON array of the run
    messages. The final result message carries usage with input_tokens,
    output_tokens, cache_creation_input_tokens, cache_read_input_tokens, plus
    total_cost_usd, duration_ms and num_turns.
  • This repository is public, so its Actions logs, job summaries and artifacts are
    public with it. None of them is a suitable home for operational telemetry.
  • The loop authenticates with a subscription token, so total_cost_usd is an estimate
    at API list prices, not a bill. Token counts are the real measurement.

Scope

A step in each of the two model-running workflows that reads the execution file and
writes one JSON record per run into the private repository drevendev/zen-telemetry,
plus the script and tests that do it.

Non-goals

  • No aggregation here. The rollup lives in the private repository, which owns the data.
  • No change to what the loop does. Telemetry observes; it does not participate.
  • Not the specification sync, which consumes no model and has nothing to record.

Acceptance criteria

  • Each author and acceptor run writes exactly one record, including runs that fail
  • The record distinguishes unknown usage from zero usage
  • A missing token or unreachable ledger produces a warning, never a failed run
  • Nothing about usage is written to this public repository
  • Tests prove the extraction, the fallback, and the unknown case

Verification

python -m unittest discover -s scripts/tests -v

End-to-end verification requires an actual model run, which cannot happen before the
loop is armed.

Dependencies, authority, and risks

Requires a secret TELEMETRY_PAT: a fine-grained token whose only access is
drevendev/zen-telemetry with Contents read and write. Without it the step warns and
the loop is unaffected.

Risk worth naming: this puts a cross-repository write token in the environment of a
public repository. It is mitigated by the token being scoped to one private repository
that holds no product, by policy-guard refusing diffs that mix workflow changes with
product code, and by the acceptor refusing to merge workflow changes that widen
authority.

Activity

  1. added
    type:processDevelopment workflow, governance, or coordination change
    area:toolingCI, scripts, guards, developer tooling
    status:in-progressClaimed work with an active branch or pull request
    on Sep 3, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    area:toolingCI, scripts, guards, developer toolingpriority:normalDefault planned prioritytype:processDevelopment workflow, governance, or coordination change

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions