Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
32 changes: 15 additions & 17 deletions templates/SKILL.md.loop-guard
Original file line number Diff line number Diff line change
Expand Up @@ -38,30 +38,27 @@ After every iteration, append what you just tried:
`outcome` is `success | failure | noop`. Always include `error` on failures —
that is how the breaker detects a repeated (stagnant) failure.

## Size the token budget from the pattern (once per run)
## Size the token budget from the pattern

Don't hand-type a token cap — derive it from the pattern's realistic per-run
cost so the breaker trips on genuine cost blowup, not a made-up number.
[`loop-cost`](https://github.com/cobusgreyling/loop-engineering/tree/main/tools/loop-cost)
already computes this from `patterns/registry.yaml`; read the loop's `pattern`
and `level` straight from the ledger:
Don't hand-type a token cap — `loop-context` can derive it directly from the
pattern's realistic per-run cost
([`loop-cost`](https://github.com/cobusgreyling/loop-engineering/tree/main/tools/loop-cost)
computes this from `patterns/registry.yaml`) so the breaker trips on genuine
cost blowup, not a made-up number. Substitute the ledger's own `pattern`/`level`
(here: `ci-sweeper` / `L2`):

```bash
# Substitute the ledger's own pattern/level (here: ci-sweeper / L2).
BUDGET=$(npx @cobusgreyling/loop-cost --pattern ci-sweeper --level L2 --json \
| node -e 'let d="";process.stdin.on("data",c=>d+=c).on("end",()=>process.stdout.write(String(JSON.parse(d).scenarios.realistic.tokensPerRun)))')
npx @cobusgreyling/loop-context --check --ledger loop-ledger.json \
--budget-from-pattern ci-sweeper --budget-level L2
```

`scenarios.realistic.tokensPerRun` is a rounded integer, so it feeds
`--token-budget` directly. The two tools stay independent — this is shell
wiring, not a code dependency.

## Before each iteration

1. Append the previous attempt to `loop-ledger.json`.
2. Run the breaker with the resolved budget:
```bash
npx @cobusgreyling/loop-context --check --ledger loop-ledger.json --token-budget "$BUDGET"
npx @cobusgreyling/loop-context --check --ledger loop-ledger.json \
--budget-from-pattern ci-sweeper --budget-level L2
```
3. Act on the exit code:
- **0** → continue. Optionally trim the next prompt first:
Expand All @@ -83,12 +80,13 @@ wiring, not a code dependency.
- Never widen thresholds just to keep looping — escalation is a feature, not a failure.
- Never edit the ledger to hide a repeated error; the breaker exists to catch it.
- Defaults: 3× same error, 5 consecutive failures, 10 iterations. Tune with
`--stagnation`, `--no-progress`, `--max-iterations`, `--token-budget`.
`--stagnation`, `--no-progress`, `--max-iterations`, or an explicit
`--token-budget` (wins over `--budget-from-pattern` if both are given).

## Interaction with other skills

- `minimal-fix` / `ci-triage` — record each attempt's outcome + error in the ledger.
- `loop-verifier` — a verifier rejection is a `failure`; log it so repeats trip the breaker.
- `loop-constraints` — honors "escalate after N attempts"; this skill makes it mechanical.
- `loop-budget` — the per-run `--token-budget` here comes from loop-cost's
realistic estimate; loop-budget.md still governs the *daily* cap across runs.
- `loop-budget` — the per-run budget here comes from `--budget-from-pattern`;
loop-budget.md still governs the *daily* cap across runs.
25 changes: 25 additions & 0 deletions tools/loop-context/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -75,6 +75,31 @@ cat run.json | loop-context --check

Exit codes: `0` continue · `2` escalate · `1` error.

## Resolving the token budget from a pattern

Typing `--token-budget <n>` by hand means guessing. [`loop-cost`](../loop-cost) already
computes a realistic per-run estimate for every pattern and readiness level, so resolve
the cap from there instead:

```bash
loop-context --check --ledger run.json --budget-from-pattern ci-sweeper --budget-level L2
```

| Flag | Default | Meaning |
|------|---------|---------|
| `--budget-from-pattern <id>` | none | Pattern id to look up in `loop-cost`'s registry |
| `--budget-level <L1\|L2\|L3>` | `L1` | Readiness level passed to `loop-cost` |
| `--budget-scenario <realistic\|action\|report>` | `realistic` | Which `loop-cost` scenario to use as the cap |
| `--budget-cadence <spec>` | pattern default | Cadence override passed through to `loop-cost` |
| `--budget-conservative` | off | Use the slower cadence in a range (`loop-cost`'s own flag) |

An explicit `--token-budget <n>` always wins over `--budget-from-pattern` — the derived
value only fills in when no number was typed. `loop-context` shells out to `loop-cost`'s
built CLI (monorepo sibling first, then an installed `@cobusgreyling/loop-cost`
dependency) — the same resolution `loop-init` already uses for `loop-audit` — so the two
packages stay independent at the source level; an unknown pattern id surfaces
`loop-cost`'s own error instead of failing silently.

## Populating the ledger

Your loop control script appends one object per iteration to `run.json` (or pipes the same shape on stdin):
Expand Down
19 changes: 19 additions & 0 deletions tools/loop-context/dist/budget-resolver.d.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
export type BudgetScenario = 'realistic' | 'action' | 'report';
export declare const VALID_BUDGET_SCENARIOS: BudgetScenario[];
export declare function assertValidBudgetScenario(scenario: string): asserts scenario is BudgetScenario;
export interface BudgetFromPatternInput {
pattern: string;
level?: string;
scenario?: BudgetScenario;
cadence?: string;
conservative?: boolean;
}
/** Locate loop-cost's built CLI: monorepo sibling first, then an installed dependency. */
export declare function resolveCostCli(): Promise<string | null>;
/**
* Resolve a token budget from loop-cost's realistic per-pattern estimate
* instead of a hand-typed number. Shells out to loop-cost's built CLI
* (same monorepo-then-installed-dependency resolution loop-init uses for
* loop-audit) so the two tools stay independent at the source level.
*/
export declare function resolveTokenBudgetFromPattern(input: BudgetFromPatternInput): Promise<number>;
85 changes: 85 additions & 0 deletions tools/loop-context/dist/budget-resolver.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,85 @@
import path from 'node:path';
import { fileURLToPath } from 'node:url';
import { access } from 'node:fs/promises';
import { spawn } from 'node:child_process';
const __dirname = path.dirname(fileURLToPath(import.meta.url));
const PACKAGE_ROOT = path.resolve(__dirname, '..');
export const VALID_BUDGET_SCENARIOS = ['realistic', 'action', 'report'];
export function assertValidBudgetScenario(scenario) {
if (!VALID_BUDGET_SCENARIOS.includes(scenario)) {
throw new Error(`Invalid --budget-scenario: ${scenario}. Valid: ${VALID_BUDGET_SCENARIOS.join(', ')}`);
}
}
async function exists(p) {
try {
await access(p);
return true;
}
catch {
return false;
}
}
/** Locate loop-cost's built CLI: monorepo sibling first, then an installed dependency. */
export async function resolveCostCli() {
const monorepo = path.resolve(PACKAGE_ROOT, '../loop-cost/dist/cli.js');
if (await exists(monorepo))
return monorepo;
try {
const { createRequire } = await import('node:module');
const require = createRequire(import.meta.url);
const pkg = require.resolve('@cobusgreyling/loop-cost/package.json');
return path.join(path.dirname(pkg), 'dist/cli.js');
}
catch {
return null;
}
}
function runCostCli(cli, args) {
return new Promise((resolve, reject) => {
const child = spawn('node', [cli, ...args, '--json'], { stdio: ['ignore', 'pipe', 'pipe'] });
let stdout = '';
let stderr = '';
child.stdout.on('data', (chunk) => (stdout += chunk.toString()));
child.stderr.on('data', (chunk) => (stderr += chunk.toString()));
child.on('error', reject);
child.on('close', (code) => resolve({ stdout, stderr, code }));
});
}
/**
* Resolve a token budget from loop-cost's realistic per-pattern estimate
* instead of a hand-typed number. Shells out to loop-cost's built CLI
* (same monorepo-then-installed-dependency resolution loop-init uses for
* loop-audit) so the two tools stay independent at the source level.
*/
export async function resolveTokenBudgetFromPattern(input) {
const scenario = input.scenario ?? 'realistic';
assertValidBudgetScenario(scenario);
const cli = await resolveCostCli();
if (!cli) {
throw new Error('--budget-from-pattern requires @cobusgreyling/loop-cost. Install it, or run from the loop-engineering monorepo.');
}
const args = ['--pattern', input.pattern, '--level', input.level ?? 'L1'];
if (input.cadence)
args.push('--cadence', input.cadence);
if (input.conservative)
args.push('--conservative');
const { stdout, stderr, code } = await runCostCli(cli, args);
if (code !== 0) {
throw new Error(stderr.trim() || `loop-cost exited with code ${code}.`);
}
if (!stdout.trim()) {
throw new Error('loop-cost produced no output.');
}
let parsed;
try {
parsed = JSON.parse(stdout);
}
catch {
throw new Error('loop-cost produced output that could not be parsed as JSON.');
}
const tokensPerRun = parsed.scenarios?.[scenario]?.tokensPerRun;
if (typeof tokensPerRun !== 'number') {
throw new Error(`loop-cost output missing scenarios.${scenario}.tokensPerRun.`);
}
return tokensPerRun;
}
43 changes: 41 additions & 2 deletions tools/loop-context/dist/cli.js
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
#!/usr/bin/env node
import { readFile } from 'node:fs/promises';
import { buildContextInjection, checkCircuitBreaker, pruneLedger, summarizeAttempts, DEFAULT_BREAKER, DEFAULT_PRUNE, } from './context-manager.js';
import { resolveTokenBudgetFromPattern, } from './budget-resolver.js';
/** Reject NaN/0/floats so a bad flag cannot silently disable the breaker. */
function parsePositiveIntFlag(raw, flag) {
if (raw === undefined || raw === '') {
Expand All @@ -18,10 +19,19 @@ function parseArgs(argv) {
let op = 'status';
let ledger;
let json = false;
let budgetFromPattern;
let budgetLevel = 'L1';
let budgetScenario = 'realistic';
let budgetCadence;
let budgetConservative = false;
const base = () => ({
op, json, breaker, prune,
budgetFromPattern, budgetLevel, budgetScenario, budgetCadence, budgetConservative,
});
for (let i = 0; i < argv.length; i++) {
const a = argv[i];
if (a === '--help' || a === '-h')
return { help: true, op, json, breaker, prune };
return { help: true, ...base() };
else if (a === '--ledger' || a === '-f')
ledger = argv[++i];
else if (a === '--check')
Expand All @@ -44,12 +54,22 @@ function parseArgs(argv) {
breaker.noProgressThreshold = parsePositiveIntFlag(argv[++i], '--no-progress');
else if (a === '--token-budget')
breaker.tokenBudget = parsePositiveIntFlag(argv[++i], '--token-budget');
else if (a === '--budget-from-pattern')
budgetFromPattern = argv[++i];
else if (a === '--budget-level')
budgetLevel = argv[++i];
else if (a === '--budget-scenario')
budgetScenario = argv[++i];
else if (a === '--budget-cadence')
budgetCadence = argv[++i];
else if (a === '--budget-conservative')
budgetConservative = true;
else if (a === '--window')
prune.window = parsePositiveIntFlag(argv[++i], '--window');
else if (a === '--max-trace-lines')
prune.maxTraceLines = parsePositiveIntFlag(argv[++i], '--max-trace-lines');
}
return { help: false, op, ledger, json, breaker, prune };
return { help: false, ledger, ...base() };
}
async function readLedger(pathArg) {
const raw = pathArg
Expand Down Expand Up @@ -81,6 +101,7 @@ Keeps a loop's context window clean and stops runaway loops. Reads a run ledger
Usage:
loop-context [operation] [--ledger <file.json>] [options]
cat ledger.json | loop-context --check
loop-context --check --ledger run.json --budget-from-pattern ci-sweeper --budget-level L2

Operations (default: --status):
--check Run the circuit breaker. Exit 0 = continue, 2 = escalate.
Expand All @@ -96,6 +117,15 @@ Options:
--stagnation <n> Same-error repeat limit (default: ${DEFAULT_BREAKER.stagnationThreshold})
--no-progress <n> Consecutive-failure limit (default: ${DEFAULT_BREAKER.noProgressThreshold})
--token-budget <n> Total token cap (default: none)
--budget-from-pattern <id>
Resolve the token cap from loop-cost's registry
estimate instead of typing a number. Ignored if
--token-budget is also given (explicit wins).
--budget-level <L1|L2|L3> Readiness level for --budget-from-pattern (default: L1)
--budget-scenario <realistic|action|report>
Which loop-cost scenario to use (default: realistic)
--budget-cadence <spec> Cadence override passed through to loop-cost
--budget-conservative Use the slower cadence in a range (loop-cost flag)
--window <n> Attempts kept when pruning (default: ${DEFAULT_PRUNE.window})
--max-trace-lines <n> Stack-trace lines kept (default: ${DEFAULT_PRUNE.maxTraceLines})
-h, --help This help
Expand All @@ -112,6 +142,15 @@ async function main() {
console.log(HELP);
return;
}
if (args.budgetFromPattern && args.breaker.tokenBudget === undefined) {
args.breaker.tokenBudget = await resolveTokenBudgetFromPattern({
pattern: args.budgetFromPattern,
level: args.budgetLevel,
scenario: args.budgetScenario,
cadence: args.budgetCadence,
conservative: args.budgetConservative,
});
}
const ledger = await readLedger(args.ledger);
switch (args.op) {
case 'check': {
Expand Down
37 changes: 35 additions & 2 deletions tools/loop-context/package-lock.json

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

5 changes: 4 additions & 1 deletion tools/loop-context/package.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"name": "@cobusgreyling/loop-context",
"version": "1.0.0",
"version": "1.1.0",
"description": "Stateful memory manager for agent loops — summarize, prune, and inject context, with a circuit breaker that escalates stagnant or no-progress runs instead of burning tokens.",
"type": "module",
"main": "dist/context-manager.js",
Expand Down Expand Up @@ -48,6 +48,9 @@
"publishConfig": {
"access": "public"
},
"dependencies": {
"@cobusgreyling/loop-cost": "^1.0.3"
},
"devDependencies": {
"@types/node": "^20.0.0",
"typescript": "^5.0.0"
Expand Down
Loading