Skip to content

feature(agent): bound the pattern catalog + add purge API - #250

Open
nghiadaulau wants to merge 1 commit into
VersusControl:mainfrom
nghiadaulau:feat/agent-catalog-cap
Open

feature(agent): bound the pattern catalog + add purge API#250
nghiadaulau wants to merge 1 commit into
VersusControl:mainfrom
nghiadaulau:feat/agent-catalog-cap

Conversation

@nghiadaulau

Copy link
Copy Markdown
Member

Problem

The AI-agent pattern catalog grew without bound. A noisy environment keeps
producing new Drain templates, so patterns.json (and the in-memory map)
inflate indefinitely — there was no cap, no retention, and no way to bulk-
remove stale entries.

Fix

Bounds, enforced on the persist tick (Catalog.Sweep):

  • agent.catalog.max_patterns (default 5000) — evict the oldest patterns
    once the catalog exceeds the cap.
  • agent.catalog.retention (default 720h) — drop patterns idle longer
    than the window.

Eviction only ever touches uncurated patterns (no verdict, no tags). A
pattern an operator has labelled (known / spike) or tagged is never
auto-removed and still counts toward the cap, so a fully-curated catalog is
never truncated. A 0/negative cap and "0" retention disable each
independently.

Admin purge endpoints (gated by X-Gateway-Secret):

  • DELETE /api/agent/patterns?service=<name>&older_than=<dur> — bulk remove
    by service and/or idle age (both optional; neither = all). This is an
    explicit operator action, so it removes curated patterns too.
  • DELETE /api/agent/services/:name — remove a service's first-seen
    tracking entry (patterns left intact).

Changes

  • pkg/agent/catalog.goSweep, PurgePatterns, DeleteService,
    protectedFromEviction.
  • pkg/agent/worker.go — sweep before each catalog flush; bounds parsed
    from config (defaults 5000 / 720h).
  • pkg/controllers/agent.go — the two new DELETE routes/handlers.
  • Config triple-touch (struct + clone_config + config.yaml) + docs.

Not in Helm

The chart exposes no agent.catalog tuning today (spike_multiplier,
persist_interval, etc. are code-default only), so these knobs follow the
same convention rather than being the lone catalog entries in values.yaml.

Testing

  • Catalog.Sweep — retention eviction, cap eviction (oldest first), curated
    protection, and disabled-by-zero.
  • PurgePatterns by service and by age; DeleteService known/unknown.
  • gofmt, go vet, go build ./..., go test -race ./pkg/agent ./pkg/config ./pkg/controllers
    green.

Checklist

  • Tests for sweep, purge, and the protection rule
  • Config triple-touch
  • Curated patterns never auto-evicted
  • House conventions (feature: prefix); docs + CHANGELOG

The pattern catalog grew without limit — a noisy environment accumulates
templates forever, inflating patterns.json and the in-memory map.

Bound it on the persist tick:
- agent.catalog.max_patterns (default 5000) evicts the oldest patterns over
  the cap.
- agent.catalog.retention (default 720h) drops patterns idle longer than the
  window.
Eviction only ever touches UNCURATED patterns (no verdict, no tags) — an
operator-labelled pattern is never auto-removed. 0/negative cap and "0"
retention disable each independently.

Admin purge endpoints (gated by X-Gateway-Secret):
- DELETE /api/agent/patterns?service=&older_than=  — bulk remove by service
  and/or idle age (explicit action; removes curated patterns too).
- DELETE /api/agent/services/:name  — remove a service's first-seen entry.

Catalog.Sweep / PurgePatterns / DeleteService carry the logic; the worker
calls Sweep before each flush. Config triple-touch (struct + clone_config +
config.yaml) + docs. Not added to the Helm chart, which exposes no other
agent.catalog tuning.

Tests: Sweep retention + cap + curated-protection + disabled-by-zero;
PurgePatterns by service and by age; DeleteService.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant