feature(agent): bound the pattern catalog + add purge API - #250
Open
nghiadaulau wants to merge 1 commit into
Open
feature(agent): bound the pattern catalog + add purge API#250nghiadaulau wants to merge 1 commit into
nghiadaulau wants to merge 1 commit into
Conversation
The pattern catalog grew without limit — a noisy environment accumulates templates forever, inflating patterns.json and the in-memory map. Bound it on the persist tick: - agent.catalog.max_patterns (default 5000) evicts the oldest patterns over the cap. - agent.catalog.retention (default 720h) drops patterns idle longer than the window. Eviction only ever touches UNCURATED patterns (no verdict, no tags) — an operator-labelled pattern is never auto-removed. 0/negative cap and "0" retention disable each independently. Admin purge endpoints (gated by X-Gateway-Secret): - DELETE /api/agent/patterns?service=&older_than= — bulk remove by service and/or idle age (explicit action; removes curated patterns too). - DELETE /api/agent/services/:name — remove a service's first-seen entry. Catalog.Sweep / PurgePatterns / DeleteService carry the logic; the worker calls Sweep before each flush. Config triple-touch (struct + clone_config + config.yaml) + docs. Not added to the Helm chart, which exposes no other agent.catalog tuning. Tests: Sweep retention + cap + curated-protection + disabled-by-zero; PurgePatterns by service and by age; DeleteService.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
The AI-agent pattern catalog grew without bound. A noisy environment keeps
producing new Drain templates, so
patterns.json(and the in-memory map)inflate indefinitely — there was no cap, no retention, and no way to bulk-
remove stale entries.
Fix
Bounds, enforced on the persist tick (
Catalog.Sweep):agent.catalog.max_patterns(default5000) — evict the oldest patternsonce the catalog exceeds the cap.
agent.catalog.retention(default720h) — drop patterns idle longerthan the window.
Eviction only ever touches uncurated patterns (no verdict, no tags). A
pattern an operator has labelled (
known/spike) or tagged is neverauto-removed and still counts toward the cap, so a fully-curated catalog is
never truncated. A
0/negative cap and"0"retention disable eachindependently.
Admin purge endpoints (gated by
X-Gateway-Secret):DELETE /api/agent/patterns?service=<name>&older_than=<dur>— bulk removeby service and/or idle age (both optional; neither = all). This is an
explicit operator action, so it removes curated patterns too.
DELETE /api/agent/services/:name— remove a service's first-seentracking entry (patterns left intact).
Changes
pkg/agent/catalog.go—Sweep,PurgePatterns,DeleteService,protectedFromEviction.pkg/agent/worker.go— sweep before each catalog flush; bounds parsedfrom config (defaults 5000 / 720h).
pkg/controllers/agent.go— the two new DELETE routes/handlers.Not in Helm
The chart exposes no
agent.catalogtuning today (spike_multiplier,persist_interval, etc. are code-default only), so these knobs follow thesame convention rather than being the lone catalog entries in
values.yaml.Testing
Catalog.Sweep— retention eviction, cap eviction (oldest first), curatedprotection, and disabled-by-zero.
PurgePatternsby service and by age;DeleteServiceknown/unknown.gofmt,go vet,go build ./...,go test -race ./pkg/agent ./pkg/config ./pkg/controllersgreen.
Checklist
feature:prefix); docs + CHANGELOG