Grilling: what triggers each background organizing job, and at what cadence?

[antejavor]

Part of #259

Question

The driving user said organizing should be "constantly happening" — does that mean a long-running daemon/worker process (a new deployable component for context-graph), or a frequently-scheduled batch sweep (extending today's per-session reconciliation_status='pending' + manual/scheduled reconcile --pending pattern to run on a timer)? Does this differ per memory type (semantic vs. episodic vs. procedural), or is one mechanism/cadence shared across all three? Resolve with the driving user via /grilling + /domain-modeling.

[antejavor]

Resolution

Trigger/cadence resolves via /grilling + /domain-modeling with the driving user:

No new daemon/worker process. A cron-style periodic sweep over existing/extended CLI commands, scheduled by the user's own ops infrastructure (cron, systemd timer, cloud scheduler, etc.) — not something context-graph runs continuously itself. This is safe specifically because both mechanisms below make "nothing pending" a cheap no-op, satisfying the driving user's "don't waste resources if nothing is pending" requirement without needing new infrastructure to guarantee it.

Episodic + semantic share one combined pass: reconcile_session() gets extended to produce both outputs — the existing entity/Chunk extraction and the new Session.summary (from #261) — from a single LLM call over the same session content, rather than two separate calls over identical input. Triggered by the existing per-session reconciliation_status='pending' flag (set synchronously at SESSION_END, as today). Recommended cadence: short interval (minutes-scale) — cheap to check, bounded by how many sessions recently ended.

Procedural mining is structurally separate: driven by a watermark/cursor over Action history, not a per-item pending flag — because a pattern can only be detected by looking across many sessions, not by processing one pending item at a time. The sweep is a no-op if no new Action data has landed since the last watermark; when there is new data, only the delta since the watermark needs checking (not a full history re-scan). Recommended cadence: longer interval (hourly/daily-scale) — cross-session pattern detection is inherently more expensive and needs accumulated data to be meaningful.

Implementation note for whoever builds this: extending reconcile_session() to also produce Session.summary in the same pass (rather than adding a second LLM call) is the recommended shape, but wasn't verified against reconcile_session()'s current internal structure in this ticket — worth a quick look before assuming it's a trivial addition.