Grill: how a derivation run aggregates into the previous model

[antejavor]

Part of #431

Question

Run N+1 must build on model N instead of starting over. How are types added, renamed, merged as synonyms, or split between versions? Where do the model and its versions live (graph or file), and how is the version an edge or entity was extracted under recorded? When the model changes, are old sessions re-extracted eagerly, lazily on read, or never?

[antejavor]

From the scope decision (#432, one omni model per user): a user's single model mixes every topic they talk about. This ticket should settle whether aggregation bounds the vocabulary (a size cap, or retiring low-use types) so a long history doesn't dilute it.

[antejavor]

From the schedule decision (#433): every session records the model version it was extracted under, and each derivation snapshots its delta (watermark) at start. Re-extraction of older sessions on adopting N+1 is left to this ticket. Each run's input is the delta plus model N, with older evidence carried as per-type observation counts — so this ticket's versioning needs to carry those counts across versions.

[antejavor]

Resolution

Grilled 2026-10-06. Builds on scope (#432: one omni model per user) and schedule (#433: delta + model N, per-type observation counts, version recorded per session).

Where the model lives: the graph. One (:OntologyVersion {user_id, version, created_at, status}) node per version, holding the model serialized as JSON plus the per-type observation counts. (:User)-[:ADOPTED]->(:OntologyVersion) points at the current version; NEXT chains versions. Adoption is one transaction moving the ADOPTED edge, so readers never see half a model. Rejected candidates stay with status='rejected' so the gate's reports can point at them. Every Session records ontology_version. A user-supplied YAML (ManualStrategy) loads as that user's version 1.

What a run may change. Exactly three operations, each written to the version node as a changelog:

• Add a type for what didn't fit N.
• Merge a type into another as a synonym — the old label becomes an alias; a rename is a merge into a better name. Vetoed automatically when the two types' start/end labels are incompatible. Counts are summed into the survivor.
• Retire a type whose observation share falls below a threshold — to a pool that keeps its counts, where it can return if evidence grows. Never deleted.

No splits. A too-broad type gets a narrower sibling added; new extractions use it, old entities stay under the broad type.

Bounding the omni model (dilution). A hard cap on active types, the default core (User, Person, value types) outside it: ~40 node types and ~60 relation types to start, tunable. A run that would exceed it retires the lowest-share types to the pool until it fits; the share threshold applies regardless. Rationale: GLiNER2 takes the whole label set on every call, so label count costs speed and accuracy. Enforced inside derivation, so the gate never sees an oversized candidate.

Old sessions on adopting N+1.

• Merges: migrated eagerly — old entities/edges relabelled alias → canonical, a cheap Cypher pass with or right after adoption.
• Retirements: left in place — labels stay readable, just no longer extracted.
• Adds: re-extract only that run's delta (≤128 sessions, the evidence the new types came from) as a detached background job, local GLiNER2 compute. Older sessions are never re-extracted. Each Session's ontology_version reflects what it was actually extracted under.

Considered and not taken: model as files; types as their own graph nodes; splits with reassignment; add-only; no cap; per-session label selection (reintroduces routing); eager full-history re-extraction; never re-extracting.