data-model.md

Data Model

Records, IDs, associations, types, and queries. For who may read or write them, see Access control; for version history and deletion, see Versioning & deletion.

Records

A Record is the fundamental unit of data in a Stack.

type StackRecord = {
  // --- Core (always present) ---
  id: RecordId; // Crockford base-32, time-sortable, unique within a stack
  typeId: TypeId; // Versioned Type ID e.g. "com.example.myapp/note@2"
  createdAt: Date;
  updatedAt: Date;
  content: Record<string, unknown>; // Validated against the Type's schema
  version: number; // Ordinal of the snapshot history. Powers rollback and soft-delete recovery; can optionally gate writes via ifVersion (see Versioning & deletion)

  // --- Optional native fields ---
  parentId?: RecordId; // ID of a parent Record (for hierarchy/folders)
  appId?: AppId; // Software that created this Record, reverse-DNS e.g. "com.example.myapp". Self-reported; never a permission input (see Identity)
  createdBy?: Actor; // The author. A scoped write always stamps it, so absent means an unscoped Stack wrote the Record (see Authorship and attribution)
  updatedBy?: Actor; // Who performed the most recent mutation. Unlike createdBy, it moves with every write
  deletedAt?: Date; // Present if soft-deleted
  unlistedAt?: Date; // Present if withheld from enumeration — reachable by get(), absent from query()/the feed by default (see Unlisted records)
  permissions?: AuthorityAssociation[]; // Who reaches this Record (see Access control)
  associations?: DataAssociation[]; // Tags, attachments, relationships
};

Design principle: native fields are things the library needs to operate (routing, querying, syncing, hierarchy). Everything semantic and domain-specific goes in content.

RecordId, TypeId, BaseId, FileId, AppId and EntityId are all string at runtime. The aliases name which identifier space a value belongs to, and the spec's type blocks use them in the same places the exported types do.

Actor

"Who did this, and through which principal" is one pair of identities, spelled one way everywhere it appears — on a Record, a version, a change event and a token session:

type Actor = {
  subjectId: EntityId; // Who the act is attributed to. Governs authorship and -own
  principalId?: EntityId; // Who authenticated, when it isn't the subject — a delegated app's own DID
};

An absent principalId means the subject acted as itself. Stack stores a principalId equal to subjectId as absent, so "acted as itself" has one stored spelling and a present principalId is itself the signal that a delegated app wrote. A TokenSession is the one Actor that always carries both — principalId equal to subjectId on an undelegated token — so a server reading it never has to default the field; it passes to Stack.asActor() as is.

Authorship and attribution

Two different questions, answered by two different Actors.

createdBy describes the Record. Its subjectId is the author, stamped once by create(); it never moves, because "whose Record is this" does not change when someone else edits it. Its principalId is the principal behind that create. Together they are the Record's provenance, and createdBy.subjectId is what every -own grant resolves against — so a later editor never acquires -own standing over what they edited.

updatedBy describes the latest mutation. Every mutation restamps it with the requester, naming the principal beside the subject only when the two differ, exactly as createdBy does for the create. At version 1 the two agree, since a Record's first actor is its author; from version 2 they diverge whenever anyone but the author writes.

Because a version snapshot captures the Record as it stood at that version, each snapshot carries the actor that produced it. A version history is therefore a record of both states and actors, and the live Record answers for the current version.

• Absent means unknown, never "the author". An unscoped Stack names no requester, so it writes no actor — and clears any the Record carried, rather than leaving the previous one in place to be read as this one.
• updatedBy is not a permission input. No grant, permission entry or gate resolves against it. It is an audit fact, not an authority one, and reading it as authority would let a write-holder acquire standing by touching a Record.
• Both are assigned from the authenticated session and ignored on input — see Wire format § Records.
• restoreVersion() stamps the restorer. A rollback is a write by whoever performs it, so it never restores the stamp along with the content.

Record IDs

IDs are Crockford base-32, lowercase, exactly 12 characters: a 9-character timestamp prefix (so lexicographic order matches creation order) plus a 3-character random suffix, monotonically incremented for IDs generated in the same millisecond.

A millisecond therefore holds a bounded number of IDs, and how many is itself random: the suffix opens on a random draw and increments from there, so the room left runs anywhere from one ID to 32³. Spending it carries the next ID into the following millisecond — the prefix advances, a fresh suffix is drawn, and the order stays total. Minting never fails for want of room, so no caller meets a capacity it had no way to predict.

A carried ID's prefix therefore reads a millisecond later than the createdAt stamped beside it — the same relationship a backward clock step (NTP correction, suspend/resume) produces, where the generator's monotonic floor clamps the prefix forward so no ID sorts before ones already handed out while createdAt reports the wall clock honestly. Ordering is unaffected either way: queries sort by createdAt with id as the tiebreak, and both agree with creation order.

Client-minted IDs are the default and stay supported. Stack.create() accepts an optional id, so an app that needs the ID before the write round-trips can supply its own; omit it and the library generates one. A server in front of the stack doesn't change this — the client still mints the ID — but the receiver validates it rather than trusting it:

• Charset and length — exactly 12 characters, lowercase Crockford base-32 (0-9, a-z excluding i, l, o, u). Violations → 400.
• Reserved prefix — an id beginning with _ is rejected; that namespace is reserved for system records (_config, _entity, etc. — see System types). Violations → 400.
• Duplicate ID — an id that already exists in the stack → 409 (StackConflictError), never a silent overwrite.

A server MAY additionally reject an id whose timestamp prefix is implausibly far from server time (a clock-skew tolerance measured in hours, not years). This is optional: legitimate offline-created records carry an honest but stale prefix, so it's a per-deployment policy choice, not a spec mandate.

IDs are guessable, and nothing here tries to fix that. The prefix decodes to the creation millisecond, and same-millisecond IDs increment rather than re-randomise, so holding one ID from a batch yields its siblings. Both properties are the point — the first is what makes IDs sortable, the second is what keeps them sortable within a millisecond — and 12 characters is a deliberate ceiling, since an ID is meant to be typed by a human and to keep a URL short enough not to be auto-shortened when shared. Widening the suffix would buy little (it does nothing about siblings) and randomising it collides too often at three characters. What closes the gap instead is refusing to confirm a candidate: see Refusals and disclosure. The duplicate-id conflict above is the one confirmation that survives, deliberately — it is what makes a create idempotent under retry — and it is bounded by the skew check and visible to the owner, since a probe has to write a record to ask.

The same rules are enforced locally, so a client-minted ID behaves identically whether it travels over the wire or stays in-process:

• Stack.create(typeId, content, { id }) validates charset, length, and the reserved prefix, and throws StackConflictError on a duplicate. This is a full-trust context (an embedded single-app stack, or the server's own code) — no clock-skew check.
• ScopedStack.create() — a grantee minting an ID — applies the same validation plus the timestamp-skew check, since a grantee is exactly the untrusted actor who could otherwise forge a sort position. The tolerance is configurable per Stack via Stack.open(adapter, { idTimestampSkewMs }) (default 24 hours; pass null to disable).

Backdating on import

Stack.create() also accepts createdAt/updatedAt (BackdatableCreateRecordOptions) so an app can import an existing corpus with its real dates instead of every record landing stamped with the import moment:

• Unconditional on unscoped Stack.create() — the same full-trust context as the id option above.
• Owner-only on ScopedStack.create(). Refused to everyone but the stack owner acting alone (undelegated, authenticated as themselves — the same ownerActingAlone tier that already gates purge, commitMigration(), and includeUnlisted): a grantee, or a delegated app acting for the owner, could otherwise forge a sort position through createdAt the same way the id skew check exists to stop it forging one through id. ScopedStack.create() refuses both fields outright for anyone else.
• Owner-authenticated only, over the wire. POST /records may carry createdAt/updatedAt when the request authenticates as the stack owner acting alone. Unlike createdBy, this is not inherited from ScopedStack for free: that is silently overridden, while createdAt/updatedAt are refused outright, and every client sends both fields on every create (a record body is a whole record). A server must therefore drop them from a non-owner body itself — forwarding one unfiltered turns an ordinary grantee create into a StackPermissionError rather than a create stamped with the current time. See Wire format § Records.
• id and createdAt must agree. Omit id and it's derived from createdAt's timestamp, so the two can't diverge. Supply both, and they're checked against each other using the same idTimestampSkewMs tolerance the id-vs-current-time check above uses (default 24 hours; null disables this check too) — disagreement beyond that tolerance throws StackValidationError rather than silently diverging. Supplying id alone, with no createdAt, is unaffected: it stays a pure position choice — including for the owner, whose plain id-only creates through ScopedStack still get the ordinary id-vs-current-time check, not this one.
• updatedAt defaults to createdAt, not to the actual current time, so a plain import doesn't fabricate a fake edit and inflate version history. Supplying an updatedAt earlier than createdAt is a validation error — including when createdAt was left to default to now.
• Both fields must be valid, representable Dates. An Invalid Date (what new Date() yields for a malformed date string, a common shape for a bad row in an imported corpus) is a StackValidationError, not a record: its getTime() is NaN, and every comparison against NaN is false, so an unchecked one would switch off the ordering and skew checks above rather than fail them. The representable range is the range a record ID's 9-character timestamp prefix can encode — 1970-01-01T00:00:00.000Z through 3084-12-12T12:41:28.831Z — since a createdAt outside it has no ID that can agree with it. Content genuinely dated outside that window belongs in the record's own content fields, not in createdAt.
• Backdated records are invisible to an updatedAt cursor. A backdated record's updatedAt predates its import by construction, so a consumer syncing incrementally by filter.updatedAt.after never sees it arrive — which is the point (an import is not a recent edit), but it means an import is picked up by a full corpus read, not by a change cursor.

Mutations

One call, one version. mutate(id, changes, opts) is the write surface for an existing Record. It takes a change set naming any combination of aspects, applies them in a single atomic write, and produces exactly one new version carrying one snapshot of the prior state — however many aspects it moved.

await stack.mutate(noteId, {
  contentPatch: { title: 'Q3 plan' },
  parentId: plansFolderId,
  unlisted: false,
});

type RecordChangeSet = {
  contentPatch?: Record<string, unknown | null>;
  parentId?: string | null;
  permissions?: AuthorityAssociation[];
  associations?: DataAssociation[];
  unlisted?: boolean;
};

contentPatch is the only key that decides whether the call bumps. A change set naming only the others writes no version and takes no snapshot, exactly as associate()/dissociate() below do not — each of them is invertible from the journal entry the write appends, so none ever needed the recoverability a snapshot buys (see Versioning § Version history, and the change journal for where their prior state is kept instead). Name any of them alongside contentPatch and the call bumps once.

One call, one act. mutate() is the only multi-aspect atomic write a Record has, and that — not version batching — is why it exists: a publish is one act, so nothing it names may half-land.

associations and permissions are two domains, not one list. They are projections over one stored set, partitioned by kind; each key replaces only within its own, and a kind named in the wrong one is a StackBadRequestError. An app editing tags is never handed the authority half, so it cannot drop it.

Every key replaces the aspect it names, except contentPatch, which says so in its name. permissions and associations replace what the Record holds within their own domain, unlisted sets or clears the withholding, and parentId names a container or null for the root. contentPatch merges instead: an omitted field keeps its current value and null removes one. The name carries the whole disambiguation — a key called content sitting beside keys that replace would have to be read against this paragraph to know that it doesn't, and the one asymmetry in the envelope is worth a longer key rather than a footnote.

Content is a patch because uniformity costs more here than it buys. limits.contentBytes bounds what travels, so a whole-document write puts a one-field edit against the full ceiling; two apps editing different top-level fields of one Record both survive a patch and clobber one another under replacement; and content read through presentAt: 'latest' cannot be written back wholesale at all, since a write validates against the Record's own stored Type and a read-modify-write would submit migrated content to the schema it was migrated away from. A patch touches only the fields the caller named, so none of the three arises.

A content patch merges at the top level only. Each key the patch names is replaced whole: patching { address: { city: 'Lisbon' } } sets address to exactly that object, dropping any postcode it held. This is not RFC 7396, which recurses — the merge is one level deep, and a caller meaning to change one field of a nested object sends that object as it should end up. Nesting is where a recursive merge stops being predictable (it cannot express "replace this object wholesale" at all), and one level is the depth at which "omitted means unchanged" stays something a caller can hold in their head.

A key is read for presence, not for truthiness. unlisted: false and parentId: null are changes and are applied; an omitted key is an aspect the write does not touch. A change set naming no key is refused with StackBadRequestError (wire: 400) — it addresses no aspect at all, so there is nothing for it to have failed to satisfy, and no reading of it a caller meant.

A change that changes nothing writes nothing. Every key present is compared against the Record as it stands, and a change set already satisfied in each one returns the Record unchanged: no new version, no snapshot, no event. The comparison is per key and the decision is over the whole set — a change set that moves one aspect and restates another still writes, and writes once. associate() and dissociate() answer the same way on their own.

patchContent(id, patch, opts) is the content-only spelling, exactly mutate() with contentPatch alone. Content edits outnumber every other kind by a wide margin, and the name says what the operation does instead of promising a symmetry with create() that a patch does not have.

associate() and dissociate() are not the associations key. Each takes a list of Associations to add or remove, matched by kind, label and payload, as one atomic write and one journal entry; an element the Record already stands that way on is a no-op for that element, and a call where every element is a no-op returns the Record without writing. Neither bumps version or updatedAt, per the rule above. The key replaces the set; the methods amend it. The difference is load-bearing under concurrency — two apps tagging one Record both succeed through the methods and race through the key — so the delta spelling is kept for the operation that most needs it, rather than folded into a declarative envelope, where "add this one" is not a thing that can be said.

amendAssociations(id, changes) is the atomic way to add and remove in one write. It takes a list of AssociationEdit — the journal's { op: 'add' | 'remove', association } shape without repoint — and associate() and dissociate() are the same call with one half each. Swapping a cover is one call, one write and one journal entry:

stack.amendAssociations(bookId, [
  { op: 'remove', association: oldCover },
  { op: 'add', association: newCover },
]);

repoint is the journal's word, never a request: it is refused with StackValidationError at the element, and 400 bad_request on the wire. An add naming an attachment the Record already holds re-points it in place, which is all a repoint could say.

A list is held to one set of rules, whichever verb carries it, before anything is read or written:

• An empty list is refused with StackValidationError.
• A list naming one identity twice is refused, as the associations key is (Associations are keyed by identity). So is a list that both removes and adds one identity.
• A list never mixes authority and data elements (Access control § Storage unifies; the API does not).
• A remove matches on identity, as dissociate() does.
• The write-implies-read and _group at-least-one-admin checks read the post-state of the whole call.

grantAccess(), revokeAccess() and amendAccess() stand in the same relation to the permissions key, and carry the reshare gate rather than the write bit — see Access control § Record-level permissions. grantAccess() and revokeAccess() take lists, and amendAccess(id, changes) is amendAssociations() for permissions. The concurrency warning above applies to permissions exactly as it does to associations: two admins granting different people through the key clobber, where the verbs let both land.

What the envelope does not carry. typeId moves only through commitMigration(), which replaces content wholesale under a new schema and carries its own owner-only gate. deletedAt moves only through delete()/undelete(): a tombstone transition is a lifecycle step rather than an edit, and mutations are refused against a tombstone rather than bundled with one. createdAt and updatedAt are settable at create time only, by the owner acting alone. Every remaining native field is stamped by the write itself or is create-only — Reparenting gives the full split.

Authority is resolved per key, against the Record as it stands. A caller who may edit content and a caller who may reshare are not the same caller, and naming both in one change set gives neither of them the other's reach — see Access control § Composing a change set.

Associations

Tags, attachments, and relationships are unified under a single Association model: each associates a Record with a labeled payload, and the label carries semantic meaning (e.g. "avatar", "parent", "reply-to").

Every edge a Record carries is one of these, authority included. The three arms below are the data half — what the associations field projects and associate()/dissociate() accept; the authority half is permissions. One shape, one delta and one durability tier across both, and two call surfaces, because they carry different authority.

type DataAssociation =
  | { kind: 'tag'; label: string }
  | { kind: 'attachment'; label: string; fileId: string; attachmentRecordId?: string }
  | { kind: 'relationship'; label: string; target: RelationshipTarget };

// The authority half — see Access control § Record-level permissions.
type AuthorityAssociation = PermissionAssociation | AnyoneAssociation;

type Association = DataAssociation | AuthorityAssociation;

type RelationshipTarget =
  | { kind: 'record'; recordId: string; stackUrl?: string }
  | { kind: 'entity'; entityId: string }
  | { kind: 'external'; ns: string; id: string };

Examples:

• A contact type uses { kind: "attachment", label: "avatar", fileId: "..." } as a profile picture.
• A tweet type uses { kind: "relationship", label: "reply-to", target: { kind: "record", recordId: "..." } } to reference another tweet.
• Any record can use { kind: "tag", label: "starred" } for user-defined labels.

parentId is a separate native field (not an Association) because hierarchical containment is fundamental enough to warrant indexing at the library level. Associations are for metadata and cross-references.

Association identity is (kind, label) plus the payload that names a referent — fileId for an attachment, the target for a relationship. An attachment's attachmentRecordId is outside it: the pointer annotates a reference rather than naming one. Identity is what dissociate() matches on, what a second associate() of the same reference lands on, and the primary key every adapter stores an association under.

An element carries only the keys its kind defines, at every depth: the keys in the types above, a relationship target's keys for its own kind, and a permission grantee's for its own (see Access control § Record-level permissions). Any other key is refused with StackBadRequestError (wire: 400) on every write that takes an element — create(), mutate(), associate(), dissociate(), amendAssociations(), grantAccess(), revokeAccess(), amendAccess() — and on a relatedTo filter target. A key from another arm counts as unknown: role on an entity grantee is refused, not ignored. An adapter stores only the keys it has a place for, so an extra key would otherwise answer 200 and then disappear. The same holds for a type-level grant target given to grantType(), revokeType() or listTypeGrants(), and for a _grant record's grantee, which as content is refused with StackValidationError (422).

An association list holds distinct identities. A list naming one identity twice — create()'s associations, or a change set's — is refused with StackValidationError (wire: 400), not collapsed to the last entry. Two entries under one identity describe a state no store can hold, since a store keys them; accepting the list would leave which of the two the record ends up with to whichever adapter is underneath, and would leave a journal entry reporting a prior state twice with no way to say which edit displaced it. Every way of producing such a list is a caller bug, which is why it is refused on the same terms as an empty change set.

Reparenting

parentId is the one native field a write-holder may change after creation, through mutate()'s parentId key — null moves the record to the root. It is a no-bump write that appends a journal entry carrying previousParentId and emits a change event. Every other native field is either create-only (id, createdAt, createdBy, appId), stamped by the write itself (version, updatedAt, updatedBy), or reached through a key or verb carrying its own authority (permissions, unlistedAt, deletedAt, typeId).

Moving a record confers nothing. Containment is not an access-control edge, so nothing is inherited from a container and nothing cascades out of one. A move changes which queries and feeds enumerate the record, and nothing about who may read it.

A parentId a caller names must be a well-formed id naming a record that exists. mutate() and create() check both, refusing a malformed id with StackBadRequestError (wire: 400) and one naming no record with StackConflictError (wire: 409). Format is checked first, so a bad id reports what is wrong with it rather than failing as a lookup that could never match; the same rule as a caller-supplied id, which is why '' is not a parentId any more than it is a record id. Existence closes an asymmetry rather than adding a cost: the reference gate already refuses a non-owner a parent they cannot read, and a parent that isn't there is one nobody can read — only the owner's path skipped it.

This is a front-door check, not an invariant. Deleting a container never touches its children, so a parentId resolving to nothing remains an ordinary state at rest, and consumers must handle one. What the check buys is that a caller cannot mint one: a dangling parent now means a container was removed, rather than that somebody asserted a reference to nothing.

restoreVersion() has no parentId to check. A snapshot carries no containment and a restore settles none, so it is never a caller naming a destination and never a site the front-door check applies to. The one dangling reference a restore can produce is a file reference — see Versioning § Restore semantics.

A record may not become its own ancestor. Every site that adds a containment edge walks the proposed ancestor chain and refuses with StackConflictError (wire: 409) on arriving back at the record: a mutate() naming parentId, and a create() supplying both id and parentId. Those are the only two: a restore adds no edge, so it has no chain to walk. A generated ID names nothing, so an ordinary create cannot close a loop and skips the walk; a caller-supplied one can, since existing records may already point at it — though with the existence check above, a chain leading to a not-yet-created id can only have come from an adapter write beneath Stack.

The walk uses unscoped storage reads, so it sees links the requester cannot: a walk that skipped them would let a cycle be assembled through unreadable records and break the invariant for every reader. The requester learns only that some chain exists between two records they can already reach — read access to the destination having been required first — and integrity is worth that much more than the inference is worth closing.

The walk is bounded at 64 levels, and the bound stops the walk rather than the move. Past the cap it returns and the write proceeds. A chain that long is past what the check can speak to, not evidence of a loop, and refusing it would advertise an invariant core does not maintain — while blocking legitimate moves into a deep tree for good, since nothing shortens the chain. The bound is what keeps each move's cost fixed, and what guarantees the walk terminates on a chain that is already cyclic. Depth itself is unbounded: an ordinary create() under a parent never walks, and moving a subtree checks only the chain above it, so records can and do sit deeper than 64.

The check is read-then-write, so it is advisory under concurrency: two moves racing on opposite ends of one chain can each pass their own check and both land. Closing it properly means the invariant lives in the adapter, where the write is atomic — a graph constraint in the storage contract that every adapter would then implement, which is a decision about where acyclicity lives rather than a local fix. Same posture as DID binding uniqueness.

So acyclicity is a guardrail against the common accident — moving a container into its own descendant — and not a property to build on. It is exact for a single writer within the cap, and nothing beyond that. A consumer that walks parentId MUST carry a visited set or a depth bound of its own; this is a requirement on consumers, not a suggestion. Core never walks the hierarchy, so a cycle costs core nothing — every read, filter and feed is single-level — but a static-site generator derives a page's path from its parentId ancestors (commons § page) and a folder view recurses, and a cycle hangs both. A cycle is always escapable: mutate(id, { parentId: null }) reaches no container, so it is never refused, and moving any member to the root breaks the loop.

A move is undone by a move. No snapshot captures parentId, so restoreVersion() never settles containment — it puts content back and leaves the record where it sits. Undoing a move is another mutate() naming parentId, reading the origin off the journal entry the move appended, and it leaves the same trail every other write leaves: a new entry, not a rewritten one.

Relationship targets

A relationship target's kind names which identifier space its value belongs to. The three are not interchangeable: the same string can be a Record ID in one and a DID in another, and matching across them would make a group roster look like a record reference.

• record — a Record. recordId is unique within one stack only, so a reference to a Record in a different stack carries that stack's stackUrl alongside it. An absent stackUrl means this stack; it is not a wildcard, and a filter that omits it does not match a target that carries one.
• entity — a "who", as a DID. This is what group rosters are made of: membership names identities, which mean the same thing in every stack, rather than the local Records that happen to describe them.
• external — something outside the stack entirely. ns names the scheme that interprets id — "atproto", "activitypub", "email", "url", anything — and is part of the association's identity, so the same id under two namespaces is two associations. Haverstack expresses the reference; the app or adapter interprets it. No protocol is privileged, and nothing in core dereferences one.

A target names exactly one thing, exactly one way. Every part of a target that names something must be a non-empty string, and a target whose kind is outside these three is rejected with StackValidationError — a discriminated union is a compile-time promise, and a Record arriving from a request body or a foreign server has made no such promise. Where absence is meaningful it is the only way to say so: this stack is named by omitting stackUrl, never by sending an empty one, and a whole namespace by omitting an external id. An empty string would claim a name while carrying none, and be stored and matched as though it were absent.

Targets and grantees share one discriminant, kind, and stay distinct by shape. An entity target and an entity grantee are the same value — a DID — so one parsed identifier serves a link, a permission and a grant alike. The other arms keep them apart: a target has record and external, a grantee has group with a required role, and neither union accepts the other's.

The whole target is part of the association's identity, so two relationships differing in any target field are two associations, and dissociate() removes only the one it names exactly.

Which arms a ScopedStack gates on creation, and why the rest are ungated rather than overlooked, is Reference-creation gating.

Types

A Type defines the schema for the content field of a Record. Types are identified by a namespaced, versioned string ID controlled by the app author — the app is the real coordination mechanism between stacks, so Type identity is scoped to the app that defined it.

A type family is every version of one Type, named by its BaseId: the TypeId with its @version suffix stripped. An API that takes only a family — RecordFilter.baseId, migrateAll(baseId) — names it baseId and types it BaseId, and so do the type-level grant verbs, grantType(baseId, …) and revokeType(baseId, …) (see Access control). A family argument takes a BaseId only and refuses a TypeId with StackValidationError, naming the family to pass instead — "com.example.myapp/note@2" names one version; pass the family "com.example.myapp/note". BaseId is a string alias, so the compiler cannot catch the mix-up and the refusal happens at runtime. An argument that means one version takes a TypeId, and nothing accepts both: dropping the suffix would leave a call that reads as naming one version while acting on all of them.

type TypeId = string; // e.g. "com.example.myapp/note@2"
type BaseId = string; // e.g. "com.example.myapp/note"

type ScalarFieldKind =
  | 'string'
  | 'number'
  | 'boolean'
  | 'date'
  | 'text' // Long-form string (e.g. markdown body)
  | 'record-ref' // Reference to another record by ID
  | 'file-ref' // Reference to an attachment file ID (SHA-256 hex)
  | 'enum'; // One of a declared set of strings

type FieldDef =
  | { kind: 'enum'; values: string[]; required?: boolean }
  | { kind: Exclude<ScalarFieldKind, 'enum'>; required?: boolean }
  | { kind: 'array'; items: FieldDef; open?: false; required?: boolean } // recursive
  | { kind: 'array'; open: true; required?: boolean } // elements unvalidated
  | { kind: 'object'; properties: TypeSchema; open?: false; required?: boolean } // recursive
  | { kind: 'object'; open: true; required?: boolean }; // keys unvalidated

type TypeSchema = {
  [fieldName: string]: FieldDef;
};

type StackType = {
  id: TypeId;
  baseId: BaseId; // Derived from id by stripping the version suffix
  version: number; // Incrementing integer
  name: string; // Human-readable label, e.g. "Note"
  schema: TypeSchema;
  schemaHash: string; // SHA-256 of canonical (minified) schema; field names and enum values sorted by UTF-16 code unit
  migratesFrom?: TypeId; // an earlier version of the same family — documents lineage
  createdAt: Date;
};

type DefineTypeOptions = {
  id: TypeId;
  name: string;
  schema: TypeSchema;
  migratesFrom?: TypeId;
};

required: false makes a field optional, not nullable: content fields are never null.

defineType() takes one DefineTypeOptions object, which a type handle is. id and name are both strings, so naming them keeps a call from swapping them silently. baseId, version, schemaHash and createdAt are derived, never supplied.

await stack.defineType({
  id: 'com.example.myapp/note@2',
  name: 'Note',
  schema: { text: { kind: 'text', required: true } },
  migratesFrom: 'com.example.myapp/note@1',
});

Array and object fields are schema-validated on write and reachable by query: a content filter key is a path, and an array along it is matched element-wise (see Filter). open: true declares the container open — a list or object whose interior the schema does not describe, and the one way to store content the schema cannot name. A container declares either its interior or open, never neither: opacity is a claim the schema makes, not something inferred from a missing items/properties, so a schema that forgets to describe its elements is a mistake rather than a silently unchecked field. Query reach is unaffected: a path still walks into an open container, since the query engine reads the content rather than the schema. See Undeclared content fields.

An enum field holds one of the strings its values list names. A value outside the list is a StackValidationError at its path, naming the allowed values. defineType() refuses an enum with missing or empty values, non-string or duplicate entries, values on any other kind, and an enum key on any field, which would otherwise be ignored and silently drop the constraint. The schema is then the one place the allowed values live, so an app's union type ('want' | 'reading' | 'finished') has something to derive from rather than drifting from it. The schema hash sorts the values, so reordering them is not a change.

An enum is its own kind rather than a constraint on string because it promises a reader something a string does not: the value is one of a known set. Every rule keyed on kind — read compatibility, drift, sort order — therefore states what it does with one.

date fields validate against an ISO 8601 shape, not bare Date.parse — YYYY-MM-DD, optionally extended with THH:mm:ss, optional fractional seconds, and an optional Z/numeric-offset suffix. A regex pins the shape; Date.parse then runs as a calendar sanity check on top of it (catching e.g. an invalid month). Date.parse alone also accepts engine-dependent, non-ISO formats ("March 1 2020"), which would let cross-runtime stacks disagree about what's valid and produce non-canonical stored values.

A date field holds a string, and a Date is refused. The error says what to pass instead: date.toISOString() for a moment in time, or a "YYYY-MM-DD" string for a calendar day. A Date is a moment, a date field often means a calendar day, and core cannot tell which the app meant, so it makes the app choose rather than converting. Converting would also store a time-zone-dependent day, and a Date written in could not be read back as one: content is JSON, and the stored form is a string. Wherever validation names the type it got, a Date is named a Date.

Content is plain JSON at every depth, open containers included. Only null, booleans, finite numbers, strings, arrays and plain objects (prototype Object.prototype or null) are accepted; anything else — a Date, NaN, Infinity, undefined inside an open value, a class instance, a Map, a function, a circular reference — is a StackValidationError at its path. open says the schema does not describe the shape, not that the content may stop being JSON; without the check, adapters that store a value as given and adapters that serialize it would read back different things. Nesting is capped at 32 levels, counted from the top-level field and the same inside an open container as in declared ones: a deeper value is a StackValidationError at the path where it crosses the cap, so no write buys an unbounded validation walk.

file-ref fields are real references, not just strings that look like fileIds. A file-ref value must be a well-formed fileId (SHA-256 hex) — validated at write time, though referential existence is not (the same stance as record-ref; upload-before-associate flows make strictness hostile). What file-ref buys over a plain string field holding the same value: the referencesFileId query filter, deleteAttachment()'s reference check, and attachment-access conveyance under ScopedStack all treat a top-level file-ref field as a real reference to the file, the same way an attachment Association is. An app that stores a fileId in a plain string field keeps working but gets none of that — no delete protection, no access conveyance, no garbage-collection protection. Only top-level scalar file-ref fields are indexed this way; a file-ref nested in an array or object is validated, and reachable by a content filter path, but not indexed as a reference — so it gets no delete protection, access conveyance or GC protection. Indexing a content field, whether as a reference or for sorting, reaches top-level scalars only, while query reach extends to depth: they are separate mechanisms, and one rule covers both.

A schema is checked for being a schema before anything reads it as one. defineType() refuses a malformed one with StackValidationError (422), naming each field and what is wrong with it: a definition that isn't an object, one naming no kind or a kind that isn't a field kind, a non-boolean required/open, a container declaring neither its interior nor open, and a container declaring both (one of the two would have to be ignored, and nothing says which). The check recurses through properties and items, and reports every malformed field rather than the first.

This matters because defineType() takes a TypeSchema but a schema arriving at POST /types is parsed JSON that no compiler has seen, so every one of those shapes is reachable. Unchecked, they fail in two unhelpful ways: a container with no interior throws out of schema hashing rather than reporting anything, and an unrecognized kind defines a field whose every write fails against an expectation the schema never stated — sending the caller looking through their content for a bug that is in their Type. It is the argument Content field names makes for declared names, applied to the definition as a whole: a schema is a promise, and one nothing can satisfy is worth catching where it is written.

Type identity: two Types are the same if their id matches (including version). Two stacks running the same app will have the same Type IDs and can rely on that for interop.

Schema drift detection

defineType() on an id that already has a stored Type is checked against it, rather than silently replacing it — same schemaHash as stored is unambiguously the same schema (a different schemaHash for the same typeId with no version bump is the exact corruption schemaHash exists to catch):

• Identical schema (schemaHash matches) — a no-op; the stored Type is returned unchanged, createdAt untouched. Calling defineType() for every Type at every app startup is therefore cheap, not a rewrite each time.
• Identical schema, different name — always persists (display metadata, not schema), createdAt still preserved.
• Different schema — legal only if the change is a pure additive-in-place evolution: new optional fields only, recursively into object properties and array items; nothing removed, no field's kind changed, no field's required flipped in either direction, no container opened or closed, and no enum that loses values. One kind change is additive: enum to string, which accepts strictly more; string to enum narrows and needs a new version, as does enum to text, for the reason string to text does (below). Adding values to an enum stays in place. An illegal change throws StackSchemaDriftError (wire: 409, code schema_drift) naming each violation — the remedy is always a new version (defineType({ id: '...@n+1', ... }) plus a migration() passed to Stack.open()), never redefining the same id in place.

POST /types (see Wire format § Types) applies the same check server-side, so the wire path can't silently replace a Type either.

Type cache

Stack caches every Type it fetches or defines in memory, keyed by versioned id — populated by getType(), defineType(), and listTypes(). Since a Type's schema is immutable once defined (a legal schema change always gets a new id via a version bump), the cache is never invalidated, only added to — create(), mutate(), and restoreVersion() validate against the cached entry instead of re-fetching the Type on every write. Over the API adapter this removes a GET /types/:id round trip from every write after a type's first use in the process. listTypes() refreshes the cache wholesale, which is the explicit way to pick up a name-only rename made by another writer — the one field defineType() permits changing without a version bump.

Type compatibility

Structural/duck-typed — a Type is read-compatible with a required schema if, for every required field, the candidate declares that same field as required, at a read-compatible kind. Array and object fields recurse: their items/properties must themselves be read-compatible. This licenses consuming Records, not writing them — a consumer writing through a "compatible" view still has to validate against the candidate's full schema (its other required fields, which compatibility checking never inspects).

Two distinct relations, easy to conflate: schema drift detection (above) answers "may this schema replace that one under the same id?" — evolution legality. Type compatibility answers "may a consumer expecting this shape read Records of that Type?" — read compatibility. They deliberately disagree on text/string: read-compatible (both are strings at the value level) but not evolution-legal — a stored kind: 'string' field silently becoming kind: 'text' is exactly the kind of change a version bump should surface, even though every existing reader could still consume the value. They disagree on enum too: a required string reads an enum, and enum to string is evolution-legal, but enum to text is not.

A field's kind is read-compatible with a required kind per this table (row = required kind, columns = candidate kinds accepted):

required →	string	text	enum	number	boolean	date	record-ref	file-ref
string	✓	✓	✓
text	✓	✓	✓
enum			⊆
number				✓
boolean					✓
date						✓
record-ref							✓
file-ref								✓

string and text are mutually read-compatible — the distinction is presentation/indexing intent — and each also accepts an enum. A required enum takes the subset rule below. Every other kind requires an exact match; notably date is not compatible with string, since date carries a parse/validity guarantee a plain string doesn't.

An enum reads as a string, and a required enum accepts only a narrower enum (⊆ above): a candidate enum whose values are all in the required list. A consumer expecting an enum has handled exactly the values it lists, so a string or text candidate, which can hold anything, is refused, as is an enum listing a value the consumer does not. The check is as of the Type it is given: an enum may gain values in place, or become a plain string, so a candidate compatible today can later hold a value the consumer does not list. A consumer handles such a value, as a collection does.

Apps that care about semantics filter by exact typeId; apps that want flexibility (e.g. "any Type with a required text field") use isCompatible() — see packages/core/src/schema.ts for the authoritative implementation and packages/core/tests/schema.test.ts for its behavior under nesting and the string/text equivalence.

System types

Reserved, library-defined types: _config@1 (Stack initialization), _entity@1, _app@1, _group@1 (Identity), _grant@1 (Access control), _attachment@1 (Attachments), and _install@1 (App installs). System types follow the same versioned ID format as user-defined types and can evolve using the same migration mechanism. All seven are pre-seeded when a Stack is opened with Stack.open() — always available without any setup by the caller.

Type migrations

A Type's defining app is the only serious writer of its own types, so migration is explicit and owner-driven, not a side effect of a read or an unrelated write. Disk state changes version only via a deliberate migrateAll() pass or a per-record commitMigration() call — the invariant is that a Record's typeId never moves except through one of these two, so query({ filter: { typeId } }) and grants targeting a type never silently miss not-yet-migrated records.

An app builds one migration per step between adjacent Type versions with migration(from, to, fn) and passes them all to Stack.open(). The library composes them into a full migration graph, so an app that only knows about v3 doesn't need to know that v1 ever existed.

const noteV1ToV2 = migration(NoteV1, NoteV2, (content) => ({ ...content, title: '' }));
const noteV2ToV3 = migration(NoteV2, NoteV3, (content) => ({ ...content, pinned: false }));

const stack = await Stack.open(adapter, { migrations: [noteV1ToV2, noteV2ToV3] });

migration() takes two type handles, so fn is checked as StoredContentOf the first to StoredContentOf the second: a step that drops a required field does not compile. A stored enum may hold a value the source handle does not list, so the step carries it through rather than losing it, and a lookup keyed on only the listed values does not compile. A value the target does not accept is refused by its runtime validation when migrateAll() writes it. Within a family it refuses a to whose migratesFrom is not from.id, so a step cannot be paired with the wrong handle.

Lineage stays inside a family; a migration can leave it. migratesFrom names an earlier version of the same family, and typeHandle() and defineType() refuse anything else with StackBadRequestError. A step whose from and to are in different families has no lineage to check, so migration() builds it from the two handles alone. This is how an app's own type moves into a shared one — a commons type, say — without the shared handle naming every family that has ever migrated into it. A migration is a separate value from either handle: a handle is data a manifest carries over the wire, and may be shared by apps that do not run its family's migrations.

The migration registry is per-stack-instance and lives in memory — different stacks can be at different migration states without interfering. Nothing about it is stored, so every Stack.open() is passed the app's migrations, including one over APIAdapter. The registry is complete before the first read, so no read or migrateAll() can run against a partial chain. Stack.open() refuses with StackMigrationError a set it could not walk: two migrations with the same from, a step whose from or to is not a versioned TypeId, a step to the same or an earlier version of its own family, or steps that form a cycle. Migrations are not an install step and not part of an app manifest.

What the library does with registered migrations:

• get() and query() return Records exactly as stored — their own typeId and content, with no implicit migration. This is what makes query() a reliable way to find every record of a family regardless of migration state.

• presentAt: 'latest' — an explicit opt-in on both get() and query() that applies the registered migration chain in memory before returning. Nothing is written to disk; this is a read-time convenience, never a persistence mechanism. It is a property of the app instance that registered the chain, so it never travels: a server rejects a request carrying presentAt rather than dropping it. Throws StackMigrationError when a matched Record's version can't be reconciled with what this app instance has registered (see stale-writer behavior below).

• A content patch never migrates. mutate() validates the merged content against the Record's own current stored Type — never the latest — and writes back at the same typeId. An unrelated content edit can never fold an invisible schema rewrite into the same version-history entry. This is also why content read through presentAt: 'latest' is not writable back wholesale, and why the content key is a patch. A collection's content write to a Record at an older version commits the migration as a write of its own first, then applies the patch.

• Path composition — migrations between adjacent versions are automatically chained (v1→v2→v3), so apps only ever register one step at a time.

• migrateAll("com.example.myapp/note") eagerly commits all pending migrations for a type family in one deliberate pass, taking each Record to the end of the registered chain rather than to a version the caller names — which is why it takes a BaseId and refuses a versioned TypeId — call it at app startup, or after a schema change. It sweeps soft-deleted and unlisted Records (includeDeleted/includeUnlisted are not caller options in either direction — see Deletion and Unlisted records); the one narrower sweep is { sweep: 'listed' }, for an installed app that can reach neither (see App installs § Migrating an installed app's types). It validates each migrated result against the target Type's schema before writing, and aborts immediately on the first validation failure (a buggy migration function is a bug to surface, not to paper over by skipping the offending records) — anything already committed earlier in the pass stays committed. Previous content is snapshotted to version history before each write.

• commitMigration(id, toTypeId, content) is the single-record counterpart, changing one Record's typeId and content together in one step. Unlike migrateAll(), content here is supplied by the caller rather than produced by a registered Migration function — the client-side app that owns toTypeId computes it, and the library validates it against toTypeId's schema exactly as create()/mutate() validate against a schema. This is what backs the wire's POST /records/:id/migrate (see Wire format). Under ScopedStack it is owner-acting-alone, matching migrateAll()'s own absence from StackClient — no grant or record-level write substitutes for it (see Access control) — except for an installed app within its own families (see App installs). Previous content and typeId are snapshotted to version history first, same as migrateAll().

  Because content is a full replacement written under a new typeId, a migration commit is create-shaped at the destination and update-shaped over the Record as it stands, and owes both sets of integrity checks. DID bindings are held to immutability across the union of the two families' binding fields — a card can neither shed its did by migrating out of _entity/_app nor pick one up on the way in — and to uniqueness in the destination family (see Identity § DID bindings). An _attachment@1 Record's fileId, mimeType and size stay immutable, and a Record arriving from outside that family is held to the same mimeType-establishment check create() applies. Migrating into _group is refused outright: a group's admin roster entry is stamped at creation and a migration cannot stamp one, so it would produce a group nobody but the owner can manage — version-to-version migration within _group stays open and carries the existing roster with it.

  migrateAll() applies these same checks, on the same shared write path. That a Migration function is app code rather than a request body is not a trust boundary here: the app calling commitMigration() is the same app that passed the function to Stack.open(), and neither is entitled to move a DID binding or repoint an attachment. A path can itself cross type families, which is precisely what these checks are about. A migration function that would violate one aborts the pass like any other validation failure.

Stale-writer behavior. A Record whose version this app instance can't reconcile — older than what it was passed and not bridged by a migration path, or newer than anything it has ever defineType()'d — is an explicit error (StackMigrationError) under presentAt: 'latest', not a silent pass-through. This covers both directions of "the same app at two versions" meeting via a shared stack. Reading the Record as stored (the default, no presentAt) always succeeds regardless — the stale-writer signal only fires when the app explicitly asks for the migrated view and the library can't honestly provide one.

Type handles

A schema written once as a literal gives the compiler everything it needs to type content. typeHandle({ id, name, schema, migratesFrom }) returns a plain, frozen value carrying the fields defineType() takes — the TypeId (id), the display name, the schema and, for a version after the first, migratesFrom — plus its family (baseId). migratesFrom may be given as the previous version's handle or its TypeId, and is stored as the TypeId. ContentOf<S> derives the content type a write supplies, and StoredContentOf<S> the one a read returns. A required field is present in ContentOf; every other field is optional. A collection derives the rest from its content type C: StoredContent<C> is what its reads return, the same as StoredContentOf<S>, and ContentPatch<C> is the contentPatch it takes, which makes every field optional and allows null only on a field that is not required, since a required field can be replaced but not removed.

A handle is a DefineTypeOptions, so defineType(handle) and a manifest's types take it as written, and name and migratesFrom are written once. It carries no migration function, which keeps it data: see Type migrations.

A handle names exactly one version. A call that means the whole family takes handle.baseId, never handle.id.

client.collection(handle) returns a Collection typed by the handle's content, ContentOf<S>: every verb that addresses records of the family, each with one signature that takes and returns the handle's types — get, query and subscribe; create, mutate, patchContent, associate, dissociate, amendAssociations, grantAccess, revokeAccess, amendAccess, delete, undelete and restoreVersion. History (getVersions(), getVersion(), getJournal()) spans versions by nature and stays on StackClient, which is otherwise unchanged and remains the API for a Type known only at runtime. A collection is built on the public StackClient verbs, so it reaches nothing its client could not. The outer types (Collection<C>, CollectionRecord<C>, CollectionPage<C>, CollectionQuery<C>, CollectionChange<C>) never flatten C, so an app that names its content with an interface sees that name.

collection() refuses a handle whose migratesFrom has no migration passed to Stack.open(), with StackMigrationError naming the pair. It runs in the app's own process, which has to hold that migration; defineType() cannot ask, since an owner applying another app's manifest defines types whose migrations live in that app.

The scope is the handle's whole family, for reads and subscriptions alike. Each record is read as stored and migrated in memory to the handle's version, one record at a time, through the opening Stack's migrations — not through presentAt: 'latest', which migrates to the newest version the instance has defined and throws for a whole page. A record of another family is not in the collection: get() answers null for it, as for a missing record or a tombstone. A collection reads live records only; includeDeleted is not offered and is refused at runtime, so a tombstone, whose content is {} under ScopedStack, is never typed as the handle's content.

A read types an enum field as its listed values or any other string ('want' | 'reading' | 'finished' | UnlistedValue). An enum may gain values, or become a plain string, within a version (additive evolution), so a reader older than the writer can meet any string there. Records are shared across apps that upgrade independently, so a closed union would promise what no handle can check; the reader handles a value it does not list, as a default branch does. A write takes only the values the handle lists, in ContentOf and ContentPatch; the stored schema's runtime validation decides what lands.

A misfit is a record of the family the handle cannot represent: one stored at a version newer than the handle's (reason: 'newer-version'), which cannot be migrated down. A misfit is reported, never thrown for a page:

• query() returns { records, misfits, cursor }, and skipped on the first page. Misfits do not count as records, so a page can hold fewer than limit records with a cursor still set: cursor === null ends the results, a short page does not. The page and its cursor survive one record the app cannot read.
• get() is a read the app asked for by ID, so it throws MisfitError, carrying the reason and the record as stored.
• subscribe() delivers a change whose record does not fit with misfit in place of record.

A record at an older version that no passed migration carries to the handle's is not a misfit: that is the app's setup, and a read of it throws StackMigrationError, as it throws whatever a failing migration function throws. A subscription cannot throw at its emitter, so such a change arrives without its record, as it does wherever the emitter cannot supply one, and get() says why. MisfitError is raised only in the app's process and has no wire representation, so it sits outside the taxonomy.

A subscription covers the family. A change's record, when present, is migrated to the handle's version; the change's own typeId stays the stored one, as RecordChange.typeId documents. A change whose record has deletedAt set carries neither record nor misfit, on either surface, and neither does a change that migrates the record out of the family, which the filter matches by its previous typeId. The change filter takes every key but typeId and baseId.

A content filter or sort reaches only records stored at the handle's version. The adapter matches and orders content as stored, before any record is migrated, and a migration is an opaque function, so a predicate written in the handle's shape cannot be translated into an older version's. Run across the family, a field a migration renamed, derived or remapped would silently miss the older records or misplace them. So a query() whose filter.content or filter.contentPresent is non-empty, or whose sort is a contentField, reads only records whose typeId is the handle's, even for a field no migration touched, since core cannot tell which fields those are. The untyped query({ filter: { baseId, content } }) still matches across the family, returning records as stored. The first page of such a query sets skipped.older or skipped.newer when records at an older or newer version match the rest of the query: the results may then be missing records the filter was meant to find. A later page carries no skipped, since the flags depend on the query, not the page. Each is one limit: 1 read, run beside the first page's own, and a scoped read that comes back empty with a cursor still set counts as a match rather than reading on, so a flag can be set when nothing was skipped but is never clear when something was. Mixed versions are a lasting state, not a backlog, since an app at an older version may keep writing its own, so the app decides what skipped.older means to it; migrateAll() clears it only until the next such write. skipped.newer says the app is behind, which an unfiltered page reports as misfits; a narrowed page carries none. A query with no content keys reads the whole family. The keys are typed against the handle's version: filter.content and filter.contentPresent take the content's paths (ContentPathOf<C>), through declared objects and arrays, with any suffix below an open object or array, and a value of the field's type as a write takes it (an array's element type, since matching is element-wise) or null. A path reaching an object takes only null, since an adapter compares the value a path reaches and an object never equals one; a value below an open node is any scalar. Typing values as writes take them means an enum value the handle does not list is filtered for with a cast, the price of refusing a misspelled one; sort.contentField takes only a top-level scalar (SortableFieldOf<C>). Path typing stops six segments deep and accepts any suffix below that, a limit of the compiler, not of the 32-segment path. query() takes neither typeId nor presentAt, which would change which records a page holds or the shape they arrive in, and drops either if an untyped caller passes it; subscribe() drops a filter's typeId the same way.

Every verb that returns a record returns it as get() would. The write already returned the record, so narrowing it costs no round trip, and a restoreVersion() to a pre-migration snapshot reads back migrated, like any older record. A write is checked against the read it already makes, before anything lands: a record outside the family is refused with StackNotFoundError, the answer get() gives, and so is a restoreVersion() to a snapshot taken before the record migrated into the family, which is not a version of a record in the collection. A content write to a record stored at a newer version is refused with MisfitError before anything is written, since its patch would be validated against a schema the handle does not know. Any other write that leaves a misfit lands, then throws MisfitError carrying the stored record, so the app knows the write landed. A deleted record, or a stale ifVersion, is refused first, as the untyped write refuses it.

A content write to a record stored at an older version migrates it first, as its own write. A patch is validated against the record's stored Type, and a content patch never migrates, so the collection commits the migration with commitMigration() and then applies the patch: two versions and two journal entries, migrate then patch — the history migrateAll() followed by the patch would leave. The caller's ifVersion fences the migration, since that is the state they read, and the patch is fenced to the version the migration produced, so a write landing in between fails the patch with StackVersionConflictError; the migration that landed is what migrateAll() would have done anyway. The same holds when the patch fails for any other reason, so a caller retrying after a failure reads the record again first: the ifVersion it held now names the state before the migration. A caller who may not commit the migration — anyone but the owner and an installed app within its own families — is refused with StackPermissionError naming the stored version, and nothing is written. The usual way a record reaches an older version is restoring a pre-migration snapshot, which keeps that snapshot's typeId, so a revert followed by an edit is an ordinary sequence.

The derived types are a convenience over runtime validation, which stays the guarantee. A Type known only at runtime has no static shape and stays Record<string, unknown>.

Additive evolution within a version

Not every schema change needs a version bump. Additive-in-place changes — new optional fields only — can be added to a Type's schema without minting a new version, and are the default path for evolving a type family:

• Readers ignore fields they don't know about — validation runs on write, so a consumer holding an older idea of the type reads a Record carrying a field it was never taught about without complaint.
• Writers preserve fields they don't touch — a content patch retains any field the caller didn't name.

This is what makes duck-typed cross-app consumption (isCompatible(), above) work in practice: most evolution needs no coordination at all, and consumers that were never taught about a field simply don't see it. Adding the field to the schema is what licenses writing it — see Undeclared content fields — and that is a defineType() call, not a version bump.

Widening an enum (adding values, or turning it into a plain string) is additive for the same reason a new optional field is: the schema accepts strictly more. Narrowing one is not, and needs a version bump. See Schema drift detection.

A version bump is a consolidation point, warranted when:

• a field becomes required (paired with a migration that backfills a real value for existing records),
• a field's meaning changes, or
• the shape is restructured.

A schema accumulating many optional fields is a named smell that a consolidating bump is due — but the bump itself stays rare and semantic ("@2 means dueDate is now guaranteed"), not a changelog entry for every field ever added. Bumping per addition costs a full-table rewrite per field and version-number noise, and — since records only reach a new version when the owning app next runs migrateAll() — doesn't even deliver per-record schema exactness in the interim; consumers face mixed-version data either way and need baseId queries plus duck typing regardless.

The boundary between "accept in place" and "bump the version" is exactly what defineType()'s schema drift detection mechanically enforces — a diff, not the hash alone, since the hash necessarily changes on any legal additive diff too.

Not yet implemented: validation of migration function output against the target schema at registration time (write-time validation, in migrateAll(), is the enforced backstop today).

Undeclared content fields

A content field the Record's Type does not declare is rejected with StackValidationError (422) — on create(), in a content patch, and on commitMigration() against the destination Type — naming the field and the path it sits at. The schema is the Record's shape, and it holds at every depth: an undeclared key inside a declared object, or inside an object an array declares as its items, is refused with its full path (address.postcode, emails[1].label).

A key outside the schema is a typo, a stale writer, or a caller reaching for something that is not content at all, and accepting it would make all three look like a write that worked. The last is the load-bearing case: content is its own namespace, so a parentId in a patch is a content field of that name and never the native one, and it would sit beside a native parentId holding something else with nothing downstream reading it. Refusing says which of the two the caller reached for, while they can still choose the other verb — the argument reserved content keys makes for __proto__, applied to every name a schema doesn't claim.

This is a rule about the schema, not about the names. A Type is free to declare parentId, version or createdAt as content — a bookmark's parentId naming the upstream Record it was clipped from is an ordinary field — and once declared it is patched like any other. Nothing is reserved by resemblance to a native field.

An object or array field declared open: true is not validated inside: the schema places a container there and says nothing about its interior. That is the deliberate way to store a shape a schema cannot describe — an imported blob, a heterogeneous list, a payload whose keys are data. An open container is still held to its own kind (an open object refuses an array, an open array refuses an object), which is why open is a flag on the container kinds rather than a "any JSON here" kind of its own — a Type that means "a list, contents unspecified" can still say so. It is exempt from the schema only: content field names are still checked at every depth inside one, since that rule is about what a filter path can address rather than about what the Type promised.

Open and declared are different shapes, not degrees of the same one: a container that accepts anything and one that accepts a named set hash differently, and changing a Type from one to the other is schema drift in both directions — closing an open container refuses content it used to accept, and opening a declared one accepts content it used to refuse. Neither is additive. For the same reason an open container satisfies isCompatible() only where the required side asks nothing of its interior: a bag promises a consumer nothing to read.

This is a Stack invariant, so every adapter and ScopedStack inherit it, and a server mapping request bodies onto storage directly has to apply it itself — see Wire format § Records.

Reserved content keys

__proto__, constructor, and prototype are rejected as top-level content keys — on create() and in a content patch alike — with StackValidationError (422), independently of whether the Type declares them.

They name JavaScript's object machinery rather than a field, so whether one survives a write would depend on how the code handling it was written rather than on the data. Assigning __proto__ with obj[key] = value — the ordinary idiom, and the obvious way to apply a merge patch — reaches the prototype setter and silently drops the write, while JSON.parse and a write that defines the key store it as an ordinary property. constructor reads as present on every plain object, whether a Record carries it or not. Refusing all three means every implementation, core's or a foreign one, gives the same answer, and says so out loud instead of accepting a write that might quietly vanish.

This is a Stack invariant, not an adapter or server concern — it holds for every backend and for ScopedStack, which delegates (the layering of System types and _config's protections). Nested occurrences are not rejected: they survive the JSON round trip as inert own properties, since JSON.parse creates __proto__ as a data property rather than invoking the setter — including inside an open container, which is exempt from the schema and not from this.

A schema may not declare one of the three as a top-level field name either — defineType() refuses it with StackValidationError (422), the same answer POST /types gives. A declaration cannot license what the write rule refuses, so accepting one would define a field no Record could ever carry; where the declaration is required, it would define a Type no Record could satisfy at all, since supplying the field is refused as a reserved key and omitting it is refused as a missing required field. This is the argument content field names makes for declared names too — a schema is a promise that a field is meaningful — and it applies at exactly the scope the write rule does: a nested declaration names a field a Record can actually carry, so it is left alone.

Undefined values in a patch

A content patch keeps an omitted field at its current value and removes one set to null. undefined is neither, and there is no third meaning left for it to carry — so a top-level patch key whose value is undefined is rejected with StackValidationError (422). This is a contentPatch rule only: create() and commitMigration() take a whole content object, where null and undefined are both absent and the schema's own required-field check speaks to the result.

It cannot arrive over the wire — JSON has no undefined, and JSON.stringify omits the key rather than emitting one — so every occurrence is an in-process caller spreading a partial object, meaning either "leave this alone" or "remove this" and spelling neither. Accepting it would resolve the ambiguity twice over, differently each time: storage drops the key on serialization, landing on the first, while the checks that ask whether a patch names a field land on the second. Those checks are load-bearing — binding immutability, attachment field immutability, and ScopedStack's owner-only fence on _app bindings all read the key as a claim on the field. A write-holder patching an _app card with { did: undefined } would be refused for repointing a DID it never sent, and the refusal would be a permission error naming a field the caller did not set.

This is a Stack invariant, so every adapter inherits it. ScopedStack.mutate() additionally applies it ahead of its own binding fences, so a patch carrying undefined is a validation error for every requester rather than a validation error for the owner and a permission refusal for everyone else.

Absent content fields

A content field is never null. The schema expresses that a field is optional, not that it is nullable, and a patch already uses null to mean "remove" — one value cannot be both removable and settable. So null means absent on every write path: create() and commitMigration() drop top-level fields set to null or undefined before storing, and a patch with null removes the field. Reads never return null for a content field, and create() returns the content as stored.

The rule recurses into declared nested objects, including those inside declared arrays, and stops at open objects and arrays, whose interior is opaque to core and keeps any nulls it holds. An app that needs "explicitly none" as distinct from "never set" says so in the schema, for example with an enum value like 'declined'.

Adapters give undefined the same meaning: a field set to undefined is stored as no key at all.

Content field names

A content filter key is a dot-separated path (see Filter), so a field named emails.value and a path reaching value inside emails would be the same string asking two different questions. The ambiguity is removed from the field name rather than from the path: a content field name may not contain ., [, ], $, ", *, or # — rejected with StackValidationError (422) on create(), in a content patch, and on commitMigration().

This holds at every depth, unlike the three reserved keys above: a nested field named b.c makes the path a.b.c ambiguous exactly as a top-level a.b does. The check walks the content itself rather than the schema, so it reaches inside open containers — precisely the subtrees no schema promised anything about — bounded by the same nesting depth validation already applies.

defineType() applies the same rule to declared field names, recursively through object properties and array items, with the same error. A schema is a promise that a field is meaningful; declaring one no filter could ever name breaks that promise at definition time rather than at query time.

The reserved set is wider than what SQLite's JSON path grammar treats as syntax today (., [, ], $, "). * and # are held back against a path grammar that later grows a wildcard or a last-element form. Reserving a character costs nothing while no record contains one and costs every stored record afterward, so the choice is deliberately made early and wide.

Field names are refused rather than escaped. An escape convention — a filter key of emails\.value meaning the literal field — fails silently in the one case that matters: app code building a key from a variable field name forgets to escape and gets a different question answered, with no error anywhere. A write-time rule fails loudly, at the moment a caller can still choose another name.

Queries

Queries are expressed as a StackQuery object passed to stack.query(). All adapters support the full query shape; performance guarantees differ.

Filter

type RecordFilter = {
  // Native fields
  typeId?: TypeId | TypeId[];
  baseId?: BaseId | BaseId[]; // matches every version of a type family
  parentId?: RecordId | null; // null = root records only
  appId?: AppId | AppId[];
  createdBy?: { subjectId?: EntityId | EntityId[]; principalId?: EntityId | EntityId[] }; // the author, never the latest actor
  createdAt?: DateRange;
  updatedAt?: DateRange;

  // Association filters
  tags?: string[]; // records that have ALL of these tags
  attachment?: { label: string; fileId?: FileId } | { label?: string; fileId: FileId };
  relatedTo?:
    | { label: string; target?: RelationshipTargetPattern }
    | { label?: string; target: RelationshipTargetPattern };
  referencesFileId?: FileId; // records that reference this file, via an `attachment` Association or a top-level `file-ref` content field

  // Content fields (exact match; the key is a dot-separated path)
  content?: { [key: string]: unknown };
  contentPresent?: string[]; // paths that must hold a value

  // Full-text search (capability varies by adapter)
  search?: string;

  // Lifecycle states excluded by default
  includeDeleted?: boolean; // see Versioning & deletion; get() takes the same option
  includeUnlisted?: boolean; // owner-only under ScopedStack — see Unlisted records
};

type DateRange = {
  before?: Date;
  after?: Date;
};

relatedTo names a label, a target, or both — never neither, and a filter naming neither is rejected with StackBadRequestError rather than widening to every Record carrying a relationship. Its target follows the same naming rules as a stored one (above), checked the same way. A RelationshipTargetPattern is the association's own target shape with the parts a query may leave open. Each half is a pattern: a bare label matches every target under it; an external target with no id matches the whole namespace, which is how a syndication tool asks what it has already published. A record target with no stackUrl matches only local targets — absence names this stack rather than acting as a wildcard, so a Record referenced in someone else's stack is reachable only by naming that stack. Matching is exact within a target kind and never across kinds.

attachment is shaped like relatedTo: a label, a fileId, or both — never neither. Both halves together match one association, so { label: 'cover', fileId: F } finds a Record whose cover is F, not one that has some cover and some other reference to F. It asks about associations only. referencesFileId asks the wider question — does anything in this Record point at the file — and also counts a top-level file-ref content field. That is the question deleteAttachment()'s reference check and garbage collection ask, and it cannot live inside attachment: a file-ref field has no label, so combined with one it would match under some filters and not others.

"Carries any relationship at all" is deliberately not expressible. tags and attachment have no match-any form either, so a relationship one would be the odd exception rather than a missing convenience — and a filter that can encode to nothing is a filter that can silently widen a query when it crosses the wire. The type refuses the empty filter rather than defining it, and Stack.query() refuses it again at runtime, where a filter decoded from query parameters is a plain object the type never saw.

contentPresent asks whether a path holds a value at all, which a filter value cannot: a value matches what is there, never whether anything is. It lists paths, all of which must hold a value — an intersection, like tags — and an empty list filters nothing, like an empty tags. A path holds a value when at least one non-null value is reachable at it, so it reads an array element-wise exactly as a content filter does: { contentPresent: ['emails.value'] } matches a contact with an address in any of its emails, and a path reaching only nulls, an empty array, or nothing at all holds no value.

contentPresent and a null content filter answer opposite questions but are not strict complements where a path is multi-valued: tags: [null, 'x'] satisfies "something there is null" and "something there is present" alike. That falls out of element-wise matching rather than being a special case, and the alternative — making one of them quantify over every element while the other quantifies over some — would make the pair harder to reason about than the overlap does.

The motivating query is a listing: an article's publishedAt is optional, so drafts are { content: { publishedAt: null } } and published articles are { contentPresent: ['publishedAt'] }. Neither is a value match, and a redundant draft boolean beside the date — two fields that can disagree — is what a type would otherwise have to carry.

Nested content paths

A content filter key is a path, its segments separated by .: { content: { 'emails.value': 'ada@example.com' } } asks which records hold that value at value inside emails. Field names cannot contain . (see Content field names), so a key never means both a path and a literal name, and no escape convention is needed to tell them apart.

An array along the path is matched element-wise. contact@1 stores emails as an array of { value, label }, and the question that motivates paths at all — which contact holds this address — is that shape. Traversal and containment are therefore one feature rather than two: a segment applies to each element of an array it meets, and the final comparison does the same, so { content: { tags: 'starred' } } matches a record whose tags array contains 'starred'. A record storing a scalar where another stores an array is matched by the same filter either way.

A path that descends through a scalar, or through a field that isn't there, reaches no value and matches nothing — it is not an error. A path may be at most 32 segments; longer is StackBadRequestError (400), as is an empty segment (a..b) or a segment containing a reserved character. Those are structural faults in the request, refused rather than answered with an empty result, on the same reasoning as a malformed cursor.

A segment is a value, never syntax. An adapter resolving a path MUST carry each segment as a bound parameter matched against a key, rather than assembling it into a path expression — so no field name a write would accept can be reinterpreted as syntax, and no key can make the statement itself malformed. The 32-segment cap is what keeps that statement inside the engine's own limits: a SQLite adapter walks a segment with two json_each joins, and 32 segments is the longest path that fits SQLite's 64-table join limit. The cap and the generated shape move together — a longer path is refused because it could not be executed, not merely because it is unusual.

Multi-segment keys need the 'path' rung of filter.content, where a single-segment key needs only 'field' — see Capability-gated filters, and Adapter capabilities for why the reach is one ordered value rather than a flag per rung.

Nested fields are not indexed, and depth multiplies cost. A path filter is an unindexed walk of every candidate record's JSON: the grammar is bounded, the execution time is not. It is a harsher bucket than full-text search, which at least runs against an index — here each segment fans out across every element of an array it meets, so a deep path over records holding large arrays costs the product of those widths. On a personal stack the only session it slows is the caller's own; a server serving many requesters owes the bound described in Wire format § Bounding query cost, and should treat path depth as an input worth limiting below the cap.

A content filter value of null means "no value at the path, or a value that is null" — not "match nothing." Plain equality (SQL = NULL, or JS === against a possibly-absent key) is never true for a missing field, which would make { content: { x: null } } silently return an empty result. Every adapter, including test doubles, matches a record whose path reaches nothing and one that stores a literal null alike, since from the caller's side both mean "no value here." A missing intermediate therefore matches too: { content: { 'address.city': null } } matches a record with no address at all, one whose address has no city, and one storing city: null. So does an empty array along the path, which reaches no value by the same reading.

baseId matches every version of a type family — resolved against registered Types (via listTypes()), not string-parsed from typeId, so it works regardless of which versions happen to exist. This is what keeps typeId-filtered queries from silently missing not-yet-migrated older-version records under explicit, owner-driven migration: filter by baseId to see the whole family, or typeId for an exact version. Given both, they intersect. Stack.query() resolves baseId client-side before dispatching to the adapter — adapters and the wire protocol only ever see a concrete typeId set, and a server rejects a request carrying baseId rather than dropping it. An unknown baseId returns an empty result set rather than throwing, but a baseId carrying an @version suffix is refused with StackValidationError — it names one version, which is typeId's job, and resolving it to an empty family would return nothing without saying why. ChangeFilter.baseId follows the same rule.

A query hides exactly three things by default, and nothing else:

• Soft-deleted Records, which includeDeleted opts back in. get() hides them too — get(id) on a tombstone answers null — and takes the same option. See Deletion.
• Unlisted Records, which includeUnlisted opts back in — under ScopedStack, for the owner acting alone only. See Unlisted records.
• _config, always (below).

So an unfiltered query() returns every Record the caller can read, from every app, system types such as _entity@1 and _grant@1 included. An app filters by typeId, baseId or appId to get its own. System Records are not hidden because anything that must see the whole stack — a reference check before deleting a file, a backup or export — would otherwise have to opt back in, and a missed opt-out would lose data silently.

By default, query() (like get()) returns Records exactly as stored — see presentAt: 'latest' to migrate results in memory instead.

query() never returns the _config record, regardless of filter — it's addressable only by ID, via get('_config') or the adapter's own typed ownerEntityId/timezone properties (see Stack initialization). This is the one exception to "adapters are storage engines, Stack is the invariant layer": the exclusion must live in the adapter's own query predicate (a WHERE clause, or the equivalent for an in-memory adapter) rather than be post-filtered by Stack, since post-filtering after the adapter applies limit would silently under-fill a page. Every adapter — including test doubles — implements this exclusion directly; it is not optional convention.

Sorting and pagination

type StackQuery = {
  filter?: RecordFilter;
  sort?:
    | { field: 'createdAt' | 'updatedAt' | 'version'; direction?: 'asc' | 'desc' }
    | { contentField: string; direction?: 'asc' | 'desc' }; // see Sorting by a content field
  limit?: number;
  cursor?: string; // Opaque cursor for page-based pagination
  presentAt?: 'stored' | 'latest'; // see Type migrations
};

type QueryResult = {
  records: StackRecord[];
  cursor: string | null; // null = no further pages
};

Pagination is cursor-based rather than offset-based, so it works consistently across adapters and doesn't drift when records are inserted mid-page. A cursor that can't be decoded — an unknown sort field, a non-numeric sort value, or a corrupted/malformed blob — is a structurally malformed request, not a content-validation failure: adapters throw StackBadRequestError, which maps to 400 (code bad_request), not 422 and not a bare 500.

Direction defaults follow SQL. A sort that names a field runs asc unless direction says otherwise, whether the field is native or a content field, so sort: { contentField: 'name' } reads A→Z. A query with no sort answers by createdAt, newest first, the order a feed-style listing reads in. The two meet in one visible quirk: sort: { field: 'createdAt' } returns oldest first, the opposite of the unsorted default on the same field. No single default direction fits both a name and a date, and core cannot tell them apart at query time (a query can span Types), so the rule keys on nothing but whether a sort was named. Stack.query() and ScopedStack.query() resolve both defaults before an adapter sees the query, so queryRecords() always receives a sort with an explicit direction, or none.

Every list's order is stated here.

Call	Order
query(), no sort	createdAt, newest first
query() with a sort	direction, default asc
getVersions()	newest first (Versioning)
getJournal()	oldest first (Journal)
getAttachmentRecords()	first-recorded order, oldest first (Attachments)
listTypeGrants()	newest first

A sort names either a native column or a content field, never both, and a request naming both is rejected with StackBadRequestError (400) rather than resolved in one direction. They are two members rather than one widened field because a content field may be named version, createdAt or updatedAt, and a 'content.publishedAt' prefix would collide with the path separator a filter key is split on.

sort.field and sort.direction are validated, not merely typed. The 'asc' | 'desc' and three-field types are a compile-time contract only; a request arriving over the wire (a server mapping ?sort=/?direction=) or from a delegated app supplies raw strings. A SQLite record adapter interpolates the direction into its ORDER BY, so Stack.query()/ScopedStack.query() reject a field or direction outside the enumerated set with StackBadRequestError (400, bad_request) before it reaches an adapter — the same posture as a malformed cursor. The value never reaches SQL as anything but one of the two keywords.

query() always paginates. An absent limit means one adapter-default page (50), never "every matching Record." Any caller — app code or library-internal — that needs the complete result set MUST follow cursor to exhaustion; treating a single query() call as exhaustive is a caller bug, not a pagination bug, however the pages are sized.

cursor: null is the only end-of-results signal. A page may be empty while cursor is non-null — possibly several in a row — and a caller that stops on an empty page is the same truncation bug as one that stops after the first page. The case is reachable, not theoretical: a permission-scoped query reads a bounded window of stored Records per call and returns only the ones the requester may read, so a low-visibility requester scanning a large Stack legitimately gets { records: [], cursor: "..." } until the window reaches something they can see. The same holds for any filter an adapter applies after paging. Loop on cursor, never on records.length.

A query result carries no count of the whole match. QueryResult is records and cursor, and nothing else. The count of matching Records, unfiltered, would reveal how many Records exist that the requester cannot read — the cardinality the permission check just hid — so a permission-scoped query could never report one, and neither could any response on the wire (see Wire format § Response envelope). That left a number reachable on exactly one path, a direct unscoped Stack.query() in process, which is the worst of both: an app written against it reads a number locally and null the moment the same code runs scoped or against a server. A field no caller can rely on is worse than no field, and computing it cost every caller a full count of the match on every page — including the scoped and wire paths, which then discarded it.

A caller that genuinely needs a count follows cursor to exhaustion and counts what it receives, which is the only number that was ever true for that requester anyway.

Sorting by a content field

sort.contentField orders by a top-level scalar content field — an article listing by publishedAt, a contact list by name. Without it, a consumer wanting a bounded page in a meaningful order has to read the whole matched set and sort it in memory, a cost that grows with the Stack while the page stays the same size. Content filtering already reaches into content; this closes the asymmetry for ordering.

Only top-level scalars are indexed for sorting, the same line file-ref indexing draws and for the same reason: a value inside an array or an object has no single position to order its Record by. Sorting reach stops at depth 1 while filter.content keeps its path traversal — query reach and indexing are separate mechanisms, and only the first extends to depth. A contentField carrying a path separator is refused with StackBadRequestError (400), and one naming a field no Type declares is a legitimate query whose Records simply have no value to order by.

Every top-level scalar is sortable, with no per-field opt-in. A sortable marker in TypeSchema would change the schema hash, so making an existing field sortable would run the schema drift path for a change that alters no stored shape. The write cost is bounded by the number of top-level scalars on a Type.

A field orders as the kind its schema declares, not as the value a Record happens to hold. number and boolean order numerically (false before true), date as epoch milliseconds — so a date field orders the way createdAt does rather than by ISO string collation — and every string-shaped kind as text. An enum is one of those: it orders by value, not by its position in values, which the schema hash does not preserve. A value that doesn't match its declared kind orders as nothing. Deriving from the schema is what keeps an order stable across Records that spell a field differently.

A date holding no UTC offset is read as UTC. The offset is optional in the shape a date accepts, and resolving an offset-less date-time in whatever zone the host runs in would make the order depend on the machine: an index written on one host and a cursor re-derived on another would disagree, dropping or repeating a Record at a page boundary. UTC is also what a date-only value already resolves to, so 2020-03-01 and 2020-03-01T00:00:00 order alike.

A Record with no value at the sort field sorts after every Record that has one, in both directions. An undated post belongs at the end of a date-ordered listing whichever way it runs; a rule that flipped with direction would put undated posts in the middle of the first page as often as not. Absence is not a value: direction reverses the order between values and never moves absence.

Where a query spans Types that declare one field name differently, numbers sort before text, and direction reverses that with everything else. Stating it here rather than leaving it to each engine's collation is what stops two adapters answering one query in two orders.

A content sort orders the stored shape, not the migrated one. Migrations run in memory after the query returns (see Type migrations), so a Stack mid-migration, sorting on a field a migration rewrites, gets an order derived from what is on disk. The distinction cannot arise for the native three.

Sorting by an association — a relationship target, or a tag — is a different mechanism with a different cost and is not expressible; order on an association is ruled out by the commons convention that associations point and mark but never carry data.

Text ordering

Text orders by a folded key, not by code point: compatibility-decompose (NFKD), drop combining marks, lowercase. apple precedes Émile precedes Zebra, where raw code-point order files every capital ahead of every lowercase letter and every accented word after both. Two values that fold together (Emile, Émile) are then ordered by the stored value itself, so the order is total and a page boundary falls in the same place on every read.

Both comparisons — the folded key, and the stored value that breaks its ties — run by code point. The distinction is not academic: UTF-16 code units, which a JavaScript < compares, file every character above U+FFFF below the U+E000–U+FFFF range, while a SQLite adapter compares the same text as UTF-8 bytes, whose order is code-point order. Comparing by code point is what lets an in-memory adapter and an indexed one answer one query in one order.

The lowercasing is locale-independent. A locale-sensitive fold orders Turkish İstanbul differently from every other runtime's, and the key is stored in an index — the divergence would be baked in rather than merely observed.

This is a fold, not a collation. It buys the two things a reader notices — case and accents — and promises nothing beyond them:

• No locale tailoring. Swedish å ä ö sort with a/o rather than after z; German ß does not fold to ss.
• No script-aware ordering. CJK orders by code point.
• No natural-number ordering: item10 precedes item9.

Locale-correct ordering is out of reach for a stored key regardless of effort, because the correct locale is the reader's and a single index can only encode one. It would need a comparator registered with the engine, which neither SQLite build this project targets can accept — and an ICU-backed comparator would make an order depend on the ICU version each runtime happens to bundle, which is exactly the divergence the conformance suite exists to catch.

Capability-gated filters

content filtering and search depend on adapter capabilities (see Adapter capabilities). query() fails loud rather than silently widening: Stack.query() checks filter.content/filter.search against adapter.capabilities before dispatching, and throws StackBadRequestError (bad_request/400) if the adapter hasn't declared the matching capability — local and remote behave identically.

A refusal names the capability it was refused for. StackBadRequestError carries a capability field holding the path into adapter capabilities the query needed — 'filter.content', 'filter.contentPresent', 'filter.search', 'sort.fields' or 'sort.contentField'. A client presenting the refusal in its own vocabulary reads the name from there rather than re-deriving it from the query, which is how one rule becomes two answers that can disagree; adapter-api raises APIAdapterCapabilityError carrying the same name. A StackBadRequestError thrown for the query's own shape — a malformed content path, a sort naming both a native field and a content field — carries none, because that is the caller's error at every capability level and naming a capability would point at a discovery entry that is not the problem. The field is local only: the wire error body carries bad_request and a message, as it does for every other StackBadRequestError.

contentPresent needs filter.contentPresent on top of a content reach, a separate flag on the same reasoning the reach ladder rests on: a server promising to match a content value has not thereby promised to answer whether one is there, and reading it as such would hand a client the unfiltered superset that ignoring the filter produces. A multi-segment presence path needs the 'path' rung too, exactly as a filter key does.

A sort is gated the same way. sort.contentField needs the adapter's sort.contentField, and a native sort.field must appear in its sort.fields; a sort an adapter hasn't declared throws StackBadRequestError rather than being answered in some other order — which a caller reading one bounded page has no way to notice. sort.contentField and filter.content are independent: a server may order by a content field without offering to filter on one, or the reverse.

A search that sanitizes to nothing (a bare *, punctuation-only input) is treated as a legitimate zero-match query rather than an omitted filter — matching nothing is honest; silently returning the full table is not.

Search text is repaired, not rejected. filter.search is the one filter carrying a query language, and it holds what a person typed into a box — where an unbalanced quote (5" nails), a trailing operator (cats AND) or a leading one is ordinary input on the way to a longer query, not a malformed request. The FTS sanitizers close an odd trailing quote, drop operators left without an operand, and reduce everything outside a phrase to letters, digits, marks, whitespace, parens and quotes — so the search runs against the terms actually present rather than failing. Text inside a phrase is left alone: "cats AND dogs" is literal to the engine, and rewriting inside it would change what was asked for.

The reduction is an allow-list, not a list of characters to strip. FTS5's column-filter syntax is wider than colname:term — -name and {a b} filter columns with no colon in them — so a strip-list written against the colon form would let -cats and even the hyphen in cats-dogs reach the engine as column names, answered with no such column. Naming what may pass means the next piece of syntax nobody anticipated arrives as a separator rather than as an error, and nothing is lost: the default tokenizer already splits terms on those characters, so cats-dogs searches for the two tokens it was indexed as. Two grammar rules the same reduction has to respect: a group needs an explicit operator beside it (cats (dogs) is an error where cats AND (dogs) is not, so the implied AND is written back in), and a control character ends the string early for SQLite's C API, so control characters are dropped inside a phrase as well as outside.

The repair claims no completeness against the engine's grammar, so adapters back it with a rule that does: whatever reaches the engine as search text and fails to parse surfaces as StackBadRequestError (bad_request/400), never as a raw engine error. Membership in the error taxonomy is what gives a server a status to answer with — an error outside it has no mapping, and ordinary search input would land as a 500. The rule is scoped to filter.search deliberately: every other clause is built from bound parameters, so a parse failure in one is the adapter's own bug and must not be relabelled as the caller's.

Sanitization bounds a search's grammar, not its cost. The FTS sanitizers strip the operators that make a query pathological to parse, but a syntactically ordinary search over a large index can still be expensive to execute. Full-text search therefore lands in the same best-effort-on-cost bucket as a nested content path, and a server owes it the same bound. See Wire format § Bounding query cost.