Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
26 changes: 26 additions & 0 deletions .changeset/21492-retire-memory-boot-store.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
---
'@objectstack/spec': minor
'@objectstack/runtime': minor
'@objectstack/cli': minor
---

fix(spec,runtime,cli)!: the in-memory (mingo) engine is no longer a boot store — every boot door refuses it and names SQLite instead (#21492, #21572)

Clause-②: yes (narrowing)

<!-- adr-0087: not-required (no-migration-prescription) no metadata body, authorable key or stored shape moves: the driver table keeps `memory`, `mingo`, `in-memory` and `inmemory` on its config-contract face, so `resolveDriverId` answers them exactly as before and a stored `datasource.driver: memory` still parses against `MemoryConfigSchema`; what narrows is the boot selection (`--database-driver`, `OS_DATABASE_DRIVER`, `databaseDriver`, a `memory://` or `mingo://` database URL, and a project's default datasource declared on the engine), which is host configuration that `objectstack migrate meta` does not rewrite — and no rewrite would be truthful, since the only replacement is a different engine the operator has to choose. The other categories are closed on facts: all three packages publish (not `unpublished`); no ADR-0087 id covers a boot selection (not `registered` / `already-registered`); and the change is runtime behaviour plus exported constant values, not an interface or a type alone (not `runtime-interface-only` / `type-surface-only`). -->

**BREAKING**: the in-memory (mingo) engine can no longer be selected as the store a server, a migration or an embedded stack boots on. It refuses every tenant-scoped read by design, so a boot on it signed a user in and then answered `503` to every data request; there was nothing working to keep. The retirement is made at the declaration: `@objectstack/spec`'s driver table withdrew `memory`, `mingo` and `in-memory` from its selection face (they stay on the config-contract face beside `inmemory`), and every boot door refuses the engine with one sentence that names the replacement.

- **`@objectstack/spec`** — `DATABASE_DRIVER_SELECTION_ALIASES` no longer lists `memory`, `mingo` or `in-memory`; `DATABASE_DRIVER_SELECTION_IDS` no longer lists `memory`; `resolveDatabaseDriverId` answers `undefined` for all four spellings. `resolveDriverId`, `DRIVER_ID_ALIASES`, `BUILTIN_DRIVER_IDS` and the `memory` config contract are unchanged.
- **`@objectstack/cli`** — `--database-driver memory` is refused while the flags parse (`os dev`, `os start`); `OS_DATABASE_DRIVER=memory` / `mingo` / `in-memory` is refused before `os dev` or `os start` prints its Database row; `os serve`'s legacy path refuses the spellings and the `memory://` / `mingo://` schemes as a fatal boot error. The help no longer offers `memory://`.
- **`@objectstack/runtime`** — `createStandaloneStack`, `createDefaultHostConfig` and `resolveStandaloneDatabase` (every ordinary `os dev` / `os start` / `os serve` boot and every `os migrate` subcommand) refuse the spellings, the `memory://` and `mingo://` schemes, and a project whose default datasource is declared with `driver: 'memory'`. `resolveProjectDatabaseUrl` refuses a retired driver selection ahead of every rung, and its `ProjectDatabaseUrlSource` type no longer has the `'memory-driver'` member. `ResolvedStandaloneDatabase.driver` never names `memory`. Two exports are added for hosts that refuse the engine themselves: `namesRetiredMemoryEngine` and `retiredMemoryEngineMessage`.
- **Unchanged:** the `@objectstack/driver-memory` package; a declared non-default datasource with `driver: 'memory'` and a directly constructed `InMemoryDriver`, both still built; SQLite's dev step-down, whose last rung is still this driver.

Migration — one flag change:

- FROM `os dev --database-driver memory` (or `OS_DATABASE_DRIVER=memory`) TO `os dev --fresh` for a throwaway database deleted on exit.
- FROM `OS_DATABASE_URL=memory://…` / `--database memory://…` / `databaseUrl: 'memory://…'` TO `:memory:` (SQLite's own in-memory database), e.g. `OS_DATABASE_URL=:memory:`.
- FROM a default datasource declared `{ driver: 'memory' }` TO a SQLite one, e.g. `{ driver: 'sqlite', config: { filename: ':memory:' } }`.

No shipped example selects the engine. It ships as `minor` under the launch-window convention for accept-set narrowings.
20 changes: 16 additions & 4 deletions content/docs/data-modeling/drivers.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -79,7 +79,7 @@ libSQL data stayed untouched. Pass the token with `--database-auth-token`
| **SQLite (WASM)** | `@objectstack/driver-sqlite-wasm` | `SqliteWasmDriver` | `sqlite-wasm` \| `wasm-sqlite` \| `wasm` |
| **MongoDB** | `@objectstack/driver-mongodb` | `MongoDBDriver` | `mongodb` \| `mongo` (single-tenant only — see [below](#multi-tenancy-not-supported)) |
| **Turso / libSQL** | `@objectstack/driver-turso` (optional install — see the callout above) | `TursoDriver` | `turso` \| `libsql` |
| **Memory** | `@objectstack/driver-memory` | `InMemoryDriver` | `memory` |
| **Memory** | `@objectstack/driver-memory` | `InMemoryDriver` | none — not a boot store (see [Memory Driver](#memory-driver)) |

> All SQL flavours (PostgreSQL / MySQL / SQLite) are served by a single
> `@objectstack/driver-sql` package — there is no separate
Expand All @@ -101,7 +101,7 @@ driver ships that contract as a zod schema, exported from `@objectstack/spec/dat
| `sqlite` \| `sqlite3` | `SqliteConfigSchema` | `filename`, `autoMigrate` |
| `sqlite-wasm` \| `wasm-sqlite` | `SqliteWasmConfigSchema` | `filename`, `persist` |
| `mongo` \| `mongodb` | `MongoConfigSchema` | `url`, `host`, `port`, `database`, `username`, `password`, `authSource`, `options` |
| `memory` \| `in-memory` | `MemoryConfigSchema` | `initialData`, `strictMode`, `persistence` |
| `memory` \| `in-memory` \| `mingo` | `MemoryConfigSchema` | `initialData`, `strictMode`, `persistence` |

An unrecognised key is rejected with its correction, at authoring time and in the
Setup → Datasources wizard alike:
Expand Down Expand Up @@ -798,13 +798,25 @@ exits unless persistence is explicitly requested.
It is the **last-resort fallback** in dev mode: `objectstack dev` prefers native
SQLite (`better-sqlite3`), falls back to the pure-JS WASM SQLite driver if the
native binary is unavailable, and only drops to the in-memory driver if WASM also
fails to load. Set `OS_DATABASE_DRIVER=memory` to select it explicitly.
fails to load.

<Callout type="warn">
**It cannot be selected as a boot store.** The driver has no row-level tenant
isolation and refuses every tenant-scoped call, so a server booted on it signs a
user in and then answers `503` to every data request. Every boot door refuses it,
naming the replacement: `OS_DATABASE_DRIVER=memory` / `mingo` / `in-memory`,
`--database-driver memory`, a `memory://` or `mingo://` database URL, and a
project whose **default** datasource is declared with `driver: 'memory'`. For a
throwaway database use `os dev --fresh`; for SQLite's own in-memory database use
the URL `:memory:` (`OS_DATABASE_URL=:memory:`). The driver package itself is
unchanged: the declared datasources and direct construction below still build it.
</Callout>

**Both ways of building the driver are ephemeral by default** (#4083, #4065):

| How it is built | Persistence |
| :--- | :--- |
| A **declared datasource** — `{ driver: 'memory' }` in a stack/app config | **Ephemeral.** Nothing is written to disk unless the declaration sets `config.persistence` — and when it does, the destination is scoped per datasource, so two memory datasources never share one file. |
| A **declared datasource** — `{ driver: 'memory' }` in a stack/app config (not as the project's default datasource, which the boot refuses) | **Ephemeral.** Nothing is written to disk unless the declaration sets `config.persistence` — and when it does, the destination is scoped per datasource, so two memory datasources never share one file. |
| `new InMemoryDriver()` **constructed directly** | **Ephemeral.** Pass `persistence` to opt in (see below). Note the scoping above is the *factory's* job: two directly-constructed `'auto'` drivers in one process still share the single default path. |

```typescript
Expand Down
26 changes: 20 additions & 6 deletions content/docs/deployment/cli.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -69,7 +69,7 @@ predicate/schema/binding mistakes that fail silently at runtime), and `os dev --
</Callout>

<Callout type="tip">
`os dev --ui` starts a dev server with the bundled Console UI, auto-loads ObjectQL, an in-memory driver when appropriate, and the Hono HTTP server.
`os dev --ui` starts a dev server with the bundled Console UI, auto-loads ObjectQL, a SQLite database (a persistent project file by default, a throwaway one with `--fresh`), and the Hono HTTP server.
</Callout>

## Commands
Expand Down Expand Up @@ -182,8 +182,8 @@ os dev --database file:./data/test.db --auth-secret $(openssl rand -hex 32)
| Flag | Env equivalent | Purpose |
|---|---|---|
| `-a, --artifact <path\|url>` | `OS_ARTIFACT_URL` / `OS_ARTIFACT_PATH` | Boot a pre-built artifact directly; **skips auto-compile** |
| `-d, --database <url>` | `OS_DATABASE_URL` | `file:…` / `libsql://` / `postgres://` / `mongodb://` / `memory://` |
| `--database-driver <kind>` | `OS_DATABASE_DRIVER` | Force `sqlite` \| `sqlite-wasm` \| `turso` \| `postgres` \| `mysql` \| `mongodb` \| `memory` |
| `-d, --database <url>` | `OS_DATABASE_URL` | `file:…` / `:memory:` / `libsql://` / `postgres://` / `mongodb://` |
| `--database-driver <kind>` | `OS_DATABASE_DRIVER` | Force `sqlite` \| `sqlite-wasm` \| `turso` \| `postgres` \| `mysql` \| `mongodb` |
| `--database-auth-token <t>` | `OS_DATABASE_AUTH_TOKEN` | libsql/Turso token |
| `--auth-secret <s>` | `OS_AUTH_SECRET` | Override the dev-fallback secret |
| `--environment-id <id>` | `OS_ENVIRONMENT_ID` | Environment identifier (default `env_local`) |
Expand Down Expand Up @@ -472,8 +472,8 @@ os start
|---|---|---|
| `-a, --artifact <path\|url>` | `OS_ARTIFACT_PATH` | File path or `http(s)://` URL to the compiled artifact |
| — | `OS_ARTIFACT_URL` | Boot a published artifact **by reference**, optionally content-hash pinned via a `#sha256=` fragment. See [Artifact-pinned boot](/docs/deployment/self-hosting#artifact-pinned-boot-os_artifact_url) |
| `-d, --database <url>` | `OS_DATABASE_URL` | `file:…` / `libsql://` / `postgres://` / `mongodb://` / `memory://` |
| `--database-driver <kind>` | `OS_DATABASE_DRIVER` | Force `sqlite` \| `sqlite-wasm` \| `turso` \| `postgres` \| `mysql` \| `mongodb` \| `memory` when the URL is ambiguous |
| `-d, --database <url>` | `OS_DATABASE_URL` | `file:…` / `:memory:` / `libsql://` / `postgres://` / `mongodb://` |
| `--database-driver <kind>` | `OS_DATABASE_DRIVER` | Force `sqlite` \| `sqlite-wasm` \| `turso` \| `postgres` \| `mysql` \| `mongodb` when the URL is ambiguous |
| `--database-auth-token <token>` | `OS_DATABASE_AUTH_TOKEN` | Auth token for libsql/Turso |
| `--auth-secret <secret>` | `OS_AUTH_SECRET` / `AUTH_SECRET` | Secret for `@objectstack/plugin-auth`. If neither the flag nor the env var is set, `os start` **auto-generates one** and persists it at `<home>/auth-secret` |
| `--home <dir>` | `OS_HOME` | Home directory for persistent state (default `<cwd>/.objectstack` when an `objectstack.config.ts` is present, otherwise `~/.objectstack`) |
Expand Down Expand Up @@ -502,10 +502,24 @@ The fall-through applies to the **conventional** locations only. Remote
(`http(s)://`) sources cannot be checked up front and are validated when fetched.
</Callout>

<Callout type="warn">
**The in-memory (mingo) engine is not a boot store.** It refuses every
tenant-scoped read, so a server booted on it signed you in and then answered
`503` to every data request. Every boot command — `os dev`, `os start`,
`os serve` and every `os migrate` subcommand — refuses it: the
`--database-driver memory` flag value (rejected while the flags parse),
`OS_DATABASE_DRIVER=memory` / `mingo` / `in-memory`, and a `memory://` or
`mingo://` database URL, each with a message naming the replacement. Use SQLite
instead: `os dev --fresh` for a throwaway database that is deleted on exit, or
`--database :memory:` (`OS_DATABASE_URL=:memory:`) for SQLite's own in-memory
database. A datasource you declare with `driver: 'memory'` still validates;
as the project's **default** datasource it is refused at boot the same way.
</Callout>

**What it boots:**
- Reads the artifact's `manifest`, `objects`, `views`, `flows`, …
- Auto-registers the platform services declared in `requires: [...]` (e.g. `ai`, `automation`, `analytics`, `auth`, `ui`). Declaring a **service** capability (`automation`, `analytics`, `ai`, `audit`, …) is a *requirement*: if its provider package isn't installed, boot **fails fast** with a clear error instead of silently starting without a capability you asked for. (`auth` and `ui` are tier-gated with their own opt-in rules — `auth`'s secret-gated skip is described below.)
- Auto-detects the driver from the database URL scheme (`memory://` → in-memory, `libsql://`/`https://*.turso.*` → Turso — via the optional `@objectstack/driver-turso` package, and a loud failure with the install command when it is missing rather than a fallback to sqlite —, `postgres[ql]://`/`pg://` → pg, `mongodb[+srv]://` → MongoDB, otherwise sqlite)
- Auto-detects the driver from the database URL scheme (`libsql://`/`https://*.turso.*` → Turso — via the optional `@objectstack/driver-turso` package, and a loud failure with the install command when it is missing rather than a fallback to sqlite —, `postgres[ql]://`/`pg://` → pg, `mongodb[+srv]://` → MongoDB, otherwise sqlite; `:memory:` is SQLite's own in-memory database). `memory://` and `mingo://` are refused — see the callout below.
- Runs standalone boot mode with one active environment.

**Authentication:**
Expand Down
4 changes: 2 additions & 2 deletions content/docs/deployment/environment-variables.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -46,9 +46,9 @@ read at startup unless noted otherwise. Boolean variables accept `true` / `false

| Variable | Type | Default | Description |
|:---|:---|:---|:---|
| `OS_DATABASE_URL` | url | — | Database connection string (e.g. `file:./data.sqlite`, `postgres://…`, `mysql://…`, `mongodb://…`, `memory://`). `libsql://` / `*.turso.io` (Turso) is inferred too, but its driver is an **optional** package: `npm install @objectstack/driver-turso`, otherwise the boot fails loudly with that command — it never falls back to SQLite. |
| `OS_DATABASE_URL` | url | — | Database connection string (e.g. `file:./data.sqlite`, `:memory:` for SQLite's own in-memory database, `postgres://…`, `mysql://…`, `mongodb://…`). `memory://` and `mingo://` name the in-memory (mingo) engine, which is not a boot store, and are refused at boot with a message naming the replacement. `libsql://` / `*.turso.io` (Turso) is inferred too, but its driver is an **optional** package: `npm install @objectstack/driver-turso`, otherwise the boot fails loudly with that command — it never falls back to SQLite. |
| `OS_DATABASE_AUTH_TOKEN` | string | — | Auth token for a libSQL/Turso connection (`--database-auth-token`). The vendor's own `TURSO_AUTH_TOKEN` is read as a fallback and is **not** renamed (see the third-party names note above). Ignored by every other driver — their credentials live in the URL. |
| `OS_DATABASE_DRIVER` | enum | inferred | Force a specific driver when the URL is ambiguous. `memory` \| `sqlite` \| `sqlite-wasm` \| `postgres` \| `mysql` \| `mongodb` \| `turso`. `mysql` is a supported deployment target that carries three dialect caveats — two of them integrity guarantees MySQL cannot enforce at the database. Read [MySQL dialect caveats](/docs/data-modeling/drivers#mysql-dialect-caveats) before choosing it. |
| `OS_DATABASE_DRIVER` | enum | inferred | Force a specific driver when the URL is ambiguous. `sqlite` \| `sqlite-wasm` \| `postgres` \| `mysql` \| `mongodb` \| `turso`. `memory` / `mingo` / `in-memory` are refused at boot: the in-memory engine cannot serve a tenant-scoped read, so use `os dev --fresh` or `OS_DATABASE_URL=:memory:` instead. `mysql` is a supported deployment target that carries three dialect caveats — two of them integrity guarantees MySQL cannot enforce at the database. Read [MySQL dialect caveats](/docs/data-modeling/drivers#mysql-dialect-caveats) before choosing it. |
| `OS_DATABASE_SQLITE_JOURNAL_MODE` | enum | `wal` | Journal mode for **file-backed** SQLite. `wal` (default) lets a dev server and CLI commands share one file without blocking each other, and is what makes the `os migrate` occupancy check reliable. Set to `delete` for SQLite's rollback journal — required when the database lives on a **network filesystem** (NFS/SMB), where WAL cannot work. The setting is applied, not merely skipped: `delete` converts a database that already adopted WAL back. Ignored for `:memory:`, for the WASM SQLite driver, and for non-SQLite drivers. A per-datasource `sqliteJournalMode` in driver config outranks it. See [Journal mode](/docs/data-modeling/drivers#journal-mode-wal-and-cross-process-access). |
| `OS_DATABASE_POOL_MAX` | integer | `5` | Maximum pooled connections **each replica** opens to `postgres` / `mysql`. The pool, not the database, is usually what caps authed throughput on a multi-replica deployment: at the default of 5 a 3-replica cluster reaches at most ~15 connections, which can plateau the data API while the database still has most of its `max_connections` free. Size it so **`replicas × OS_DATABASE_POOL_MAX` stays below the database's `max_connections`**, leaving headroom for migrations and admin connections — e.g. 3 replicas × 40 = 120 against `max_connections=200`. A per-datasource `pool.max` in the datasource's own definition outranks it, and with the variable unset nothing changes. A value that is not a positive integer refuses the boot rather than falling back to the default. Ignored by `memory` / `sqlite` / `sqlite-wasm` / `turso`, which open no pool — declaring a `pool` block on those is an error, not a silent drop. |
| `OS_ALLOW_DRIVER_CONNECT_FAILURE` | boolean | `false` | Escape hatch for the driver-connect boot guard. By default a data driver that fails to connect at startup **refuses the boot** — a server that cannot reach its database must not report itself started and then fail every request. The same guard covers a **declared datasource** that objects bind to via `datasource: '…'`, or an `external` one with `validation.onMismatch: 'fail'`: those objects have no fallback datasource, so an unconnected one means they are all dead. Set to `1` to boot anyway, in an explicitly degraded state logged loudly at startup. There is **no reconnection**: whatever failed stays dead for the process lifetime and every query and schema sync routed to it fails. |
Expand Down
5 changes: 3 additions & 2 deletions content/docs/plugins/index.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -222,8 +222,9 @@ Security features including field-level and row-level security.
- Middleware-based security

### `@objectstack/driver-memory`
In-memory (mingo) data driver for development and testing.
- Opt in explicitly with `OS_DATABASE_DRIVER=memory`, `--database-driver memory`, or a `memory://` URL — when no driver is configured the dev default is SQLite, not this driver
In-memory (mingo) data driver for testing and embedding.
- Not a boot store: `OS_DATABASE_DRIVER=memory`, `--database-driver memory` and a `memory://` URL are refused at boot, because the driver refuses every tenant-scoped read. For a throwaway dev database use `os dev --fresh`; for SQLite's own in-memory database use `OS_DATABASE_URL=:memory:`
- Still built for a declared `driver: 'memory'` datasource and by direct construction (`new InMemoryDriver()`)

---

Expand Down
Loading
Loading