fix(driver-sql, driver-turso): four filter-refusal doors read the provenance mark before naming a read scope field or comparand (#20020)

.changeset/20020-refusal-doors-provenance.md

@@ -0,0 +1,38 @@
+---
+'@objectstack/driver-sql': patch
+'@objectstack/driver-sqlite-wasm': patch
+'@objectstack/driver-turso': patch
+---
+
+fix(driver-sql, driver-turso): four filter-refusal doors stop naming a read scope's field and comparand unless the refused predicate is marked as the caller's own (#20020)
+
+Clause-②: no
+
+A read scope is the RLS, sharing or tenant predicate that `plugin-security` (ordinary reads) and `service-analytics` (the ObjectQL analytics face) AND into the caller's `where`. Both merges mark the scope `'policy'` and the caller's own predicate `'author'` (`markFilterSubtreeProvenance`, `@objectstack/spec/data`). When `SqlDriver` refused a scope at one of the four doors below, the `INVALID_FILTER` / 400 message named the scope's field, and for three of them its comparand too. It did not check the mark. Measured on both faces, through `POST /api/v1/analytics/query` and through an ObjectQL `find` under a merge shaped like `plugin-security`'s:
+
+- a column the table does not have. This is reachable from a real CEL rule on a field that is declared but has no column yet;
+- a retired operator (`$regex`, `$options`) or an operator outside the vocabulary;
+- `$and` / `$or` whose operand is not a list;
+- a `$null` / `$exists` whose comparand is not a boolean.
+
+Each of these doors now reads the mark on the node it refused, the same way the cross-field and target-field refusals already did:
+
+- **`'policy'`, unmarked or ambiguous:** same `INVALID_FILTER` / 400. The message says which kind of refusal fired, but names no field, operator, comparand or filter path. Those go to the server log. For the unresolvable column, the message is the unnamed wording the driver already used when it could not parse the dialect's message.
+- **`'author'`:** the full message, the same text the door answered before.
+
+To find the node, the unresolvable-column door looks up the column name the database reported. It discloses only when every node that names that column is marked `'author'`. A `$and` / `$or` with a primitive operand is judged by the node that carries the key.
+
+`SqliteWasmDriver` (`@objectstack/driver-sqlite-wasm`) and `TursoDriver` in local mode extend `SqlDriver`, so they inherit this change from it: the same four doors answer the same way there.
+
+**What an unmarked caller loses:** its own diagnostic from these four doors. Measured cases where the caller's own predicate reaches the driver unmarked:
+
+- no security plugin in the stack;
+- a system-context call;
+- an anonymous call;
+- a `where` that holds a `{placeholder}` token, which the engine rewrites before the merge.
+
+That caller gets the withheld wording with the same code and status. A member's plain `where` under `plugin-security` is marked `'author'` and keeps the full text.
+
+The same three door classes on the Turso REMOTE transport (`RemoteTransport`) now read the mark too: the retired or unknown operator (including a non-operator key in an operator map), the non-list combinator, and the non-boolean `$null` / `$exists`. The operands go to its diagnostic sink. `TursoDriver`'s remote mode rebuilds every filter node before the transport sees it, so no mark reaches the transport there, and these refusals keep the withheld wording for every caller in that mode. The unresolvable WHERE column has no refusal on the remote face (the transport answers `[]`) and is not changed here.
+
+Not changed: which filters are refused, and the code and status of every refusal. The engine's declared-type, temporal-comparand and filter-token doors are not changed here.

packages/drivers/driver-sql/src/sql-driver-11541-aggregate-unresolvable-column-refusal.test.ts

@@ -62,6 +62,7 @@
 
 import { describe, it, expect, beforeAll, afterAll } from 'vitest';
 import type { DriverQuery } from '@objectstack/spec/contracts';
+import { markFilterSubtreeProvenance } from '@objectstack/spec/data';
 import { SqlDriver } from './sql-driver.js';
 import { DIALECT_CELLS, declareDialectCell, dialectCell, type DialectCell } from './live-dialect-matrix.testkit.js';
 
@@ -140,7 +141,15 @@ describe(`[#11541] driver-sql — aggregate() attributes an unresolvable column
   // THE PROBE TABLE — five rows, three doors, one class of answer
   // ───────────────────────────────────────────────────────────────
 
-  const WHERE_NOSUCHCOL: NonNullable<DriverQuery['where']> = { nosuchcol: SECRET_LITERAL };
+  // [#20020] Marked 'author', as a read-scope merge boundary marks a caller's
+  // own predicate: the column NAME reaches the wire only for a predicate the
+  // caller is known to have written (the #8220 contract), and every row below
+  // pins that author-facing answer. The unmarked and policy-marked answers are
+  // pinned in `sql-driver-refusal-door-provenance.test.ts`.
+  const WHERE_NOSUCHCOL: NonNullable<DriverQuery['where']> = markFilterSubtreeProvenance(
+    { nosuchcol: SECRET_LITERAL },
+    'author',
+  );
 
   it('rows 1-2: find() and count() still answer the #8790 refusal — the baseline this door joins', async () => {
     for (const [half, run] of [
@@ -383,7 +392,8 @@ describe('[#11541] aggregateBackendFault — the three arms and their fences', (
 
   it('arm 2: a name in neither clause is the WHERE — #8790 verbatim', () => {
     const err = classify(
-      { where: { nosuchcol: 1 }, aggregations: [{ function: 'count', alias: 'n' }] },
+      // [#20020] The caller's own predicate, marked as a merge boundary marks it.
+      { where: markFilterSubtreeProvenance({ nosuchcol: 1 }, 'author'), aggregations: [{ function: 'count', alias: 'n' }] },
       'select count(*) as `n` from `t` where `nosuchcol` = 1 - no such column: nosuchcol',
     );
     expect(err.code).toBe('INVALID_FILTER');
@@ -413,7 +423,7 @@ describe('[#11541] aggregateBackendFault — the three arms and their fences', (
   // MySQL (DOTTED_STATUS_QUO, #8371 owns the dotted-path verdict).
   it('FENCE: a dotted where key is not attributed to an aggregation over its tail segment', () => {
     const err = classify(
-      { where: { 'title.x': 1 }, aggregations: [{ function: 'avg', field: 'x', alias: 'n' }] },
+      { where: markFilterSubtreeProvenance({ 'title.x': 1 }, 'author'), aggregations: [{ function: 'avg', field: 'x', alias: 'n' }] },
       'select avg(`x`) as `n` from `t` where `title`.`x` = 1 - no such column: title.x',
     );
     expect(err.code).toBe('INVALID_FILTER');

packages/drivers/driver-sql/src/sql-driver-17857-distinct-unresolvable-column-refusal.test.ts

@@ -83,6 +83,7 @@
 import { describe, it, expect, beforeAll, afterAll } from 'vitest';
 import type { DriverQuery } from '@objectstack/spec/contracts';
 import type { FilterCondition } from '@objectstack/spec/data';
+import { markFilterSubtreeProvenance } from '@objectstack/spec/data';
 import { SqlDriver } from './sql-driver.js';
 import { DIALECT_CELLS, declareDialectCell, type DialectCell } from './live-dialect-matrix.testkit.js';
 
@@ -182,7 +183,13 @@ describe(`[#17857] driver-sql — distinct() attributes an unresolvable column (
     const onCount = await caught(() => driver.count(TABLE, { where: { [MISSING_COLUMN]: 1 } }));
     const onFind = await caught(() => driver.find(TABLE, { where: { [MISSING_COLUMN]: 1 } }));
     const onAggregate = await caught(() => driver.aggregate(TABLE, GROUP_BY_MISSING));
-    const onDistinctFilter = await caught(() => driver.distinct(TABLE, 'title', { [MISSING_COLUMN]: 1 }));
+    // [#20020] The caller's own filter, marked as a read-scope merge boundary
+    // marks it: the column NAME reaches the wire only for a predicate the
+    // caller is known to have written (the #8220 contract). The unmarked and
+    // policy answers are pinned in `sql-driver-refusal-door-provenance.test.ts`.
+    const onDistinctFilter = await caught(() =>
+      driver.distinct(TABLE, 'title', markFilterSubtreeProvenance({ [MISSING_COLUMN]: 1 }, 'author')),
+    );
     const onDistinctField = await caught(() => driver.distinct(TABLE, MISSING_COLUMN));
 
     // Rows 1-3: the ruled answers, unmoved by this card.

packages/drivers/driver-sql/src/sql-driver-backend-fault-envelope.test.ts

@@ -86,6 +86,7 @@
 
 import { describe, it, expect, beforeAll, afterAll } from 'vitest';
 import type { DriverQuery } from '@objectstack/spec/contracts';
+import { markFilterSubtreeProvenance } from '@objectstack/spec/data';
 import { SqlDriver } from './sql-driver.js';
 import { DIALECT_CELLS, declareDialectCell, type DialectCell } from './live-dialect-matrix.testkit.js';
 
@@ -110,7 +111,13 @@ const SECRET_LITERAL = 'zz-backend-fault-must-not-leak';
  */
 const DOTTED_KEY: NonNullable<DriverQuery['where']> = { 'title.x': SECRET_LITERAL };
 const TYPE_REJECTED: NonNullable<DriverQuery['where']> = { rank: SECRET_LITERAL };
-const UNRESOLVABLE_COLUMN: NonNullable<DriverQuery['where']> = { nosuchcol: SECRET_LITERAL };
+// [#20020] Marked 'author', as a read-scope merge boundary marks a caller's own
+// predicate: the #8790 refusal names the column only for a predicate the caller
+// is known to have written (the #8220 contract).
+const UNRESOLVABLE_COLUMN: NonNullable<DriverQuery['where']> = markFilterSubtreeProvenance(
+  { nosuchcol: SECRET_LITERAL },
+  'author',
+);
 
 async function caught(run: () => Promise<unknown>): Promise<any> {
   try {

packages/drivers/driver-sql/src/sql-driver-boolean-identity.test.ts

@@ -54,6 +54,7 @@
 import { describe, it, expect, beforeEach, afterEach } from 'vitest';
 import { SqlDriver } from '../src/index.js';
 import type { FilterCondition } from '@objectstack/spec/data';
+import { markFilterSubtreeProvenance } from '@objectstack/spec/data';
 
 const FIXTURE = [
   { id: '1', stage: 'won', owner: 'u1', amount: 10 },
@@ -199,7 +200,11 @@ describe('[#5134] SqlDriver compiles empty $and/$or/$not to their boolean identi
 
     for (const [name, where, position] of cases) {
       it(`${name} → 400 INVALID_FILTER naming ${position}`, async () => {
-        const err = await refusalOf(where);
+        // [#20020] Marked 'author', as a read-scope merge boundary marks a
+        // caller's own predicate: the non-list combinator refusal names its
+        // position only for a predicate the caller is known to have written
+        // (the #8220 contract); the other rows name it either way.
+        const err = await refusalOf(markFilterSubtreeProvenance(where, 'author'));
         expect(err.code).toBe('INVALID_FILTER');
         expect(err.status).toBe(400);
         expect(err.message).toContain(position);

packages/drivers/driver-sql/src/sql-driver-filter-no-silent-drop.test.ts

@@ -42,7 +42,7 @@
 
 import { describe, it, expect, beforeEach } from 'vitest';
 import { SqlDriver } from '../src/index.js';
-import { parseFilterAST, type FilterCondition } from '@objectstack/spec/data';
+import { markFilterSubtreeProvenance, parseFilterAST, type FilterCondition } from '@objectstack/spec/data';
 
 describe('SqlDriver rejects an uncompilable filter instead of dropping it', () => {
   let driver: SqlDriver;
@@ -134,7 +134,10 @@ describe('SqlDriver rejects an uncompilable filter instead of dropping it', () =
   it('still throws on an unsupported operator inside a well-formed condition', async () => {
     // Pre-existing behaviour on the shape the driver still compiles, pinned so
     // the refusal cannot regress into a silent drop one layer down.
-    await expect(find({ stage: { $sounds_like: 'won' } }))
+    // [#20020] Marked 'author', as a read-scope merge boundary marks a caller's
+    // own predicate: the operator and field are named only then (the #8220
+    // contract). Unmarked, the refusal is the same code, withheld.
+    await expect(find(markFilterSubtreeProvenance({ stage: { $sounds_like: 'won' } }, 'author')))
       .rejects.toThrow(/Unsupported filter operator "\$sounds_like"/);
   });
 

packages/drivers/driver-sql/src/sql-driver-filter-refusal-envelope.test.ts

@@ -31,6 +31,7 @@
 import { describe, it, expect, beforeEach } from 'vitest';
 import { SqlDriver } from '../src/index.js';
 import type { FilterCondition } from '@objectstack/spec/data';
+import { markFilterSubtreeProvenance } from '@objectstack/spec/data';
 
 /** The shape `mapDataError` / `sendError` read off a thrown driver error. */
 interface WireBearingError extends Error {
@@ -74,7 +75,13 @@ describe('[#4436] SqlDriver filter refusals carry INVALID_FILTER and leak no dri
 
   // The issue's own repro, at the layer that produces the envelope.
   it('the $-object unsupported-operator branch — the exact shape #4436 reported', async () => {
-    const err = await refusalOf(() => find({ stage: { $bogusop: 'x' } }));
+    // [#20020] Marked 'author', as a read-scope merge boundary marks a caller's
+    // own predicate: WHICH operator on WHICH field is the predicate's detail,
+    // named only for a predicate the caller is known to have written (the
+    // #8220 contract). The vocabulary is a capability statement and survives
+    // either way — pinned for the withheld wording in
+    // `sql-driver-refusal-door-provenance.test.ts`.
+    const err = await refusalOf(() => find(markFilterSubtreeProvenance({ stage: { $bogusop: 'x' } }, 'author')));
     expect(err.code).toBe('INVALID_FILTER');
     expect(err.status).toBe(400);
     expect(err.message).not.toContain('[sql-driver]');

packages/drivers/driver-sql/src/sql-driver-icontains-and-retired-operators.test.ts

@@ -79,7 +79,7 @@
 import type { Knex } from 'knex';
 import { describe, it, expect, beforeAll, afterAll } from 'vitest';
 import type { DriverOptions, FilterCondition } from '@objectstack/spec/data';
-import { FILTER_TEXT_CASES, FILTER_TEXT_ROWS } from '@objectstack/spec/data';
+import { FILTER_TEXT_CASES, FILTER_TEXT_ROWS, markFilterSubtreeProvenance } from '@objectstack/spec/data';
 import { SqlDriver } from './sql-driver.js';
 import { dialectCell } from './live-dialect-matrix.testkit.js';
 
@@ -141,6 +141,16 @@ describe('[#5702] SqlDriver — $icontains, and the retired $regex/$options', ()
     return err;
   };
 
+  /**
+   * [#20020] The caller's own predicate, marked as a read-scope merge boundary
+   * marks it — on a shallow COPY, so a shared case constant is never marked.
+   * The retired-operator refusal names the operator and its replacement only
+   * for a predicate the caller is known to have written (the #8220 contract);
+   * the withheld wording is pinned in `sql-driver-refusal-door-provenance.test.ts`.
+   */
+  const authored = (where: FilterCondition): FilterCondition =>
+    markFilterSubtreeProvenance({ ...where }, 'author');
+
   it('seeded the fixture (the premise)', async () => {
     expect(await ids({})).toEqual(['1', '2', '3', '4', '5', '6', '7', '8', '9']);
   });
@@ -232,7 +242,7 @@ describe('[#5702] SqlDriver — $icontains, and the retired $regex/$options', ()
     ],
   ] as const) {
     it(`REFUSES ${label}, naming the replacement`, async () => {
-      const err = await refusalOf(where);
+      const err = await refusalOf(authored(where));
       expect(err.code).toBe('INVALID_FILTER');
       expect(err.status).toBe(400);
       expect(err.message).toContain('RETIRED');
@@ -339,7 +349,7 @@ describe('[#5702] SqlDriver — $icontains, and the retired $regex/$options', ()
     for (const testCase of FILTER_TEXT_CASES) {
       it(testCase.name, async () => {
         if (testCase.expectRejection) {
-          const err = await refusalOf(testCase.filter);
+          const err = await refusalOf(authored(testCase.filter));
           expect(err.code).toBe(testCase.code);
           expect(err.status).toBe(400);
           for (const mention of testCase.mustMention) expect(err.message).toContain(mention);

packages/drivers/driver-sql/src/sql-driver-null-operators.test.ts

@@ -1,7 +1,7 @@
 // Copyright (c) 2025 ObjectStack. Licensed under the Apache-2.0 license.
 
 import { describe, it, expect, beforeEach, afterEach } from 'vitest';
-import { parseFilterAST, type FilterCondition } from '@objectstack/spec/data';
+import { markFilterSubtreeProvenance, parseFilterAST, type FilterCondition } from '@objectstack/spec/data';
 import { SqlDriver } from '../src/index.js';
 
 /**
@@ -78,8 +78,12 @@ describe('SqlDriver — null / empty operators (#2704)', () => {
       // ENGINE door gates on `isFilterAST` first and refuses this a layer
       // earlier (engine-filter-array-lowering.test.ts). Either way it throws;
       // what #2704 forbids is the whole-table answer.
+      // [#20020] The refusal is the INVALID_FILTER identity whatever the
+      // predicate's provenance; its wording names the operator only for a
+      // predicate marked 'author' (the #8220 contract), which this lowered,
+      // unmarked one is not — so the assertion is on the identity.
       await expect(lowered([['assignee', 'totally_bogus', null]]))
-        .rejects.toThrow(/Unsupported filter operator/);
+        .rejects.toMatchObject({ code: 'INVALID_FILTER', status: 400 });
     });
 
     it('count with is_null is scoped, not the whole table', async () => {
@@ -134,8 +138,11 @@ describe('SqlDriver — null / empty operators (#2704)', () => {
     });
 
     it('$regex is REFUSED, in the ADR-0112 envelope, naming $icontains', async () => {
+      // [#20020] Marked 'author', as a read-scope merge boundary marks a
+      // caller's own predicate: the operator and its replacement are named
+      // only then (the #8220 contract).
       const err = await driver
-        .find('tasks', { where: { assignee: { $regex: 'aro' } } })
+        .find('tasks', { where: markFilterSubtreeProvenance({ assignee: { $regex: 'aro' } }, 'author') })
         .then(() => null, (e: any) => e);
       expect(err).toBeInstanceOf(Error);
       expect(err.code).toBe('INVALID_FILTER');
@@ -161,9 +168,10 @@ describe('SqlDriver — null / empty operators (#2704)', () => {
     });
 
     it('unknown $-operator throws instead of a silent equality compare', async () => {
+      // [#20020] Identity, not wording — see the lowered-array case above.
       await expect(
         driver.find('tasks', { where: { assignee: { $bogus: 1 } } }),
-      ).rejects.toThrow(/Unsupported filter operator/);
+      ).rejects.toMatchObject({ code: 'INVALID_FILTER', status: 400 });
     });
   });
 });

packages/drivers/driver-sql/src/sql-driver-out-of-contract-filter-input.test.ts

@@ -58,6 +58,7 @@
 import { describe, it, expect, beforeEach } from 'vitest';
 import { SqlDriver } from './index.js';
 import type { FilterCondition } from '@objectstack/spec/data';
+import { markFilterSubtreeProvenance } from '@objectstack/spec/data';
 
 interface WireBearingError extends Error {
   code?: string;
@@ -185,7 +186,11 @@ describe('[#5347/#5348] SqlDriver refuses out-of-contract filter input', () => {
 
     for (const [label, value] of NON_BOOLEAN) {
       it(`refuses ${label} with INVALID_FILTER / 400`, async () => {
-        const err = await refusalOf({ stage: { $null: value } });
+        // [#20020] Marked 'author', as a read-scope merge boundary marks a
+        // caller's own predicate: the field, the value and the path are named
+        // only then (the #8220 contract). The withheld wording is pinned in
+        // `sql-driver-refusal-door-provenance.test.ts`.
+        const err = await refusalOf(markFilterSubtreeProvenance({ stage: { $null: value } }, 'author'));
         expect(err.code).toBe('INVALID_FILTER');
         expect(err.status).toBe(400);
         expect(err.message).toContain('Operator "$null" on field "stage" requires a boolean comparand');
@@ -253,7 +258,11 @@ describe('[#5347/#5348] SqlDriver refuses out-of-contract filter input', () => {
 
     for (const [label, value] of NON_BOOLEAN) {
       it(`refuses ${label} with INVALID_FILTER / 400`, async () => {
-        const err = await refusalOf({ stage: { $exists: value } });
+        // [#20020] Marked 'author', as a read-scope merge boundary marks a
+        // caller's own predicate: the field, the value and the path are named
+        // only then (the #8220 contract). The withheld wording is pinned in
+        // `sql-driver-refusal-door-provenance.test.ts`.
+        const err = await refusalOf(markFilterSubtreeProvenance({ stage: { $exists: value } }, 'author'));
         expect(err.code).toBe('INVALID_FILTER');
         expect(err.status).toBe(400);
         expect(err.message).toContain('Operator "$exists" on field "stage" requires a boolean comparand');