Skip to content

feat(spec): FILTER_TEXT_CASES declares what a text operator answers over a stored non-string value, and every face answers it (#14079) - #15686

Merged
os-project-manager merged 3 commits into
mainfrom
claude/issue-14079-filter-text-nonstring-value
Sep 5, 2026
Merged

os-project-manager merged 3 commits into
mainfrom
claude/issue-14079-filter-text-nonstring-value

Conversation

@claude

@claude claude Bot commented Sep 5, 2026 •

Copy link
Copy Markdown
Contributor

Fixes #14079

Maintainer ruling recorded on the card (comment 5548481659, decision batch #39, option A): a stored value that is not a string never satisfies a positive text operator and satisfies $notContains — complementarity holds, on every face. This PR is that ruling landed as one contract row set that every face answers. Option C (a declared-type door) is deferred to its own decision card, which the PM files; nothing here builds it and nothing here forecloses it.

CORRECTION — patch lap, head 29903ee1d (⛔ not a quiet edit of the readings below)

What went wrong. The first lap's service-analytics reading (9 targeted files, 265 tests, at e92f5a89a) was a narrowing that was not declared as one. It did not include src/__tests__/objectql-echo-operator-coverage.test.ts, which reaches the ObjectQLStrategy operator renderer DIRECTLY with the four-argument shape it has always had (buildFilterClauseSql('stage', operator, ['w'], []), through a local mirror of the private signature). Lap one made target and ctx REQUIRED on that renderer, so the direct call handed undefined and the [#14079] gate threw TypeError: Cannot read properties of undefined (reading 'object') on target.object. CI: Test Core (4/6) (run 33940680441, job 101237407118) and Temporal Conformance (live PG + MySQL) (job 101237406991) both red on e92f5a89a, one root cause — Test Files 1 failed | 89 passed (90), Tests 1 failed | 1982 passed (1983). Reproduced locally before the fix with the WHOLE package: pnpm --filter @objectstack/service-analytics test under the lock, VERDICT command-exit 1, the same one failure at objectql-echo-operator-coverage.test.ts:388.

The fix (producer side, 29903ee1d). target and ctx are optional on buildFilterClauseSql; the declared-type gate applies only when both are handed in. A caller that hands no target cannot be asked the declared-type question and keeps the LIKE it always got — the "cannot answer, do not block" posture nonTextColumnResolver already takes for a context with no hook. renderFilterNodeSql still always passes both, so every statement the echo prints through its public door is unchanged from lap one. The coverage suite was NOT edited: its contract (a four-argument direct call, every emittable operator renders a predicate) is genuinely unchanged. No other direct caller of buildFilterClauseSql / renderFilterNodeSql / buildFilterClause exists in src or tests (git grep over the package; the remaining mentions are CHANGELOG prose and comments). The posture is pinned in text-operator-non-text-column.test.ts: the renderer called without a target keeps its LIKE and binds the escaped pattern.

Readings now, at 29903ee1d. Whole package: pnpm --filter @objectstack/service-analytics test under the lock — VERDICT command-exit 0 · held the lock 70s, Test Files 90 passed (90), Tests 1984 passed (1984) (1983 + the new pin). typecheck (service-analytics) exit 0. Targeted eslint --no-inline-config over the two patched files: 0 errors, 0 warnings. Gates re-derived with node scripts/pm/dispatch-gates.mjs (no paths) at 29903ee1d: 41 named (the first lap's 37 plus check:driver-memory-census, check:refd-timer-probe, check:watch-hint-literal, check:nul-bytes); every one exit 0 — check:doc-formula-expressions first answered exit 3 (Nothing was measured: @objectstack/lint had no dist in the re-created worktree), re-run green after building that package's closure — except check:dual-build-cjs-loads, still PREREQUISITE NOT MET (no package scope; 50 packages unbuilt locally) and therefore NOT MEASURED here, declared to CI as on lap one.

What landed

  • @objectstack/spec — FilterTextRow gains its first non-string column, score: number (a 0 among the nine values, so a truthiness guard is caught too). Five new evaluated FILTER_TEXT_CASES rows over it: $contains / $startsWith / $endsWith / $icontains answer [], $notContains answers all nine. NON_TEXT_STORED_VALUE_TYPES (field-value.zod.ts) — the numeric and boolean value classes — is the one list the SQL faces classify a column by, since they cannot read the value at compile time. The spec's own reference evaluator reads the filter's field and type-gates; a coercing evaluator is proved to fail exactly the five new rows.
  • @objectstack/driver-memory — the reference matcher's $notContains arm answers the predicate (typeof value === 'string' && value.includes(target) fails the row) instead of the type test that failed both polarities. The no-value cells keep their driver-memory's reference matcher still answers $notContains / $nin the pre-ruling way on a no-value row — the #5499 freeze that excused it dissolved 2026-08-11, so the divergence is now unexcused and untracked #13166 answer.
  • @objectstack/driver-sql — SqlDriver.isNonTextColumn reads the numericFields / booleanFields registries initObjects and registerExternalObject already fill; applyFilterCondition routes the seven text operators over such a column to applyTextOperatorOverNonTextColumn, which emits 1 = 0 for a positive operator and 1 = 1 for $notContains — after every comparand refusal, before either emitter, on every dialect. driver-sqlite-wasm and turso local inherit it.
  • @objectstack/driver-turso — the remote transport keeps no schema, so TursoDriver hands it the same rule (setNonTextColumnResolver, the shape of the existing temporal setFilterColumnSql), answered from the registries registerRemoteFieldMetadata fills at schema sync; each text arm asks it after its own comparand gate.
  • @objectstack/service-analytics — compileScopedFilterToSql(filter, alias, options?) takes an optional nonTextColumn(field) predicate; DatasetScopedStrategyContext.declaredFieldType exposes the field type the service already holds (sourceFieldMeta); both strategies pass it for the RLS read scope, and (declared in-place addition, see below) NativeSQLStrategy.buildFilterClause and the ObjectQLStrategy echo apply the same test to the query's own text filters, so a query and its scope answer one cell one way and the echo prints the statement that ran. A host that wires no field metadata keeps the LIKE it always got.
  • @objectstack/objectql — having-filter.ts:415-422: the sentence "which driver-sql's polarity table already follows for the same operator" is corrected to what is true (the polarity table covers the no-value cell only; the non-string cell is now the contract row every face answers). Comment-only; no changeset.
  • Every suite that materialises the fixture adds the column (SQL initObjects DDL included); generated followers api-surface/data.json and export-origins/data.json regenerated by check:generated --fix (one added export each).

Compile-surface declaration — every face, one conclusion

face conclusion evidence
(1) driver-sql applyFilterCondition / textMatchPredicate / likePatternPredicate changed — declared-type gate ahead of both emitters sql-driver-text-case-conformance.test.ts: the five rows on the sqlite cell AND the live PostgreSQL 16.13 cell; compiled-shape pins on sqlite / pg / mysql probes (where 1 = 0, where 1 = 1, no LIKE / GLOB / CAST built against the column); $not composition; comparand refusals still ahead of the constant; an unregistered table keeps its LIKE
(1a) driver-sqlite-wasm (inherits) changed by inheritance sqlite-wasm-icontains-and-retired-operators.test.ts drives the whole table on sql.js with score REAL
(1b) driver-turso local (inherits) changed by inheritance turso-local-remote-text-parity.test.ts
(2) driver-turso RemoteTransport.buildWhereSQL (pushLike / pushLikePattern) changed — injected resolver, constant pushed inside each arm after its comparand gate turso-local-remote-text-parity.test.ts: the five rows on both transports, LOCAL vs REMOTE compared first; $like / $ilike pins over score on both
(3) service-analytics read-scope-sql.ts compileScopedFilterToSql changed — optional nonTextColumn predicate; TRUE_CLAUSE added beside FALSE_CLAUSE text-operator-non-text-column.test.ts: compiled text (1 = 0 / 1 = 1, NOT (("t"."score" IS NOT NULL AND 1 = 0))), params alignment beside a text column, no-option byte-identity, comparand refusal ahead of the constant; executed through applyReadScope on sql.js
(4) service-analytics filter-normalizer.ts lowerAnalyticsWhere already compliant as a lowering — it produces a FilterCondition; its SQL emission is NativeSQLStrategy.buildFilterClause plus the echo, which are changed (in-place addition, below) same suite: the shared table's five rows executed through NativeSQLStrategy.generateSql on sql.js over a REAL column; the echo prints the same constant
(5) formula matchesFilterCondition already compliant — typeof actual === 'string' guards on every positive arm, !(…) on $notContains (measured on the card, 5542868802) matches-filter-icontains.test.ts now drives the table's five score rows through it, plus $like / $ilike and a boolean
half-face objectql having (applyHaving / matchesHaving) already compliant — typeof value === 'string' && value.includes(target) on $notContains (#5905); docblock sentence corrected having-filter-text-conformance.test.ts drives the whole table (rows spread the new column)
driver-memory reference matcher checkCondition changed (the card) memory-filter-text-conformance.test.ts: whole table on the query path, the matcher, and face agreement; the $notContains-over-score case named as the one legitimate whole-set answer in the WIDENS pin; $like / $ilike pins on both faces
driver-memory live mingo path / analytics face already compliant — negated regex never matches a number (measured on the card) same suite; the analytics cube declares score as number, expressible subset 12 → 15
driver-mongodb translateFieldOperators already compliant — { $regex } is string-only in MongoDB and { $not: { $regex } } inverts it mongodb-filter-text-conformance.test.ts drives the five rows through its strict in-process matcher with zero code change; real mongod NOT MEASURED

Which faces ran against a real engine, and which are modelled

  • Real: SQLite via better-sqlite3 (driver-sql, driver-turso local), sql.js (driver-sqlite-wasm, service-analytics), libsql-over-SQLite stub (driver-turso remote transport — the transport's real compiler on a real SQLite engine, not a Turso cloud endpoint); PostgreSQL 16.13, a real server brought up in-container (/usr/lib/postgresql/16, OS_TEST_POSTGRES_URL on port 54329) for the driver-sql conformance suite's live cell — 26 tests on that cell including the five non-string rows; its statement log shows select * from "os6518_text_case" where 1 = 0 and … where 1 = 1 against a "score" real column, "name" LIKE $1 ESCAPE $2 beside them, and zero SQLSTATE 42883 — the 500 became a result; driver-memory's three faces; formula; having.
  • Modelled / NOT MEASURED: MySQL (no server; the compiled shape is pinned via the knex mysql2 probe: where 1 = 0 / where 1 = 1, no CAST); a real mongod (proxy blocks the binary; the emitted document is evaluated by the suite's strict matcher, the same substitute that package states); a Turso cloud endpoint.

The three hypotheses on the dispatch

  • H1 holds for driver-sql (the registries) and was falsified as stated for the other two faces, then resolved without a silent coerce: RemoteTransport keeps no schema (syncSchema reads one and retains nothing) — the driver injects the rule, the shape the temporal seam already uses; compileScopedFilterToSql(filter, alias) had no schema and the spec's StrategyContext carries none — the service already holds sourceFieldMeta, now exposed as the package-local declaredFieldType hook (declared on DatasetScopedStrategyContext, not on the spec contract, for the reason getDatasetScope is).
  • H2 holds: the constants compose with the 非否定路径上的 $ne / $nin / $notContains:driver-sql 排除 NULL 行,driver-memory / formula 返回它们(#5146 只裁定了 $not) #5298 null polarity ($notContains admits a NULL row already) and with the $not rewrite (NOT (col IS NOT NULL AND 1 = 0) is TRUE everywhere; NOT (col IS NULL OR 1 = 1) is FALSE) — pinned on driver-sql (pg probe), read-scope-sql and the native strategy (executed).
  • H3 — consumer list re-derived in-branch: identical to the dispatch's 22-file list; having-icontains.test.ts spreads its own n column over the rows (untouched); search-filter.test.ts, filter-ascii-fold.test.ts, filter-comparand-type-conformance.ts, scripts/check-driver-conformance.mjs and filter.zod.ts reference the table by name only and were not edited (no sentence in filter.zod.ts became false — its docblocks describe folding and literalness).

Judgment singled out for review, with its rollback

The ruling names read-scope-sql; the analytics where face (NativeSQLStrategy.buildFilterClause + the ObjectQLStrategy echo) is the dispatch's compile surface (4) and was fixed in place under the bounded exemption: same defect class (a LIKE over a numeric dimension coerces on SQLite and is refused on Postgres), mechanical with the shape pinned by the ruling, no other claim on those files (PM's 01:49Z in-flight check), same gate family. Without it a query's where and its RLS scope would answer one cell two ways on the same request. Rollback: revert the if (shape) { gate in native-sql-strategy.ts, the if (like) { gate plus the target / ctx parameters and resolveStorageTarget in objectql-strategy.ts, and the two nonTextColumnResolver / textOperatorPolarity imports; the read-scope option, the hook and non-text-column.ts stand on their own.

Deviations from the letter of the ruling, stated

Pin sweep (repo-wide, both sentences)

Grepped every *.test.ts / *.testkit.ts for text operators beside non-string / numeric / typeof / a numeric comparand. Found and flipped: driver-memory/src/memory-matcher-no-value-negated-operators.test.ts ("a present non-string value still fails $notContains on the type test", toEqual([])) — now asserts the row IS in the negation, is NOT in the positive twin, and that 0 / false behave the same. Kept verbatim (already the ruled direction): objectql/src/having-icontains.test.ts:65 (positive over a number answers []), formula/src/matches-filter-like.test.ts:69 (positive $like over a number answers false). Unchanged and green: driver-memory/src/memory-driver-filter-logic-conformance.test.ts (temporal-column text pins), service-analytics/src/__tests__/native-sql-datetime-filter-column.test.ts.

Test readings, per consumer package, at e92f5a89a (one lock hold, after the final commit)

package files result
@objectstack/spec filter-text-conformance, field-value, filter-ascii-fold, filter-comparand-type-conformance 65 passed
@objectstack/driver-memory 8 files (text conformance, icontains, like-pattern, four matcher suites, filter-logic conformance) 301 passed
@objectstack/driver-sql (sqlite + live PostgreSQL 16.13) text-case conformance, icontains-and-retired-operators, like-pattern 132 passed, 1 skipped (the unprovisioned MySQL cell, by name)
@objectstack/driver-sqlite-wasm icontains-and-retired-operators 32 passed
@objectstack/driver-turso text parity, remote text predicates, like parity, null parity 112 passed
@objectstack/driver-mongodb text conformance, filter, icontains 88 passed
@objectstack/objectql having text conformance, having-icontains, search-filter, having-filter 83 passed
@objectstack/formula matches-filter-icontains, matches-filter-like, matches-filter 61 passed
@objectstack/service-analytics 9 files (the new suite, like-metacharacter-escape, four read-scope suites, datetime filter column, value-type fidelity, canonical operator) 265 passed

Also: the whole @objectstack/driver-sql package once on SQLite (154 files, 2377 passed, 140 skipped). typecheck green on spec, driver-memory, driver-sql, driver-turso, service-analytics, objectql, formula (spec's check:test-typecheck included). Targeted eslint --no-inline-config over the 23 changed .ts files: 0 errors, 0 warnings (a targeted run, not the repo-wide gate — CI owns that one).

Gates (derived with node scripts/pm/dispatch-gates.mjs, no paths, at f4f2d0c90; re-run at e92f5a89a)

All 37 derived gates green (17 @objectstack/spec / @objectstack/lint gates including check:api-surface, check:export-origins, check:generated — "All 15 generated artifacts are up to date" — check:docs, check:liveness, check:exported-any; 20 repo-level gates including check:driver-conformance, check:cross-package-test-inputs, check:test-source-alias, check:merge-driver, check:changeset-gate-self-tests), plus check:nul-bytes. One derived gate is NOT MEASURED locally: check:dual-build-cjs-loads answers PREREQUISITE NOT MET (it reads every package's dist and 50 packages were not built here; it has no package scope) — CI runs it after a full build.

Changesets

@objectstack/spec minor · @objectstack/driver-memory patch · @objectstack/driver-sql minor · @objectstack/driver-turso minor · @objectstack/service-analytics minor — each with the ruled note that a text operator over a non-text column now answers the declared row instead of a dialect accident (Postgres: a 500 becomes a result).

Out of scope, filed unassigned (both remain open)

🤖 Generated with Claude Code


Generated by Claude Code


Generated by Claude Code

…ll; every face answers it

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01M59rPZZFzqhfMUPFqqZTkf
…generate spec artifacts

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01M59rPZZFzqhfMUPFqqZTkf
@github-actions

github-actions Bot commented Sep 5, 2026 •

Copy link
Copy Markdown
Contributor

📓 Docs Drift Check

This PR changes 6 package(s): @objectstack/driver-memory, @objectstack/driver-sql, @objectstack/driver-turso, @objectstack/objectql, @objectstack/service-analytics, @objectstack/spec, touching 41 documentable anchor(s). ⚠️ 2 changed file(s) yielded no anchor (packages/spec/api-surface/data.json, packages/spec/export-origins/data.json), so the pages documenting them are NOT COVERED by this run — this is not a clean bill of health for those files.

11 hand-written doc(s) NAME something this change touched and may need an implementation-accuracy re-verification:

  • content/docs/data-modeling/drivers.mdx (via SqlDriver (symbol, a top-level class))
  • content/docs/data-modeling/formulas.mdx (via endsWith (literal, a string literal in textOperatorPolarity), startsWith (literal, a string literal in textOperatorPolarity))
  • content/docs/data-modeling/index.mdx (via SqlDriver (symbol, a top-level class))
  • content/docs/data-modeling/validation.mdx (via endsWith (literal, a string literal in textOperatorPolarity), startsWith (literal, a string literal in textOperatorPolarity))
  • content/docs/permissions/tenant-audit-census.mdx (via SqlDriver (symbol, a top-level class))
  • content/docs/plugins/packages.mdx (via SqlDriver (symbol, a top-level class))
  • content/docs/protocol/kernel/index.mdx (via SqlDriver (symbol, a top-level class))
  • content/docs/protocol/kernel/lifecycle.mdx (via SqlDriver (symbol, a top-level class))
  • content/docs/protocol/objectql/query-syntax.mdx (via FILTER_TEXT_CASES (symbol, a top-level const object), SqlDriver (symbol, a top-level class))
  • content/docs/protocol/objectql/schema.mdx (via endsWith (literal, a string literal in textOperatorPolarity), startsWith (literal, a string literal in textOperatorPolarity))
  • content/docs/protocol/objectql/types.mdx (via startsWith (literal, a string literal in textOperatorPolarity))

⛔ 2 release-owned page(s) also name something this change touched. These are read-only:

  • content/docs/releases/v14.mdx (via generateSql (symbol, a method of class ObjectQLStrategy))
  • content/docs/releases/v17.mdx (via ObjectQLStrategy (symbol, a top-level class), SqlDriver (symbol, a top-level class), generateSql (symbol, a method of class ObjectQLStrategy))

content/docs/releases/ is RELEASE-OWNED (AGENTS.md "Documentation Guardrails"): release
notes are written centrally at release time, and a code PR that edits them is the exact PR
that guardrail exists to stop. They are still audited — read-only. If one of them is actually
wrong, file an issue or open a dedicated docs-only PR; do not edit it here.

What this run could not see
  • 2 changed file(s) yielded no anchor (packages/spec/api-surface/data.json, packages/spec/export-origins/data.json) — pages documenting those are invisible to this run
  • 5 name(s) were too generic to anchor anything (single lowercase words)
  • the SDK route bridge reached 61 of 219 client-bound route-ledger rows — the other 158 have no registrar path: tail to select them, so pages documenting THEIR client methods cannot appear above, on this or any run. Of those 158: 0 are remediable by widening that discovery convention (an in-repo file declares the path; the convention did not scan it); 56 are structural — on a ledger where NOT ONE row is declared in-repo, so no discovery change reaches them at any price; 102 are undecided (no in-repo declaration, on a ledger that has other in-repo registrars — absence and an unreadable spelling are not distinguishable here). The rows themselves: node scripts/docs-audit/affected-docs.mjs --bridge-coverage
  • a page that states a rule by its inputs shares no identifier with the emitter that implements the rule, so an emitter-only diff cannot list it — not on this run and not on any run. Measured on fix(driver-sql): emit varchar(maxLength) for a text field a declared index keys on #11430: content/docs/protocol/objectql/types.mdx documents the text-family column mapping by the ObjectQL type names it maps FROM (text / textarea / html) while the diff changed createColumn; it went unlisted, and it was the page that diff falsified, in four places. No shared token exists to detect this on, so a rule your change carries has to be re-read by hand in the pages that restate it.

Coarse fallback — 134 page(s) merely mention a changed package (the pre-#9192 predicate, kept for the deliberately-wide backstop): node scripts/docs-audit/affected-docs.mjs --json 95d5cbb31687558ac63dfdde90950d06f82820eb → packageMentionDocs.

Which tree this was computed on

This run read content/docs from f1dd1846b60a56049ef7ce69ef270257c25de5f8 — the merge of head 29903ee1d0baf653044e0f4e8c42318df26be55e into base 95d5cbb31687558ac63dfdde90950d06f82820eb, which is what actions/checkout gives a pull_request run. Not the PR head.

A worktree cut from an older main holds a different content/docs, so re-deriving there can legitimately return a different list — that is a different tree, not a wrong row. To answer on the same tree:

# while this PR is open — GitHub drops the merge commit once it closes
git fetch origin f1dd1846b60a56049ef7ce69ef270257c25de5f8 && git checkout f1dd1846b60a56049ef7ce69ef270257c25de5f8
# afterwards, rebuild it from the two parents, which stay fetchable
git fetch origin 95d5cbb31687558ac63dfdde90950d06f82820eb 29903ee1d0baf653044e0f4e8c42318df26be55e && git checkout -B drift-repro 95d5cbb31687558ac63dfdde90950d06f82820eb && git merge --no-ff 29903ee1d0baf653044e0f4e8c42318df26be55e

node scripts/docs-audit/affected-docs.mjs --json 95d5cbb31687558ac63dfdde90950d06f82820eb

⚠️ That checkout carried uncommitted changes, so the commit above does not fully identify what was read.

Advisory only, and a precision-first one (#9192): a page is listed because it names a
symbol, wire route or SDK method this diff touched — not because it mentions a changed
package. Each row says which anchor put it there, so a wrong row is reportable rather than
merely annoying. To re-verify, run the docs-accuracy-audit workflow scoped to these files:
node scripts/docs-audit/affected-docs.mjs 95d5cbb31687558ac63dfdde90950d06f82820eb → pass the list as
args.docs, on the commit named under Which tree this was computed on.

Copy link
Copy Markdown
Collaborator

CI red on e92f5a89a — Test Core (4/6) (run 33940680441): src/__tests__/objectql-echo-operator-coverage.test.ts:388 — TypeError: Cannot read properties of undefined (reading 'object') when the echo-coverage suite renders contains through the private operator renderer without the new target argument this PR added. This PR's defect, not the base's (the suite is unchanged on main and the method signature moved here). A patch lap is dispatched to the implementing dev on the same branch: tolerate a missing target/ctx (keep the LIKE), reproduce with the whole @objectstack/service-analytics suite, push, and record the correction in the PR body. Landing stays deferred; the card's ACCEPT (5548928220) stands pending the green head.

domain:spec seat, session_01M59rPZZFzqhfMUPFqqZTkf, 2026-09-05T03:10Z.


Generated by Claude Code

…ct call without a target or context

The declared-type gate applies only when both are handed in; the coverage suite's four-argument direct call keeps its LIKE instead of throwing on target.object.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01M59rPZZFzqhfMUPFqqZTkf
@os-project-manager
os-project-manager marked this pull request as ready for review September 5, 2026 03:52
@os-project-manager
os-project-manager added this pull request to the merge queue Sep 5, 2026
Merged via the queue into main with commit a646120 Sep 5, 2026
53 checks passed
@os-project-manager
os-project-manager deleted the claude/issue-14079-filter-text-nonstring-value branch September 5, 2026 05:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation protocol:data size/xl tests tooling

Projects

None yet

Development

Successfully merging this pull request may close these issues.

driver-memory's reference matcher answers $notContains NO for every valued NON-STRING row — the live mingo path answers YES

2 participants