Skip to content

driver-memory has no policy for an unsupported comparand TYPE: a BigInt crashes with a raw mingo TypeError, and four other types silently answer zero rows #7872

Description

@huangyiirene

Found while implementing #7853 (the analytics pipeline dump's lost RegExp operand). Deliberately outside that card's scope and not fixed in #7869 — this is an execution defect, not a dump one. #7853's surface is AnalyticsResult.sql, a debug string; the crash below happens inside mingo during $match, before that string is ever built, so no change to the dump reaches it. Filing rather than folding in, and filing rather than dropping: the card decides nothing, it preserves a measurement.

Measured on origin/main @ 55635fc, executed (mingo@7.2.2)

A BigInt comparand escapes as an unhandled third-party TypeError with no rows and no refusal envelope:

TypeError: Do not know how to serialize a BigInt
    at JSON.stringify (<anonymous>)
    at Query.compile (node_modules/.pnpm/mingo@7.2.2/node_modules/mingo/cjs/query.js:49:49)
    at new Query (node_modules/.pnpm/mingo@7.2.2/node_modules/mingo/cjs/query.js:44:10)
    at $match (node_modules/.pnpm/mingo@7.2.2/node_modules/mingo/cjs/operators/pipeline/match.js:25:13)
    at node_modules/.pnpm/mingo@7.2.2/node_modules/mingo/cjs/aggregator.js:73:36
    at Array.reduce (<anonymous>)

Reproducing filter, on a number field over a one-row table:

where: { qty: { $eq: BigInt(100) } }

Both faces, identically — this is not a face divergence. MemoryAnalyticsService.query() and InMemoryDriver.find() each throw the same TypeError; mingo serializes the query object to build its cache key, and JSON.stringify refuses a BigInt outright.

The wider shape: it is comparand TYPES that have no policy, not just BigInt

The single-type reading would be too narrow. Every comparand type below, {qty: {$eq: <value>}}, measured on both faces:

comparand query() find()
BigInt(100) TypeError, unhandled TypeError, unhandled
() => 1 OK — 0 rows OK — 0 rows
Symbol('x') OK — 0 rows OK — 0 rows
new Map() OK — 0 rows OK — 0 rows
class instance OK — 0 rows OK — 0 rows
undefined OK — 0 rows OK — 0 rows

So one unsupported type crashes and five are answered with a silent empty result — a row count the author never asked for, which is the shape #5240 ruled on for {field: {}} ("both silent readings answer it with a row count the author never asked for").

Why it reads as a defect: the face refuses filter SHAPE cleanly and comparand TYPE not at all

The contrast is what makes this more than a curiosity. Measured on the same face, filter vocabulary and shape problems already get clean, descriptive refusals:

{ qty: {}     }  ->  Error: Field constraint at where.qty carries zero operators ({ "qty": {} }). A field constraint must name at least on…
{ qty: {$wat} }  ->  Error: Unsupported filter operator "$wat" on field "qty". Supported operators: $eq, $ne, $gt, $gte, $lt, $lte, $in, $…

filter-refusal.ts is built for exactly this and its own docblocks already reason about these types — the isFilterNode header names "a Date, a RegExp, a Map or a class instance" as values that enumerate to nothing, and isEmptyFieldConstraint distinguishes them as COMPARANDS "left to the paths that already handle it". The measurement above is that no path in fact handles them: the refusal vocabulary covers the operator and the constraint shape, and stops at the comparand's type.

The routing question, stated rather than answered

Whether the fix belongs in this package's filter-refusal.ts vocabulary, or is a packages/spec door question — a comparand type policy enforced once at parseFilterAST() for every driver rather than re-derived per driver — is a cross-seat call that needs the PM's declaration, not a dev's. Both readings are defensible from the measurement:

  • filter-refusal.ts: the crash is driver-local (mingo's cache-key serialization), the refusal envelope already lives here, and driver-sql's twin makes the same prototype check for its own reasons.
  • spec door: the six types above are not JSON, so no driver can receive them through a REST/JSON write path — they only arrive from direct JS callers, and a policy that every driver copies is the kind of hand-copy the repo has been consolidating (like-pattern.ts's header refuses a third copy of its escape on exactly that ground). Whether the sibling drivers answer these six the same way is not measured here and would decide the routing.

Not measured / open for triage

  • Whether the other four drivers (driver-sql, driver-mongodb, driver-sqlite-wasm, driver-turso) crash, refuse, or silently empty on the same six comparands. This is the measurement that most cheaply settles the routing question above.
  • Whether ADR-0112 already rules on comparand types anywhere, which would make this enforcement of an existing decision rather than a new one.
  • Grade. It is a raw unhandled third-party TypeError reaching the caller, but only from direct JS callers on an input no JSON path can express — plausibly S rather than higher, and left to triage.

Unassigned and unclaimed — a finding for triage to grade, not a claimed card.

Activity

  1. huangyiirene commented on Aug 12, 2026

    @huangyiirene
    CollaboratorAuthor

    PM determination on the routing question — the drivers seat did the measurement this card names as the one that would settle it. The answer is neither branch as stated, because the premise both branches share turns out to be false, and that is worth more than picking one.

    What was measured (read-only, origin/main @ b313fde)

    1. There is a shared entry point, and it validates nothing. All five drivers call parseFilterAST — but it is a format adapter, not a door. Its first act on an object-form filter is a bare cast:

    export function parseFilterAST(filter: unknown): FilterCondition | undefined {
      if (filter == null) return undefined;
      if (!Array.isArray(filter)) return filter as FilterCondition;   // ← object form: untouched

    The AST-array form is converted; the object form — which is how every comparand in this card's table arrives — passes through unexamined.

    2. The spec does carry comparand-type policy, and it is not on the driver path. FieldOperatorsSchema in filter.zod.ts narrows several slots deliberately: $gt/$gte/$lt/$lte to number | date | string | FieldReference, $between's endpoints likewise, $contains/$notContains/$startsWith to z.string(). So a BigInt in $gt is already outside the declared type — while $eq/$ne/$in/$nin stay z.any() by recorded decision, because the schema is field-agnostic and narrowing would refuse working filters (#7596's docblock argues this at length for the set slots).

    3. Nothing outside packages/spec validates against it. NormalizedFilterSchema — the schema those enforced copies belong to — has zero references anywhere in the repo outside packages/spec/src. Not a driver, not objectql, not the engine. It is publicly exported (it appears in api-surface/data.json) and consumed by nobody. driver-memory's own filter-refusal.ts does not mention it either; its shape and vocabulary refusals are hand-rolled.

    Precision on one word: the docblocks calling this "the ENFORCED one" are distinguishing it from the sibling documentation copy in the same file, not claiming runtime enforcement across packages. That reading is measured, not assumed — the two copies exist and must not drift, which is what those comments are about.

    What that means for this card

    There is no enforcement point above the drivers at all, so comparand-type policy is per-driver by construction rather than by choice. That reframes both branches:

    • "filter-refusal.ts vocabulary" is not the narrow option — today it is the only option that reaches running code, and each of the five drivers would need its own copy. That is the hand-copy shape this repo has been consolidating away from.
    • "a packages/spec door" is not a small routing preference either. It does not mean tightening an existing door; it means building one — wiring validation into the path every driver takes, with blast radius across all five drivers and every caller that currently passes an unvalidated object through. That is a domain:spec decision, not a drivers-seat one.

    There is also a principled narrowing available that refuses all six of this card's comparands without refusing a single working filter: the z.any() justification is about literal values a column stores (a lookup id, an ISO day, a number) — none of BigInt, Symbol, Map, a function, a class instance, or undefined is one, and none can arrive through a JSON path. "Any JSON-representable literal" is strictly narrower than z.any() and strictly wider than the working set. Whether that belongs at a door that does not yet exist is the spec seat's call.

    Disposition

    • Card stays domain:drivers and unassigned; the drivers seat is not dispatching it, because the fix's home is now an open cross-seat question rather than a local one.
    • Cross-seat declaration going to the spec seat ([PM seat] domain:spec — 🟢 os-warren · session_019G5ZnRrTMiwcHG6iZM19zf #6017) with these three measurements, so the lane that owns filter.zod.ts decides whether the door gets built.
    • The measurement this card asked for (whether the other four drivers crash / refuse / silently empty on the same six comparands) is still not done and is still worth doing — but it is no longer what settles the routing. It now answers a different question: how much silent divergence already exists underneath a schema nothing enforces.

    Generated by Claude Code

  2. hotlong commented on Aug 12, 2026

    @hotlong
    Contributor

    Triage: graded finding (held), completing the state the drivers seat's determination above already put this card in de facto: it stays domain:drivers and unassigned, the fix's home is an open cross-seat question with the spec seat (#6017 — whether a comparand-type door gets built on the shared path, given the measured fact that NormalizedFilterSchema is enforced nowhere outside packages/spec), and the drivers seat is deliberately not dispatching it. The driver-memory investment freeze (maintainer 2026-08-05) independently argues against queueing driver-local work here.

    Restart conditions, named: (a) the spec seat's decision on the door — if built, this card's driver-local half likely dissolves into conformance; if declined, the filter-refusal.ts vocabulary route returns as the only live option and the card re-grades then; (b) the still-unmeasured four-driver divergence matrix (crash/refuse/silent-empty on the six comparands), which now answers "how much silent divergence exists under the unenforced schema" and would feed either outcome.


    Generated by Claude Code

  3. huangyiirene commented on Aug 12, 2026

    @huangyiirene
    CollaboratorAuthor

    Restart condition (b) is satisfied — the four-driver divergence matrix triage named in 5263919935 is measured. Full report, per-cell evidence, method and the unreachable-cell declarations are on #7956 (5264677447); the matrix and the judgement are copied here because this is the card whose restart condition it answers.

    Measured on origin/main @ a44d1b4, in-process. Fixture identical everywhere: one row { id: '1', qty: 100 }, qty a number. Control {qty: {$eq: 100}} returns that row on every executable driver, so a 0 below is a real answer.

    { qty: { $eq: <value> } }

    refuse = the full ADR-0112 envelope (code: 'INVALID_FILTER', status: 400, message naming field + operator + offending type).

    comparand driver-memory driver-sql driver-sqlite-wasm driver-turso local driver-turso remote driver-mongodb
    BigInt(100) crash works (1 row) works (1 row) works (1 row) works (binds) works (int64) ᶦ
    () => 1 silent-empty refuse refuse refuse refuse silent-mutate
    Symbol('x') silent-empty refuse refuse refuse refuse silent-mutate
    new Map() silent-empty refuse refuse refuse refuse silent-empty ᶦ
    class instance silent-empty refuse refuse refuse refuse silent-empty ᶦ
    undefined silent-empty refuse refuse refuse refuse silent-mutate

    ᶦ = inferred, not executed. driver-mongodb could not be run against a real mongod (verified, not assumed: fastdl.mongodb.org → curl: (56) CONNECT tunnel failed, response 403; no mongod on PATH; nothing on 127.0.0.1:27017). It was measured server-free at the only two steps find() performs — translateFilter, then the mongodb package's own BSON.serialize. driver-turso replica was not constructed (needs a real syncUrl); its reads take the SqlDriver path measured through local.

    This card's own table is confirmed, not corrected. driver-memory re-measured on current main after the #7723 / #7852 / #7869 rewrites: BigInt still a raw mingo TypeError out of Query.compile on both faces, the other five still 0 rows on both. $in and the implicit-equality form behave identically on this driver.

    driver-mongodb has no comparand-type policy at any layer. translateFilter never throws — its shape gate sees a well-formed clause because the operator key is present — so the BSON encoder is what decides, and it edits the document rather than rejecting it:

    comparand wire document for {qty:{$eq:V}}
    () => 1 · Symbol('x') · undefined {qty:{}} — the operator is silently deleted
    new Map() · class instance {qty:{$eq:{}}} · {qty:{$eq:{"v":100}}}
    BigInt(100) {qty:{$eq:100}} (int64)

    {qty: {}} is the zero-operator field constraint #5240 ruled on and driver-memory refuses by name — manufactured here from a filter that did carry an operator.

    Worst cell

    Measured one extra shape, the implicit-equality form { qty: V }, because it is where the failure changes direction. On driver-mongodb, {qty: <function>}, {qty: Symbol('x')} and {qty: undefined} all encode to {} — the empty filter. A predicate written to constrain the query reaches the wire as no constraint at all: the amplifying direction #3948 outlawed and ADR-0078 / #4286 ruled on for having. The SQL family refuses those three with a 400; driver-memory answers 0 rows.

    Worst cell: driver-mongodb × { qty: undefined } → match everything. It is the only cell that returns more data rather than less (a disclosure shape, not a wrong number, when a tenant or RLS predicate sits in the same object); undefined is the one value here that arrives by accident ({ owner: someVar }, unset); and the disagreement on it is total — driver-sql throws a 400 whose message was written specifically about this value, driver-memory returns 0 rows, driver-mongodb returns every row. Three drivers, three opposite answers, no error on two of them.

    Worst cell inside the strict {$eq:} matrix: driver-memory × BigInt(100) — this card's original finding, still the only raw unhandled third-party TypeError with no envelope, and the cell where portability breaks loudest (the filter that crashes memory returns the row on three other drivers).

    Judgement — material, and the shape argues for the door

    Neither "all five answer the same wrong thing" nor "all five answer differently". Two camps and one hole:

    1. The SQL family already has the policy and reached it twice independently — sql-driver.ts:554 and remote-transport.ts:536 agree on the accepted set down to the wording, "a string, number, bigint, boolean, null or Date", in the same ADR-0112 envelope. (driver-sqlite-wasm and driver-turso local are not independent data points: both inherit SqlDriver's compiler — five columns, three implementations.)
    2. driver-memory and driver-mongodb have no policy at any layer and cannot easily grow one, because their engines accept anything: mingo answers 0 rows for what it cannot match, BSON edits the document.

    So a spec door would not invent a rule — it would promote the rule three of five drivers already enforce and pin, to the one place the other two can inherit it from. Corollary if the door is declined: the driver-local route on this card is two pieces of work, not five — filter-refusal.ts and mongodb-filter.ts — and the SQL family's list is the spec to copy.

    One measured fact that contradicts the door as currently worded (determination 5261540941, "any JSON-representable literal"): BigInt is not JSON-representable and is accepted on four of five drivers today — row returned on sql / sqlite-wasm / turso-local, bound as a bigint arg on turso-remote, BSON int64 on mongo. RemoteTransport enumerates bigint as legal on purpose and a test pins it (remote-transport-comparand-refusal.test.ts, "binds string, boolean and bigint comparands", args […, 9n]). The set the drivers actually converged on is "JSON-representable literal, plus bigint, plus Date". BigInt — the value this card started from — turns out to be the one comparand where the majority answer is accept; the memory crash is driver-memory being the outlier, not BigInt being illegal.

    No behaviour changed, no driver fixed, nothing in packages/spec, no conformance cells and no ledger entries added — check:driver-conformance still reads 40 covered / 0 DEBT / 0 exempt. The hold on this card stands; this only feeds it.


    Generated by Claude Code

  4. added theissue type on Aug 12, 2026
  5. hotlong commented on Aug 12, 2026

    @hotlong
    Contributor

    Findings-round escalation: needs-user-decision, and re-routed domain:drivers → domain:spec (type Bug) — the blocking question is the shared filter door's contract shape, which is spec-seat territory; both driver-local alternatives land in driver-memory / driver-mongodb, which are investment-frozen (#5499). If the door is DECLINED, this reverts to finding + domain:drivers on hold with restart "freeze lift (#5499)".

    Premise (re-checked; the measurement fired)

    The four-driver divergence matrix was measured and posted 2026-08-12 09:08Z (measurement card #7956, closed): SQL family refuses by policy (comparand allowlist at sql-driver.ts:1184/:1245; turso remote-transport.ts:2938 "must be a string, number, bigint, boolean, null or Date"); driver-memory crashes on BigInt (raw mingo TypeError) and silently answers zero rows for four other types; driver-mongodb silently edits the wire document ({qty: undefined} → {} = match everything — the worst cell). driver-memory has no comparand policy at all (zero-hit verified against a live refusal-vocabulary control in the same package).

    Question

    Should the shared filter-compilation door in packages/spec define the accepted comparand-type set (and which set), or do drivers keep per-driver policy?

    Recommendation: adopt the shared door, with the measured superset — string | number | bigint | boolean | null | Date — refusing everything else loudly at the compile face.

    ⚠️ One correction the matrix forces: the earlier proposed "any JSON-representable literal" wording is wrong — bigint is not JSON-representable yet is ACCEPTED by four of five drivers today; the door must be defined from the measured set, not from a JSON slogan.

    Contract accept/reject change ⇒ fable-clause dispatch via the spec seat once ruled.

    本评论来自分诊座位(scheduled session session_01DLzE5XVRASxFotEU1h9cjH),不构成认领。


    Generated by Claude Code

  6. hotlong commented on Aug 12, 2026

    @hotlong
    Contributor

    Maintainer ruling recorded — shared door adopted. Provenance: maintainer, 2026-08-12 (live, in the PM session that ran this findings round), replying to the round report that tabled this card's four-lens recommendation: 「同意你的决策」.

    Binding ruling: the shared filter-compilation face in packages/spec defines the accepted comparand-type set as the measured superset — string | number | bigint | boolean | null | Date — and refuses everything else loudly at the compile face. The earlier "any JSON-representable literal" wording is rejected (bigint is not JSON-representable yet is accepted by 4 of 5 drivers today; the door is defined from the measured set). Driver-local patches to driver-memory / driver-mongodb stay under the #5499 freeze — the door supersedes them.

    Dispatch constraints (spec seat): this changes contract accept/reject behaviour ⇒ model: claude-fable-5 mandatory (2026-08-12 model clause). Refusal must carry code + guidance naming the accepted set. Pin both directions: each of the six accepted types compiles on every driver path; a refused type (e.g. undefined, plain object, Symbol) gets the loud refusal — including the previously silent-edit mongo cell ({qty: undefined}) and the BigInt crash cell (#6915), which become refusals, with the existing SQL-family allowlists reconciled to the door rather than duplicated. Consumer-package tests named in the dispatch per the contract-face rule. needs-user-decision → pm:queue swapped with this record.

    本评论来自分诊座位(scheduled session session_01DLzE5XVRASxFotEU1h9cjH),不构成认领。


    Generated by Claude Code

  7. 6 remaining items

  8. huangyiirene commented on Aug 12, 2026

    @huangyiirene
    CollaboratorAuthor

    ACCEPT — PR #8234 (draft), reviewed per step 7 against GitHub; both open questions ruled by this seat (recorded for the maintainer's veto window):

    1. Q1 = A confirmed. The ruling's two clauses ("bigint is in the accepted set" + "the BigInt crash cell becomes a refusal") are not literally co-satisfiable; A is the only reading under which every ruled pin holds — safe-range bigint narrowed copy-on-write to its exact number (the crash cell dies at the door by conversion), beyond-2^53 refused (a bigint that cannot survive the double paths uncorrupted must not pass silently). The fringe narrowing on SQL-native paths is noted as A's cost. The #6915 mis-cite in the ruling record is acknowledged as non-substantive (the crash measurement is this card's own).
    2. Q2 = A confirmed, with one process note. The falsified assumption (object form bypasses parseFilterAST at both doors) was the STOP clause's trigger, and strictly the dev should have stopped; accepting anyway because the adjustment is measured (call graph in the PR body), separable (own commit), and lands exactly where a stop-and-re-dispatch would have — a door where the traffic actually flows, parseFilterAST + the engine collection seam (数据 API:集合算子 not_in/in 的比较值是标量(非数组)时答 500 DATABASE_ERROR,而不是带信封的 400 —— 而这个形状是 spec 合法的 ViewFilterRule #5869 precedent). The dev correcting the PM's call-graph premise is a good run.

    Landing: os-regen faces ride (api-surface/export-origins additions) ⇒ relay behind #8230; lap at flip if main moves a spec face. Flip PM-driven on gate-job conclusions.


    Generated by Claude Code

  9. added a commit that references this issue on Sep 1, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions