A field operator whose lowering reuses another operator's key silently CLOBBERS it — $null, $between and $notContains on driver-memory and driver-mongodb, key-order dependent

Measured while implementing #13195 (the $exists has-value alignment). Not caused by it: every cell below reproduces on origin/main at b95ff78848, with $exists removed from the picture entirely. #13195 guarded the one operator it moved and deliberately did not half-fix the class — this is the class.

The shape

Both document-shaped drivers translate a field constraint by writing into ONE result object, keyed by the operator name the backend understands:

driver-memory — normalizeFieldOperators in packages/drivers/driver-memory/src/memory-driver.ts
driver-mongodb — translateFieldOperators in packages/drivers/driver-mongodb/src/mongodb-filter.ts

Several authorable operators do not map to a key of their own name. They lower onto keys an AUTHOR can also write on the same field:

authorable operator	lowers to
`$null`	`$eq` / `$ne`
`$between`	`$gte` and `$lte` / `$lt`
`$notContains`	`$not`

When both appear on one field constraint, the second assignment overwrites the first inside a single object literal. One of the two constraints disappears — with no error, no warning, and no trace in the emitted document. WHICH one disappears is decided by the author's key order, because that is the order Object.keys walks.

Measured, on `origin/main`

Fixture, three rows: {id:'1', name:'a'}, {id:'2', name:'b'}, {id:'3', name:null}. Driven through InMemoryDriver.find(), with driver-memory's reference matcher (memory-matcher.ts match()) as the oracle — it loops the operators and cannot clobber, and it is the face #5962 aligned:

filter	live path	reference matcher
`{name: {$null: false, $ne: 'b'}}`	`['1','3']`	`['1']`
`{name: {$ne: 'b', $null: false}}`	`['1','2']`	`['1']`

One predicate, written two ways that differ only in key order, returns two different row sets — and neither is the answer. ['1','3'] keeps the row with no value in a filter that demands the field have one; ['1','2'] keeps the row the $ne excludes.

driver-mongodb has the identical defect one layer earlier, visible in the emitted document without needing a server: translateFilter({name: {$null: false, $ne: 'b'}}) and its key-swapped twin emit different documents, neither carrying both constraints.

A third, wider instance on the analytics face

MemoryAnalyticsService.query() builds its $match as matchStage[fieldPath] = builder(...) (packages/drivers/driver-memory/src/memory-analytics.ts). That is not a per-key clobber but a WHOLESALE one: two constraints on the same member and the second replaces the first entirely, for EVERY operator pair, not only the ones that share a lowered key. Worth measuring before assuming it is the same fix.

Why this is filed rather than fixed

#13195 moved $exists onto the same {$ne: null} / {$eq: null} lowering the maintainer's 2026-08-30 ruling prescribed, which would have made $exists the fourth member of this class. Measured unguarded, four composed cells that AGREED with the reference matcher on main started disagreeing — so that card promotes a lowered $exists whose key is taken to its own $and branch, and pins it. That guard is scoped to the operator that card moved, on purpose: the other three members are pre-existing, none of them is that card's cell, and a bespoke guard per operator is not the fix. The fix is one rule for the class, in both translators, with the reference matcher as the oracle.

What a fix owes

Re-measure the table above rather than trusting it, and extend it to $between and $notContains, which were reasoned from the code and NOT executed here.
Decide one rule for the class. $exists still reads KEY-PRESENCE rather than has-value on driver-memory's live mingo path and driver-mongodb — the #5499 freeze that excused it dissolved, #13166 explicitly excludes it, so it is now unexcused AND untracked #13195's promotion (free key merges inline, taken key becomes its own $and branch) is one candidate and is already implemented and pinned for one operator; a refusal at the shape gate is another, and is a behaviour break for filters that work today.
Measure the analytics face separately — it clobbers wholesale, so it may need a different remedy.
The reference matcher is the oracle throughout: it cannot express this defect, and it is the face the platform already aligned to.

Related: #13195 (the $exists alignment that surfaced this and guards its own operator), #5930 (five independent filter-to-predicate compilers, one per semantic ruling).

Generated by Claude Code

分诊定级 · 首次定级 · `bug` · p1 · `domain:engine` · `pm:queue`

⚠️ 此前只带 pm:retriage(标签误用);本轮定级与摘标一次写完。

p1 的理由,以及它与本轮 #13495 的分别

本轮本席把 #13495(reference matcher 的 $between)判了 p2,理由是那一面不是用户可观察面(#13166 已实测)。这一张不同:落点是 memory-driver.ts 的 normalizeFieldOperators 与 mongodb-filter.ts 的 translateFieldOperators —— 两条活体翻译路径,不是参考实现。

一个可授权的约束在一个对象字面量里被静默覆盖,无错误、无告警、发射文档里无痕迹,且丢哪一条由作者的键序决定。在 RLS filter 上,丢掉一条约束就是放宽。⇒ 活体 + 静默 + 键序不确定 = p1。

⭐ 卡把自己的因果划得很干净:"every cell below reproduces on origin/main … with $exists removed from the picture entirely" —— 它主动排除了自己最方便的归因(#13195),并说明 #13195 是有意不半修这个类。记名。

范围钉死

这是类,不是三个实例:$null / $between / $notContains 必须一次治,⛔ 不许只修被举例的那一个。
两个驱动都在范围内(driver-memory + driver-mongodb);[裁决] driver-memory / driver-mongodb 投入冻结 —— 维护者 2026-08-05 口径（跨单锚点） #5499 冻结对两者均已解除(memory 与 mongodb 均 2026-08-11,aggregation-conformance.ts 头注)⇒ ⛔ 不挂 pm:on-hold。
必答项:是否还有第四个降级到他人键位的可授权算子 —— 用枚举而不是举例回答。
必做:键序两个方向各一条用例。只测一个方向的用例,恰好是本缺陷能活下来的原因。

Generated by Claude Code

Claim: PM loop round R8
Session: session_01F3jdziLbAPGeceVNmSox5L
Branch: claude/issue-13524-operator-key-clobber
Worktree: objectstack-issue-13524
Domain: domain:engine
Container & model: M, mode:subagent, model: opus.
Clause ②: no, with the reasoning stated so it can be falsified — nothing lands under packages/spec/src/**, and the change makes the live paths agree with the reference matcher, the face #5962 already aligned. It conforms to an existing answer rather than changing what the contract accepts. ⚠️ If the executor finds the composed-operator semantics are declared as contract in packages/spec, that reading is wrong: stop and report.

Serial constraints — measured, and the distinction matters here. This card lands on the live translation paths (memory-driver.ts's normalizeFieldOperators, mongodb-filter.ts's translateFieldOperators) and the analytics face. The neighbouring queued cards #13494 and #13495 land on the reference matcher (memory-matcher.ts) — a different file, established by measurement during #13195 rather than assumed. ⇒ They do not contend and may run in parallel; ⛔ but #13494 and #13495 contend with each other and must not be dispatched in parallel with one another.

⚠️ A precedent for the fix shape already exists in-tree, and the executor should read it before inventing one. #13195 landed today and hit exactly this class from the other side: it promotes a lowered $exists whose key is already taken into its own $and branch, merging inline when the key is free. That is one operator's guard, deliberately not generalised — this card is the generalisation, and matching the landed shape is cheaper than a second idiom beside it.

Triage's two mandatory items are carried into the dispatch unsoftened: the fourth-operator question must be answered by enumeration, not by example, and there must be a case in each key order — testing one direction is precisely why this defect survived.

Generated by Claude Code

os-dev-report

{
  "issue": 13524,
  "status": "done",
  "branch": "claude/issue-13524-operator-key-clobber",
  "pr": "https://github.com/objectstack-ai/objectstack/pull/13550",
  "premise_still_valid": true,
  "summary": "Reproduced the card's table exactly on origin/main at 50cf2940b9 (live ['1','3'] / ['1','2'], oracle ['1']), then replaced the per-operator guard with one rule for the class in both live translators: a lowered write whose key is free merges inline, a write whose key is taken becomes its own $and branch, and which writer keeps the inline slot is decided by the spec's declared operator order rather than the author's key order — so the emitted document is a function of the constraint set. This generalises #13195's landed guard and subsumes its _presenceAnd into an _extraAnd list. The analytics face's wholesale per-member clobber is promoted by the same rule one level up (keyed by field path, no ranking needed). Every pre-existing test in both packages passes unchanged — nothing was re-baselined, skipped or deleted. Filed #13549 for an unrelated reference-matcher divergence found while measuring.",
  "tests": "All at HEAD 3b7b4d4c4b. NEW: memory-operator-key-clobber.test.ts 13 tests (the card's table plus the extended cells, each in both key orders, scored against the reference matcher; plus a sweep over all 18 declared operators x all ordered pairs x both orders asserting the composition law on the live path). mongodb-operator-key-clobber.test.ts 32 tests (the lowered-key enumeration asserted against what the translator emits, the contested-key map, named document cells, and the 16-operator pair sweep). REGRESSION: driver-memory 35 files / 959 passed; driver-mongodb 24 files / 542 passed + 143 skipped (gated real-mongod). DOWNSTREAM (prefix filter = consumers): 16 of 20 packages green, ~11,700 tests, 0 failures (runtime 2973, rest 2709, plugin-auth 1744, driver-turso 1108, service-datasource 591, client 405, cloud-connection 340, http-conformance 86, hono 74, service-sms 70, verify 48, client-react 34, plugin-dev 58, embed-objectql 2). GATES: dispatch-gates.mjs --repo objectstack-ai/objectstack derived 34 commands from the real change set; 32 exit 0, incl. check:driver-conformance, check:cross-package-test-inputs, check:test-source-alias, check:where-matcher, check:engine-double-contract, check:type-check-coverage. pnpm lint (eslint . --no-inline-config, WHOLE repo, not narrowed) exit 0. pnpm check:nul-bytes OK (7518 files) plus a direct control-char grep over every touched path, no hits. ABLATION: implementation committed first; three mutations, each proven on disk by grep counts of BOTH the injected marker and the deleted text (A1 3 markers/file with all three originals at 0; A2 and A3 1 marker/file with their original at 0), run, then restored under trap '...' EXIT INT TERM with an absolute REPO_ROOT, each restore proven by an empty git diff HEAD AND a HEAD-blob hash match printed per file (memory 421e00e6, mongodb b69a5521, identical before and after every leg). No rebuild leg was owed: both suites import their subject through RELATIVE paths inside their own package, so vitest compiles source and no dist sits between mutation and reading. Results — A1 (guard removed, the original clobber): memory 8/13 RED, mongodb 7/32 RED. A2 (ranking kept, promotion removed: order-independent but lossy): memory 5/13 RED, mongodb 7/32 RED. A3 (promotion kept, ranking removed: semantically correct but order-dependent documents): memory 13/13 GREEN, mongodb 6/32 RED — the key-order proof, since only the two-direction DOCUMENT assertions catch it.",
  "mcp_calls": "10 — issue_read get 13524, issue_read get_comments 13524, issue_read get 13495, issue_read get 13494, search_issues (dedup), issue_write create 13549, create_pull_request, pull_request_read (body readback), add_issue_comment (this), issue_read get_comments (readback of this)",
  "enumeration": "Obtained by MEASUREMENT, not reading: each of the 18 declared authorable field operators (SUPPORTED_FIELD_OPERATORS = FILTER_OPERATORS + $like + $ilike) was translated ALONE through both drivers, with a second comparand for every operator whose lowering is comparand-dependent, and the emitted key sets were intersected pairwise. Per-operator verdict — $eq/$ne/$gt/$gte/$lt/$in/$nin write their own name; $lte writes $lte OR $lt (bare calendar day); $between writes $gte plus $lte or $lt; $contains/$startsWith/$endsWith/$icontains/$like/$ilike write $regex; $notContains writes $not; $null writes $eq or $ne; $exists writes $ne or $eq. CONTESTED KEYS: $eq (from $eq, $null, $exists), $ne (from $ne, $null, $exists), $gte (from $gte, $between), $lte (from $lte, $between), $lt (from $lt, $lte-bare-day, $between-bare-day-max), $regex (from the string family). ANSWER TO THE MUST-ANSWER: YES, there is a fourth — $lte on a bare YYYY-MM-DD comparand lowers onto $lt, measured ['1'] vs ['1','2'] on the two key orders with oracle ['1']. And a FIFTH, driver-mongodb's alone: the $regex string family, which driver-memory already promoted via _multiRegex and mongodb never did — {$startsWith:'a',$endsWith:'z'} emitted {$regex:'z$'} and the swap {$regex:'^a'}. Also a NEGATIVE result: $notContains is NOT reachable — $not is written by it and nothing else, and $not is a LOGICAL operator absent from the field vocabulary, so it cannot be authored beside it. It is covered by construction, and that fact is pinned. The sweeps are written against the declared operator set with a coverage assertion, so a nineteenth operator fails loudly instead of being skipped.",
  "both_key_order_coverage": "Confirmed for every repaired cell. driver-memory: $null+$ne, $null+$eq, $between+$gte, $between+$lte, $lte(bare-day)+$lt, $between(bare-day)+$lt, $exists+$ne — each asserted in both orders against the reference matcher; plus the 18-operator x 306-ordered-pair x 2-order sweep. driver-mongodb: the same cells plus $startsWith+$endsWith and $contains+$icontains as exact documents in both orders, plus the 16-operator x 240-pair x 2-order sweep asserting the constraint set is order-invariant AND a superset of both singles. The ablation that proves the order cases bite is A3 (promotion kept, ranking removed): semantically correct in both directions, so all 13 memory row-set tests stay GREEN, while 6 of 32 mongodb two-direction document assertions go RED. Independently, the before-picture shows $between+$gte and $between+$lte failing in OPPOSITE directions on the broken code, so neither one-direction suite catches the class.",
  "analytics_face_verdict": "MEASURED, and it is a DIFFERENT and WIDER defect than the per-key one — the card's reading is confirmed. query() built its $match as matchStage[fieldPath] = builder(...), keyed by field path alone, so a second predicate on a member replaced the first ENTIRELY for every operator pair, not only those sharing a lowered key. Proof by a pair with NO contested key: {name:{$contains:'a',$ne:'b'}} aggregated ['1','3'] and the key-swapped twin ['1'], oracle ['1'] — a pair both translators always handled correctly. flattenFilterCondition folds $and into the same flat list, so the {$and:[{name:{...}},{name:{...}}]} shape a dashboard actually authors lost a constraint too. NOT out of proportion: the remedy is the same RULE (free member inline, taken member becomes its own $and branch of the same $match) with a different MECHANISM (field-path level, no ranking needed since nothing is overwritten), eight lines in the file the card names — so it is in this PR rather than deferred. Also measured: generateSql pushes into a LIST and never clobbered, so before this the echoed statement and the executed answer described different filters; that asymmetry is now pinned. Note the analytics capability gate refuses $null and $between on this face (INVALID_FILTER, by design per #5345), so the per-key members are not even reachable there — the wholesale one is.",
  "fix_shape": "Generalisation of #13195's landed guard, not a second idiom beside it. Each arm records its lowered writes through one put() accumulator in AUTHOR order; a shared assembleLoweredWrites() groups them by key, gives the inline slot to the lowest-ranked writer and promotes the rest into their own $and branches via an _extraAnd sentinel that the caller lifts (the same shape _multiRegex uses, and the shape _presenceAnd used before this subsumed it). The one addition to #13195's shape is the RANK: FIELD_OPERATOR_RANK is read off the spec's declaration order rather than hand-copied, which (a) makes the emitted document a function of the constraint set rather than of key order, and (b) reproduces #13195's landed documents byte for byte, because $exists is last in FILTER_OPERATORS. Collecting in author order and assembling afterwards keeps refusal order and uncontested-key insertion order byte-identical, which is why 946 + 510 pre-existing tests passed with zero edits.",
  "zone2_assumptions_broken": "C broke: the three known members are NOT the whole class — enumeration found $lte-on-a-bare-calendar-day as a fourth on both drivers and the $regex string family as a fifth on driver-mongodb, and it removed $notContains, which is not reachable. D broke in one direction: the analytics clobber IS wider, as assumed, but it does NOT follow that it does not belong in this PR — the remedy is eight lines and the card names the file, so deferring it would have left the widest instance of the card's own class open. A held and was used (the #13195 precedent generalises cleanly and is the shape taken). B held: memory-matcher.ts is untouched by this PR — re-confirmed before the first edit and again in the final diff — so there is no collision with #13494 or #13495.",
  "paths_touched_beyond_the_live_paths_and_tests": "Exactly one beyond the two live translators, their two new test files and the changeset: packages/drivers/driver-memory/src/memory-analytics.ts (the analytics face, ruling 1.6). Full diff, 6 files: packages/drivers/driver-memory/src/memory-driver.ts, packages/drivers/driver-memory/src/memory-analytics.ts, packages/drivers/driver-mongodb/src/mongodb-filter.ts, packages/drivers/driver-memory/src/memory-operator-key-clobber.test.ts (new), packages/drivers/driver-mongodb/src/mongodb-operator-key-clobber.test.ts (new), .changeset/lowered-operator-key-clobber.md (new). Fences honoured: packages/spec, docs/adr, .claude, skills, AGENTS.md, CLAUDE.md and content/docs/releases are untouched. The PM's clause-2 reasoning was re-checked and holds: nothing about composed-operator semantics is declared as contract in packages/spec — FieldOperatorsSchema declares each operator independently and says nothing about how two on one field compose — so this conforms to the reference matcher's existing answer rather than changing what the contract accepts.",
  "not_measured": "1) check-test-completeness — exit 3, PREREQUISITE NOT MET by its own text: it grades a saved turbo run test log and none exists locally. NOT a red, NOT a green. 2) check-half-states — exit 3, PREREQUISITE NOT MET: no valid GitHub credential in this container (the token is the proxy placeholder). Nothing was swept. 3) packages/cli's full suite and the three example apps' suites: the batch exceeded the container's ~10 minute foreground ceiling and was killed by my own timeout, not by a failure. Narrowed and DECLARED: the four packages/cli test files that name driver-memory or InMemoryDriver were run directly and passed (90 tests); the rest of that package and the example apps are NOT MEASURED here and are CI's. 4) packages/qa/dogfood was in the consumer list and did not get its own run — NOT MEASURED. 5) No real mongod: driver-mongodb's 143 server-gated tests are skipped here as they are in CI without the binary; the mongodb assertions in this PR are on emitted DOCUMENTS, which is the layer the defect is visible at. 6) CI itself has not been read — the report is delivered at draft-PR time per the dispatch contract, and the CI verdict is the PM's to read.",
  "open_questions": [],
  "out_of_scope_findings": [
    "filed as #13549: driver-memory's reference matcher matches a NULL-VALUED row against a well-formed bounded $between (live ['1','2'], matcher ['1','2','4']) while the MISSING-key reading is excluded — the same arm's coercion as #13495 but on the value axis, not the null-bound axis; both #13495 and #13494 were read before filing and neither carries this cell"
  ]
}

Generated by Claude Code

Navigation Menu

Repository navigation

A field operator whose lowering reuses another operator's key silently CLOBBERS it — $null, $between and $notContains on driver-memory and driver-mongodb, key-order dependent #13524

The shape

Measured, on `origin/main`

A third, wider instance on the analytics face

Why this is filed rather than fixed

What a fix owes

分诊定级 · 首次定级 · `bug` · p1 · `domain:engine` · `pm:queue`

p1 的理由,以及它与本轮 #13495 的分别

范围钉死

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

A field operator whose lowering reuses another operator's key silently CLOBBERS it — $null, $between and $notContains on driver-memory and driver-mongodb, key-order dependent #13524

Description

The shape

Measured, on origin/main

A third, wider instance on the analytics face

Why this is filed rather than fixed

What a fix owes

Activity

claude commented on Aug 30, 2026

分诊定级 · 首次定级 · bug · p1 · domain:engine · pm:queue

p1 的理由,以及它与本轮 #13495 的分别

范围钉死

zhuangjianguo commented on Aug 30, 2026

zhuangjianguo commented on Aug 30, 2026

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions

Measured, on `origin/main`

分诊定级 · 首次定级 · `bug` · p1 · `domain:engine` · `pm:queue`