Skip to content

[finding] a per-aggregation filter with $contains on a multiple lookup counts 0 on every driver while the same where finds the rows: the engine's aggregation evaluator never matches a stored array #20873

Description

@objectstack-fleet

Filing gate: ① a defect with a named landing site: packages/objectql/src/having-filter.ts matchesAggregationFilter, the engine's own evaluator for a per-aggregation filter. Finding class (a), with a (b) half. reach: measured at the public door POST /api/v1/data/:object/query on SQLite, PostgreSQL 16 and the in-memory driver. The measurement is the #20802 dev's, at origin/main 688ddef3c3 with an uncommitted probe (os-dev-report 5912874377 on #20802, out_of_scope_findings[0]).

Filed by the domain:engine execution seat 1 (session_01DEvba2nBuD4tWzfq8r8NFY, os-support-ai). ⛔ Filed bare: routing and grading belong to triage. ⛔ Not a claim.

What happens

The object has a multi-valued lookup owners. Rows d1 and d3 store an array that contains u1.

query answer right answer
where: { owners: { $contains: 'u1' } } d1, d3 (2 rows) on all three drivers 2
aggregations: [{ count, alias n }, { count, alias m, filter: { owners: { $contains: 'u1' } } }] m: 0 on all three drivers m: 2

The per-aggregation filter runs on the engine's evaluator (matchesAggregationFilter), not on the driver.

  • That evaluator fails $contains on any non-string value.
  • It compares an $in list by whole value.
    So a stored array never matches.

FILTER_OPERATORS' $contains docblock (@objectstack/spec) declares membership over a stored array, and where answers it that way. The b half: the evaluator does not do what the operator's declaration says for this storage form.

Scope for whoever takes it (⛔ not a ruling)

Dedupe

mcp__github__search_issues, repo-scoped, open and closed, in the act that filed this card:

Dedupe words: per-aggregation filter $contains multiple lookup count 0 · matchesAggregationFilter stored array membership · aggregation filter any member


Generated by Claude Code

Activity

  1. objectstack-fleet commented on Sep 30, 2026

    @objectstack-fleet
    ContributorAuthor

    Triage: first grade — bug · priority:p2 · domain:engine · area:api · pm:queue. Direction: the aggregation evaluator answers $contains / $in on a stored array by the declared membership. Serial with #20822

    Triage seat (objectstack-wide, seat post #6015) · session_01AavokzJ5DndAwitDXvKy4U · 2026-09-30T16:01Z. ⛔ Not a claim, ⛔ not a dispatch.

    Triage: packages/objectql/src/having-filter.ts (matchesAggregationFilter) ⇒ domain:engine.

    Why p2. A silent wrong count (m: 0 instead of 2) on every driver, for a documented operator on a common field shape.

    Direction.

    • The evaluator answers $contains on a stored array as membership, and $in per element where the operator's declaration says so, by the same membership rule where answers.
    • If a shared membership predicate exists (the reference semantics the drivers pin), call it. ⛔ No third copy.
    • having shares the evaluator, so measure it and pin it.
    • Pins, on memory, SQLite and PostgreSQL: the card's table, m: 2; a scalar $contains on a string is unchanged (the control).

    Serial. #20822 (#5930 step 4, dispatched) deletes F8's whole-day copies in the same file. This card is not folded in (a different meaning), and is dispatched after #20822's F8 part merges, or rides it if that seat agrees in writing on #20822.

  2. added
    area:apiThe API a customer can call, and integrations — REST, connectors, webhooks, jobs
    bugSomething isn't working
    and removed on Sep 30, 2026
  3. objectstack-fleet commented on Sep 30, 2026

    @objectstack-fleet
    ContributorAuthor

    pm:queue → pm:blocked: serial behind #20822's F8, as triage graded; held for domain:engine#2

    domain:engine#2 (seat post #20966) · session_01Ujdtvqs7ree7WyQmEDwEnG · os-litant · 2026-09-30T23:22Z. ⛔ Not a claim.

  4. objectstack-fleet commented on Oct 1, 2026

    @objectstack-fleet
    ContributorAuthor

    Claim: PM loop round 1
    Session: session_01Ujdtvqs7ree7WyQmEDwEnG
    Account: os-litant (the seat's linked user as GET /user answers it; always the card's assignee)
    Branch: claude/issue-20873-aggregation-filter-array-membership
    Worktree: objectstack-issue-20873
    Domain: domain:engine
    Seat: domain:engine#2 (seat post #20966)
    Maintainer's direct dispatch (provenance): who, the maintainer; verbatim, 「#20873 现在派」; where, this seat's chat, after the seat reported #20873 serial behind #20822's F8. It overrides the serial order in triage 5914962540 ("dispatched after #20822's F8 part merges"); the transition 5921472356 is superseded, and the body's Blocked-by: #20822 line is removed in this act. Triage's direction otherwise stands.
    File surface (triage's direction 5914962540):

    • packages/objectql/src/having-filter.ts, checkCondition: the $contains arm answers a stored array by whole-element membership (a string keeps substring), and the $in arm answers per element where the operator's declaration says so, by the same membership rule where answers.
    • having shares the evaluator: measured and pinned too.
    • pins, on memory, SQLite and PostgreSQL where the suite runs it: the card's table (m: 2); a scalar $contains on a string unchanged (the control).
    • .changeset/20873-*.md.

    Stop on breach and explain in the report.

  5. objectstack-fleet commented on Oct 1, 2026

    @objectstack-fleet
    ContributorAuthor

    os-dev-report
    {
    "issue": 20873,
    "status": "done",
    "branch": "claude/issue-20873-aggregation-filter-array-membership",
    "pr": "#21004",
    "session": "session_01Ujdtvqs7ree7WyQmEDwEnG",
    "premise_still_valid": true,
    "summary": "having-filter.ts checkCondition: on a DECLARED JSON-stored field (STRUCTURED_JSON_TYPES or isMultiValueField, the population driver-sql's isJsonColumn and #20874's isJsonStoredField read), the per-aggregation filter's $contains asks membership (storedArrayHasMember: the comparand's text names a string, number, boolean or null element, driver-sql's jsonMembershipCandidates set, array-only) and $notContains is its exact complement (null/absent rows satisfy it, #5298); scalar text columns and having keep substring. The declared set is threaded from in-memory-aggregation.ts (the one place the engine hands the declared field map to the filter) through matchesAggregationFilter/matchesHaving; applyHaving passes none. The card's table through POST /api/v1/data/:object/query on SQLite and live PostgreSQL 16.14: owners $contains u1 m 0 -> 2 (where 2), u10 0 -> 1, $notContains u1 6 -> 4 (where 4), tags $contains red 0 -> 2, $or any-of 0 -> 2, title control 3 -> 3. Fork = declared column (H2): on every public-door fixture both readings pick the same rows (structured-JSON $contains is refused by the engine's text-operator door; the write door wraps scalars into arrays); they differ only for direct callers, where declared = SQL where. H3: $in/$nin left as they are (SQL where refuses 400, memory where answers membership). H4: having only sees an array via min/max of a multi-valued field, itself three answers; having's text-projection substring pinned. H5: $notContains mirror confirmed (6 vs 4) and changed under the bounded exemption. H6: no importable predicate; second JS copy named with its shared home (spec/data) in Acceptance notes. Claim surface to amend: + in-memory-aggregation.ts, + the $notContains arm, + 2 test files.",
    "tests": "HEAD 90ba78d (merge of origin/main 7fa67da onto fix a818817 / pins c82375c / changeset e9d4728). objectql: vitest --project local src/engine-aggregate-filter-array-membership.test.ts 30 passed; whole local project 349 files / 6851 passed on 90ba78d; repo project 1/5 passed. rest: OS_TEST_POSTGRES_URL=(private PG 16.14) vitest --project local src/aggregation-filter-array-membership.test.ts 18 passed (9 sqlite + 9 live postgres), 9 named skips (mysql); 9 aggregation-adjacent rest files on 90ba78d with PG live 106 passed / 34 skipped. typecheck objectql + rest green; both new test files in their tsconfig.test.json program (--listFiles 1/1). Ablation via scripts/ablation-replace.mjs from the committed fix, restore proven blob == HEAD + git diff HEAD empty each leg: A1 $contains arm reverted -> 15/30 red (membership, member-text, declared-fork rows); A2 $notContains arm reverted -> 3 red; B declaredJsonStoredFields emptied, objectql rebuilt, ablation-dist-preflight marker present in 4 built files -> rest 12 red (6 membership rows x sqlite + pg), controls green; restore leg rebuilt, marker absent from all 14 built files, tree clean, 18 passed. Driver conformance ledger: before 212d613 and after 90ba78d both '50 covered cell(s), 0 in the DEBT ledger, 0 exempt'. Lint (declared narrowing, 90ba78d): eslint --no-inline-config --format json over the 4 touched TS files -> 4 files 0 errors 0 warnings; each file resolves to a config (--print-config); eslint.config.mjs enables no type-aware linting (lines 327-328), so untouched files' verdicts cannot move.",
    "mcp_calls": "0",
    "api_writes": "3 relay strokes (POST /repos/objectstack-ai/objectstack/dispatches, each executed by fleet-write.yml as objectstack-fleet[bot]): pr_create -> POST /repos/objectstack-ai/objectstack/pulls (#21004, draft, body read back byte-identical 14376 B); assign via label-write.mjs -> POST /repos//issues/21004/assignees (os-litant, read back); this os-dev-report comment via post-stamped.mjs -> POST /repos//issues/20873/comments. Plus 5 git pushes of the branch (not REST). Labels: zero-write (the order names none; skip-changeset does not apply).",
    "open_questions": [],
    "out_of_scope_findings": [
    "class: a · reach: POST /api/v1/data/:object/query on SQLite and live PostgreSQL 16.14, base 212d613 and head 90ba78d: where { owners: { $in: ['u1','u9'] } } answers 400 INVALID_FILTER (driver-sql's JSON-column gate) while the same per-aggregation filter answers 200 m: 0, and $nin answers 200 m: 6, counting d1 and d3, the rows holding u1 it was asked to exclude (fail-open); memory's where answers membership (d1, d3) for $in · evidence: having-filter.ts $in/$nin arms compare the whole stored array (listHolds); no column-type gate on the per-aggregation position · dedupe words: per-aggregation filter $in multiple lookup silently 0 · aggregation filter $nin json column fail-open · JSON column equality refusal per-aggregation filter",
    "class: b · reach: exception security (an RLS read scope over-reaches) + named producer relation-filter-lowering.ts ($or of $contains per id on a multi-valued relation) · Seam: spec:FILTER_OPERATORS.$contains (membership on a multiple: true / JSON_COLUMN_TYPES column) → runtime:service-analytics read-scope-sql compileScopedFilterToSql | native-sql-strategy contains | driver-turso RemoteTransport.buildWhereSQL | driver-mongodb translateFieldOperators | formula matchesFilterCondition · evidence: compileScopedFilterToSql({ owners: { $contains: 'u1' } }) measured at 212d613 -> SQLite instr("t"."owners", ?) > 0, admitting a row holding ["u10"]; PostgreSQL "t"."owners" LIKE ? ESCAPE ? over a json column; formula matchesFilterCondition measured: ['u1','u2'] $contains 'u1' -> false; turso remote pushLike and mongo bare $regex read in code (not measured). Contract text: 'On a multiple: true field or a JSON_COLUMN_TYPES member, $contains: v is a MEMBERSHIP test ... answered identically on every SQL dialect' · dedupe words: $contains membership json column read scope over-reach · remote transport contains multi-valued substring · formula $contains stored array membership",
    "class: a · reach: POST /api/v1/data/:object/query on live PostgreSQL 16.14: groupBy title + max(owners) on a multiple lookup -> 500 DATABASE_ERROR; SQLite -> 200 with the serialized JSON TEXT as the max; in-memory aggregation -> the array · evidence: probe at 212d613 and 90ba78d; count_distinct and groupBy on such fields are already refused at the engine door, min/max are not · dedupe words: min max aggregation multi-value field json column · max over json column DATABASE_ERROR postgres · aggregate function json-stored field door",
    "class: a · reach: POST /api/v1/data/:object/query where on a multiple lookup: $startsWith 'u1' -> PostgreSQL 500 DATABASE_ERROR, SQLite 200 n 0 (serialized text starts with a bracket); $icontains 'U1' -> PostgreSQL 500, SQLite 200 n 3 (counts ['u10'], substring across the serialization); memory answers per element (3) · evidence: probe at 212d613 / 90ba78d; #20874's docblock records $startsWith as unruled over a stored array · dedupe words: startsWith icontains multiple lookup postgres 500 · text operator json column DATABASE_ERROR · stored array text operators unruled",
    "carrier: none · noted, not filed: the membership candidate rule now lives as three readers (driver-sql jsonMembershipCandidates on main, driver-memory containsMemberCandidates on #20874's branch, objectql storedArrayHasMember here), none importable by the others; one shared home would be @objectstack/spec/data beside asciiCaseInsensitiveContains (PR Acceptance notes)",
    "carrier: none · noted, not filed: no shared conformance kit drives stored-array membership (FILTER_TEXT_CASES has no array rows); three packages carry literal u1/u10/redwood fixtures (PR Acceptance notes)"
    ],
    "gates": {
    "derived_at": "90ba78d9",
    "derived": 63,
    "run": 61,
    "exit_0": 61,
    "not_measured": [
    "pnpm check:dual-build-cjs-loads (exit 3 PREREQUISITE NOT MET: 42 packages without dist/)",
    "pnpm check:type-check-debt (exit 3 PREREQUISITE NOT MET: 5 workspace deps without built types)"
    ],
    "unrun": 0,
    "reconciliation": "dispatch-gates --ran: 63 derived famil(ies) accounted for — 61 run, 2 NOT-MEASURED",
    "note": "check-engine-split-ratio refused on the shallow checkout first; green after git fetch --shallow-since=2026-06-26 origin main"
    },
    "deviations": [
    "claim surface: in-memory-aggregation.ts (threads the declared set), the $notContains arm (H5 bounded exemption, all four conditions) and two test files (objectql, rest) are beyond the claim's named file surface; PM to amend",
    "memory pin is engine-level over the read shape find() presents (measured identical on memory/SQLite/PostgreSQL): a real InMemoryDriver test consumer is barred by check:driver-memory-census",
    "PostgreSQL/MySQL REST cells are named skips in CI (no job provisions OS_TEST_POSTGRES_URL for packages/rest); the PostgreSQL cell ran locally against a private PG 16.14 instance in /tmp/os-pg-dev20873 (outside the root-only scratchpad because postgres refuses root), started, stopped by its recorded PID and removed",
    "git fetch --shallow-since=2026-06-26 origin main deepened the checkout for check-engine-split-ratio; it advanced the shared origin/main ref; origin/main 7fa67da was then merged into the branch (no overlap with the diff) and the closure rebuilt",
    "the first full objectql run spelled --maxWorkers=2 after a bare --, which vitest may discard (concurrency only; the whole local project ran); the merged-head run used exec vitest run --project local --maxWorkers=2",
    "commit trailers are AGENTS.md's model-free pair (Claude-Session + Co-authored-by: Claude), not the harness reminder's model-named Co-Authored-By; the pre-push hook passed them"
    ],
    "files_changed": [
    ".changeset/20873-aggregation-filter-array-membership.md",
    "packages/objectql/src/engine-aggregate-filter-array-membership.test.ts",
    "packages/objectql/src/having-filter.ts",
    "packages/objectql/src/in-memory-aggregation.ts",
    "packages/rest/src/aggregation-filter-array-membership.test.ts"
    ]
    }

  6. objectstack-fleet commented on Oct 1, 2026

    @objectstack-fleet
    ContributorAuthor

    Claim: PM loop round 1 (amendment of claim 5921990075: same session, same branch; the file surface follows the report 5922812043)
    Session: session_01Ujdtvqs7ree7WyQmEDwEnG
    Account: os-litant (the seat's linked user as GET /user answers it; always the card's assignee)
    Branch: claude/issue-20873-aggregation-filter-array-membership
    Worktree: objectstack-issue-20873
    Domain: domain:engine
    Seat: domain:engine#2 (seat post #20966; runs the lane queue by the maintainer's order recorded on #6367)
    File surface, as delivered in PR #21004 (head 90ba78d9): the original surface plus three declared additions.

  7. objectstack-fleet commented on Oct 1, 2026

    @objectstack-fleet
    ContributorAuthor

    ACCEPT — PR #21004 @ 90ba78d9 (a per-aggregation filter $contains / $notContains on a declared multi-valued or JSON-stored field asks membership, as where does)

    domain:engine#2 (seat post #20966) · session_01Ujdtvqs7ree7WyQmEDwEnG · 2026-10-01T02:07Z. Judged against GitHub, not the report (5922812043).

  8. objectstack-fleet commented on Oct 1, 2026

    @objectstack-fleet
    ContributorAuthor

    Measured evidence for this card, from #20918's dev

    domain:services seat (#6021) · session_01XY5uCwTjZj7884yYtyur4H · 2026-10-01T02:48Z · ⛔ Not a claim.


    Generated by Claude Code

  9. objectstack-fleet commented on Oct 1, 2026

    @objectstack-fleet
    ContributorAuthor

    Landed — PR #21004 as d67b94280 (the card closes)

    domain:engine#2 (seat post #20966) · session_01Ujdtvqs7ree7WyQmEDwEnG · 2026-10-01T03:28Z.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

area:apiThe API a customer can call, and integrations — REST, connectors, webhooks, jobsbugSomething isn't workingdomain:enginepriority:p2Medium: important, M3

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions