feat(spec,objectql,metadata-protocol): grouped and aggregated queries honour search

.changeset/20358-aggregate-honours-search.md

@@ -0,0 +1,17 @@
+---
+"@objectstack/spec": minor
+"@objectstack/objectql": minor
+"@objectstack/metadata-protocol": patch
+---
+
+A grouped or aggregated query now honours `search`: the groups and every aggregated number are computed over the searched rows, exactly the rows the same query without `groupBy` / `aggregations` returns.
+
+Clause-②: yes (widening) — `EngineAggregateOptionsSchema` gains two OPTIONAL keys, `search` and `searchFields`, so the accept set of the aggregate options grows. Nothing previously admitted is refused, no key is renamed or retired, and no producer is required to write them.
+
+`QuerySchema.search` (ADR-0061) is declared on the query beside `groupBy` and `aggregations`, with no carve-out. Until now, `POST /data/:object/query` accepted a body such as `{ groupBy: ["business_unit"], aggregations: [{ function: "count", alias: "count" }], search: "harbour" }` and answered it with the UNSEARCHED groups — no error and no warning — while the same body without `groupBy` / `aggregations` returned only the searched rows. A grouped list view under a toolbar search would therefore show group headers that ignore what the user typed.
+
+- **`@objectstack/spec`** — `EngineAggregateOptionsSchema` declares `search` (the bare string, or the structured `FullTextSearchSchema` form) and `searchFields`, identically to `EngineQueryOptionsSchema`. A parse used to strip them.
+- **`@objectstack/objectql`** — `engine.aggregate()` (and `ctx.api.object(name).aggregate()`) accepts the two keys it used to refuse as unknown options, and expands them through the same ADR-0061 expansion `find()` uses: the same server-resolved searchable fields, the same `searchFields` narrowing, AND-ed with `where` before the security middlewares run. There is one expander, not two. It applies on both aggregate paths, native `driver.aggregate()` and the in-memory lowering. A key the verb still does not execute, such as `$search`, is refused as before.
+- **`@objectstack/metadata-protocol`** — `findData`'s grouped branch passes `search` / `searchFields` to `engine.aggregate()`. `searchFields` is validated on that branch exactly as on the flat one: a column search cannot scan is `400 INVALID_FIELD`.
+
+Nothing to migrate. A caller that worked around the gap, for example by grouping a page of searched rows on the client, can send the grouped query with its `search` instead.

content/docs/references/data/data-engine.mdx

@@ -152,6 +152,8 @@ Options for DataEngine.aggregate operations
 | **aggregations** | `{ function: Enum<'count' \| 'sum' \| 'avg' \| 'min' \| 'max' \| 'count_distinct'>; field?: string; alias: string; filter?: any }[]` | optional |  |
 | **having** | `any` | optional | HAVING — filter over the aggregated rows (aggregation aliases + groupBy projections); applied engine-side after aggregation |
 | **timezone** | `string` | optional |  |
+| **search** | `string \| { query: string; fields?: string[]; fuzzy?: boolean; operator?: Enum<'and' \| 'or'>; … }` | optional |  |
+| **searchFields** | `string[]` | optional |  |
 | **filter** | `Record<string, any> \| any` | optional | Data Engine query filter conditions |
 
 
@@ -708,6 +710,8 @@ This schema accepts one of the following structures:
 | **aggregations** | `{ function: Enum<'count' \| 'sum' \| 'avg' \| 'min' \| 'max' \| 'count_distinct'>; field?: string; alias: string; filter?: any }[]` | optional |  |
 | **having** | `any` | optional | HAVING — filter over the aggregated rows (aggregation aliases + groupBy projections); applied engine-side after aggregation |
 | **timezone** | `string` | optional |  |
+| **search** | `string \| { query: string; fields?: string[]; fuzzy?: boolean; operator?: Enum<'and' \| 'or'>; … }` | optional |  |
+| **searchFields** | `string[]` | optional |  |
 | **filter** | `Record<string, any> \| any` | optional | Data Engine query filter conditions |
 
 ---
@@ -898,6 +902,8 @@ QueryAST-aligned options for DataEngine.aggregate operations
 | **aggregations** | `{ function: Enum<'count' \| 'sum' \| 'avg' \| 'min' \| 'max' \| 'count_distinct'>; field?: string; alias: string; filter?: any }[]` | optional |  |
 | **having** | `any` | optional | HAVING — filter over the aggregated rows (aggregation aliases + groupBy projections); applied engine-side after aggregation |
 | **timezone** | `string` | optional |  |
+| **search** | `string \| { query: string; fields?: string[]; fuzzy: boolean; operator: Enum<'and' \| 'or'>; … }` | optional |  |
+| **searchFields** | `string[]` | optional |  |
 
 ### Nested Shape: `EngineAggregateOptions.context`
 
@@ -954,6 +960,19 @@ QueryAST-aligned options for DataEngine.aggregate operations
 | **distinct** | `never` | optional | [REMOVED] `query.aggregations[].distinct` was removed in @objectstack/spec 17 (ADR-0049) — exactly ONE of the six faces that read an aggregation honoured it. The objectql in-memory fallback deduplicated the values before applying the function, while `driver-sql`, `driver-turso`, `driver-mongodb`, `driver-memory` and the service-analytics SQL builder all ignored it — so `{ function: 'sum', field: 'amount', distinct: true }` answered a DEDUPLICATED sum when the engine fell back in memory and an ordinary sum on every SQL datasource: one query, two numbers, chosen by which backend happened to serve it. Both answers are plausible, so nothing surfaced the divergence. Delete the key. For a deduplicated COUNT the live spelling is the `count_distinct` aggregation function, which every SQL face compiles to `COUNT(DISTINCT field)` and the in-memory fallback computes identically. `SUM(DISTINCT …)` / `AVG(DISTINCT …)` get no replacement: no backend ever computed them here, and a per-row measure that needs deduplicating is a modelling problem to fix in the data, not a flag on the read. |
 | **filter** | `any` | optional | Per-aggregation filter (SQL FILTER (WHERE …) semantics): narrows the source rows THIS aggregation reads, leaving sibling aggregations unfiltered. Enforced by engine.aggregate: lowered in memory for drivers without native conditional aggregation; a driver reached directly refuses rather than silently dropping it. |
 
+### Nested Shape: `EngineAggregateOptions.search`
+
+| Property | Type | Required | Description |
+| :--- | :--- | :--- | :--- |
+| **query** | `string` | ✅ | Search query text |
+| **fields** | `string[]` | optional | Fields to search in (if not specified, searches all text fields) |
+| **fuzzy** | `boolean` | optional (default: `false`) | [EXPERIMENTAL — not enforced] Fuzzy matching (tolerate typos). The ADR-0061 expansion reads only `query` + `fields`; no executor receives this flag. |
+| **operator** | `Enum<'and' \| 'or'>` | optional (default: `"or"`) | [EXPERIMENTAL — not enforced] Logical operator between terms. The ADR-0061 expansion applies its own term semantics; no executor receives this flag. |
+| **boost** | `Record<string, number>` | optional | [EXPERIMENTAL — not enforced] Field-specific relevance boosting (field name -> boost factor). No executor scores results. |
+| **minScore** | `number` | optional | [EXPERIMENTAL — not enforced] Minimum relevance score threshold. No executor scores results. |
+| **language** | `string` | optional | [EXPERIMENTAL — not enforced] Language for text analysis (e.g., "en", "zh", "es"). No executor selects an analyzer. |
+| **highlight** | `boolean` | optional (default: `false`) | [EXPERIMENTAL — not enforced] Search result highlighting. No executor emits highlights. |
+
 
 ---
 

packages/metadata-protocol/src/protocol.ts

@@ -11067,6 +11067,16 @@ export class ObjectStackProtocolImplementation implements
                 // was finding 1: the one wire path to aggregate() lost the
                 // clause before any executor could ever see it.
                 having: options.having,
+                // ADR-0061 `search` is declared on the query beside `groupBy` /
+                // `aggregations` with no carve-out, and the flat branch below
+                // hands it to `engine.find`. Leaving it out of THIS bag answered
+                // a grouped query under a search with the UNSEARCHED groups —
+                // no error, no warning. The engine expands it through the same
+                // expander `find` uses, so the header numbers are the grouping
+                // of exactly the rows the flat query returns. `searchFields` was
+                // validated above on both branches (#4254 gate).
+                search: options.search,
+                searchFields: options.searchFields,
                 context: options.context,
             } as any);
             // Apply limit client-side (EngineAggregateOptions doesn't carry limit).