Add DuckDB PIVOT query support#1025
Open
SeanCassiere wants to merge 6 commits into
Open
Conversation
Introduce a first-class PivotQuery node for DuckDB simplified PIVOT queries. The new query type can be constructed with Query.pivot(source), normalizes string sources to table refs, participates in generic query type checks, and exposes an isPivotQuery guard. Wire the pivot node through public AST exports, visitor dispatch, DuckDB shell rendering, and traversal metadata so it can be used anywhere a Query source is accepted, including nested SELECT FROM sources. Add focused tests covering construction, type guards, source normalization, exported API usage, and basic composability as a SELECT source.
Add a chainable PivotQuery.on(...) API for DuckDB simplified PIVOT queries. ON expressions now use the existing expression-list coercion path, so string inputs are recorded as column references while existing AST expressions are preserved. Render recorded ON expressions in DuckDB SQL generation as a single ordered ON clause. Repeated on(...) calls append to the existing expression list. Add pivot tests covering string columns, variadic and array inputs, repeated calls, AST expression inputs, and use of pivot clauses when a PivotQuery is nested as a SELECT source.
Add PivotQuery.in(...) for DuckDB simplified PIVOT queries so callers can constrain and order generated pivot columns with ON ... IN (...). IN values are stored independently from ON expressions, appended across repeated calls, and coerced as SQL literals by default while preserving existing SQL expression nodes. Empty dynamic calls now throw a clear error instead of producing invalid IN () SQL. Render IN clauses immediately after pivot ON expressions, preserve pivot IN values during cloning and deep cloning, and add coverage for literal coercion, expression passthrough, ordering, repeated calls, nested pivot-as-source rendering, and clone isolation.
Add a chainable PivotQuery.using(...) API for DuckDB simplified PIVOT queries. USING expressions now accept unaliased expression inputs as well as object-style aliases, reuse SelectClauseNode rendering for AS aliases, and preserve caller-provided expression order across chained calls. Render USING clauses in the DuckDB code generator and include pivot USING nodes in clone/deep-clone traversal. Add focused pivot tests for unaliased, aliased, multiple, cloned, deep cloned, and nested SELECT source USING expressions without changing the existing pivot test coverage.
Add grouping support to PivotQuery with groupby and setGroupby methods matching the SelectQuery API. Pivot grouping expressions now use the existing expression coercion path, append across repeated groupby calls, and participate in clone and visitor recursion behavior. Update DuckDB SQL generation to emit GROUP BY after simplified PIVOT USING clauses when grouping expressions are present. Add tests covering string and AST grouping inputs, multiple and repeated groups, reset behavior, cloning, deep cloning, and use as a SELECT source.
Collaborator
|
Looking forward to this!
@domoritz brought this up in the issue, I'm not sure the best way to go about confirming this works with those materializations |
Collaborator
|
Comparing the DuckDB simplified syntax vs the SQL standard syntax for convenience here. 
https://duckdb.org/docs/current/sql/statements/pivot#simplified-pivot-syntax 
https://duckdb.org/docs/current/sql/statements/pivot#sql-standard-pivot-syntax My gut feeling is that there's no need to support the less convenient syntax in the AST, even if it is the standard. Since we have the pivot AST node, translations to other SQL syntax should still be possible, and the convenience of the DuckDB setup is 100x better for the user. |
There was a problem hiding this comment.
Pull request overview
Adds first-class DuckDB PIVOT support to the mosaic/sql AST and DuckDB SQL codegen, enabling Query.pivot(source) with chainable on, in, using, and groupby clauses and use as a FROM source.
Changes:
- Introduces
PivotQuery+Query.pivot(...)/WithClause.pivot(...)builder APIs and type guards/exports. - Extends visitor recursion + SQL codegen dispatch to support a new
PIVOT_QUERYnode type. - Adds focused Vitest coverage for PivotQuery construction, rendering, cloning, and clause composition.
Reviewed changes
Copilot reviewed 8 out of 8 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
| packages/mosaic/sql/test/pivot.test.ts | New tests covering PivotQuery builder behavior, rendering, and cloning. |
| packages/mosaic/sql/src/visit/recurse.ts | Adds PivotQuery node recursion configuration for visitors (walk/clone/rewrite). |
| packages/mosaic/sql/src/visit/codegen/sql.ts | Adds PivotQuery to the codegen dispatch + abstract visitor surface. |
| packages/mosaic/sql/src/visit/codegen/duckdb.ts | Implements DuckDB-specific SQL rendering for PivotQuery. |
| packages/mosaic/sql/src/types.ts | Adds Pivot-specific expression input types. |
| packages/mosaic/sql/src/constants.ts | Introduces PIVOT_QUERY node type constant. |
| packages/mosaic/sql/src/ast/query.ts | Adds PivotQuery AST node, builders, type guard, and WithClause integration. |
| packages/mosaic/sql/src/ast/index.ts | Re-exports PivotQuery APIs from the AST index barrel. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Render PivotQuery WITH clauses in DuckDB SQL generation so Query.with(...).pivot(...) and Query.pivot(...).with(...) preserve CTEs. Clone inherited mutable query clause arrays for PivotQuery to avoid sharing WITH and ORDER BY state between originals and clones. Add focused regression tests for pivot CTE rendering and clone isolation.
Collaborator
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Building on the discussion in #892, this PR is basically my first stab at adding support for DuckDB’s simplified
PIVOTsyntax to the sql package.This add
Query.pivot(source)as a new query-level AST node, with chainable support forON,IN,USING, andGROUP BY:The new node is exported publicly, works with the existing SQL visitor/codegen path, can be used as a
FROMsource inSelectQuery, and is covered by focused tests for construction, rendering, cloning, and clause behavior.This PR intentionally sticks to DuckDB’s simplified
PIVOTform, but of-course should be extendable to support the SQL specific implementation. It does not add SQL-standard PIVOT, runtime validation thatUSINGexpressions are aggregates, or renderedORDER BY/LIMIT/OFFSETbehavior for pivot queries. So any feedback here would be great!