Skip to content

Rewrite APPLY over json_each into a join on SQLite#38688

Open
Aykuttonpc wants to merge 1 commit into
dotnet:mainfrom
Aykuttonpc:sqlite-apply-to-join
Open

Rewrite APPLY over json_each into a join on SQLite#38688
Aykuttonpc wants to merge 1 commit into
dotnet:mainfrom
Aykuttonpc:sqlite-apply-to-join

Conversation

@Aykuttonpc

Copy link
Copy Markdown
Contributor

SQLite doesn't support APPLY, so projecting a JSON collection currently fails with ApplyNotSupported. SQLite's table-valued functions can reference preceding tables in the FROM clause though, so the APPLY that EF produces for these queries can be expressed as a regular join.

This adds SqliteApplyFlatteningExpressionVisitor, which runs before the existing APPLY validator and rewrites CROSS/OUTER APPLY over a subquery whose only table is json_each into a CROSS/LEFT JOIN over that function, inlining the subquery's projection into the containing SELECT.

Json_collection_in_projection_with_anonymous_projection_of_scalars now translates to:

SELECT "j"."Id", "o"."value" ->> 'Name' AS "Name", "o"."value" ->> 'Number' AS "Number", "o"."key"
FROM "JsonEntitiesBasic" AS "j"
LEFT JOIN json_each("j"."OwnedCollectionRoot", '$') AS "o" ON 1
ORDER BY "j"."Id", "o"."key"

Scope

Only subqueries that do nothing beyond projecting out of json_each are rewritten:

Tables: [JsonEachExpression], Predicate: null, GroupBy: [], Having: null,
Orderings: [], Limit: null, Offset: null, IsDistinct: false

Lifting a predicate or a limit out of the subquery would change what the query means, so anything else still goes through the validator and reports ApplyNotSupported as before. That's why this says "Part of" rather than "Fixes".

Tests

The full SQLite functional suite passes (38367 tests, 0 failures). Nine tests that previously asserted ApplyNotSupported now run against the real query, so their overrides were removed:

  • JsonQuerySqliteTest.Json_collection_in_projection_with_anonymous_projection_of_scalars
  • ComplexJsonProjectionSqliteTest and OwnedJsonProjectionSqliteTest: SelectMany_associate_collection, SelectMany_nested_collection_on_required_associate, SelectMany_nested_collection_on_optional_associate

The SelectMany ones weren't something I targeted, they started working as a side effect of the rewrite.

One open question

The LEFT JOIN uses a constant true predicate, which renders as ON 1, since there is no real join condition. Let me know if you'd rather see something else there.

Part of #34375

SQLite doesn't support APPLY, so queries projecting a JSON collection failed
with "APPLY not supported". Its table-valued functions can reference preceding
tables in the FROM clause though, which is what APPLY is needed for here.

Add SqliteApplyFlatteningExpressionVisitor, which runs before the APPLY
validator and rewrites CROSS/OUTER APPLY over a subquery whose only table is
json_each into a CROSS/LEFT JOIN over that function, inlining the subquery's
projection into the containing SELECT. Only subqueries that do nothing else
(no predicate, ordering, paging, grouping or DISTINCT) are rewritten, since
lifting those out would change the meaning of the query.

Part of dotnet#34375
@Aykuttonpc
Aykuttonpc requested a review from a team as a code owner July 23, 2026 16:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant