Skip to content

fix(gfql): fixes found by the route sweeps: source dtypes on every route, polars prune decline, polars-gpu fused lane (#2058, #2053, #2064) - #2062

Merged
lmeyerov merged 4 commits into
masterfrom
fix/gfql-2058-general-path-int-dtypes
Sep 7, 2026
Merged

fix(gfql): fixes found by the route sweeps: source dtypes on every route, polars prune decline, polars-gpu fused lane (#2058, #2053, #2064)#2062
lmeyerov merged 4 commits into
masterfrom
fix/gfql-2058-general-path-int-dtypes

Conversation

@lmeyerov

@lmeyerov lmeyerov commented Sep 6, 2026

Copy link
Copy Markdown
Contributor

The three product fixes the route flow found, folded into one PR at the owner's request (formerly #2062 + #2065 + #2066). Stacked on #2054; top of the stack.

  1. Every route keeps source dtypes (gfql: the general chain path upcasts integer node/property columns to float where every fast lane keeps int (route-dependent dtypes) #2058): one stub-row widening artifact (id-only endpoint rows concatenated then deduplicated) mirrored on purpose at three other seams. The endpoint closure appends only missing endpoints; the bindings row frame restores an alias property's node-table dtype once its rows hold no null; the 0-row connected-join path no longer mirrors the widening; the Cypher seeded lanes keep source dtypes on every route. Contract change: pandas RETURN projections and 0-row aggregates of int/bool properties carry the source dtypes; four pins restated. The cuDF 26.02 prune_to_endpoints pins pass on the real GPU at this tree (gfql cuDF 26.02 (real GPU): chain default route and policy-forced full path disagree on prune_to_endpoints hops and duplicate node ids; tests use the removed cudf.DataFrame.from_pandas #2043 prune half; marker dropped).
  2. Polars declines prune_to_endpoints on single-hop edges (gfql polars: the plain single-hop chain branch ignores prune_to_endpoints (pandas keeps only the arrival side) #2053): typed NotImplementedError and predicate decline; variable-length hops keep their native pruning.
  3. The fused grouped-aggregate lane keeps the group_by plan on the GPU target (gfql polars-gpu: the fused grouped-aggregate lane and the polars-gpu strictness tests fail on the RAPIDS 26.02 image (plan not GPU-executable on its cudf-polars) #2064): its value_counts alternative lowers to unnest, which cudf-polars has no implementation for; GPU receipt 607/5 vs master 520/91 on the two files.

Receipts on this head below; earlier receipts on the closed #2065 and #2066.

🤖 Generated with Claude Code

https://claude.ai/code/session_01QztW7jYsDd66e8rb8pJNQA

@lmeyerov
lmeyerov force-pushed the fix/gfql-2058-general-path-int-dtypes branch from ef1d294 to 612c85b Compare September 6, 2026 01:19
@lmeyerov
lmeyerov force-pushed the test/gfql-route-harness branch from 3213f7d to 21f662c Compare September 6, 2026 02:30
@lmeyerov
lmeyerov force-pushed the fix/gfql-2058-general-path-int-dtypes branch from 612c85b to 676232a Compare September 6, 2026 02:36
@lmeyerov lmeyerov changed the title fix(gfql): the endpoint-closure backfill keeps node attribute dtypes on the general chain path (#2058) fix(gfql): every route keeps source dtypes: closure backfill, row-frame projection, 0-row joins, Cypher lanes (#2058) Sep 6, 2026
@lmeyerov

lmeyerov commented Sep 6, 2026

Copy link
Copy Markdown
Contributor Author

GPU receipt (dgx-spark, graphistry/test-rapids-official:26.02-gfql-polars, cudf 26.02.01 / cupy 13.6.0 / polars 1.35.2, safe_run, TEST_CUDF=1; suites test_chain.py, test_hop.py, test_chain_alias_column_collision.py, tests/compute/gfql/**) on the stack head 676232a (#2062, which contains #2055, #2056, #2054 and #2061): 13383 passed / 93 failed / 55 skipped / 44 xfailed / 4 xpassed. The 93 failures are exactly master 86de0f5's set in the same image (13227 passed / 93 failed: the polars-gpu fused grouped-aggregate lane and strictness tests, #2064): set difference empty, zero branch-specific failures.

@lmeyerov
lmeyerov force-pushed the fix/gfql-2058-general-path-int-dtypes branch 2 times, most recently from 640167c to 203845b Compare September 6, 2026 02:50
@lmeyerov

lmeyerov commented Sep 6, 2026

Copy link
Copy Markdown
Contributor Author

Head moved 676232a640167c (squash, tree identical) → 203845b (the row-frame helper's annotations changed from Any to DataFrameT for the type-hygiene guard; no behavior change). The GPU receipt above stands for this tree.

@lmeyerov

lmeyerov commented Sep 6, 2026

Copy link
Copy Markdown
Contributor Author

CI receipt at 203845b: 80 check-runs success, 2 path-skipped (tck-gfql, test-polars ×6 incl. coverage audit, gfql-benchmarks, lint/type/comment guards all green). Local broad CPU 12687 / cuDF 1351 / native-fast-off 549 with 0 failures; GPU receipt on the stack head above. Marking ready.

@lmeyerov

lmeyerov commented Sep 6, 2026

Copy link
Copy Markdown
Contributor Author

GPU receipt (dgx-spark, graphistry/test-rapids-official:26.02-gfql-polars, cudf 26.02.01 / polars 1.35.2, safe_run, TEST_CUDF=1; suites test_chain.py, test_hop.py, test_chain_alias_column_collision.py, tests/compute/gfql/**) on the restacked stack head a54ac4b (#2065, containing #2055 rework, #2056, #2054, #2061, #2062): 13386 passed / 95 failed / 38 xfailed / 4 xpassed. 93 of the failures are master 86de0f5's polars-gpu fused-lane set (#2064, fixed separately in #2066). The other 2 are strict XPASS of the #2043 prune_to_endpoints markers on cuDF 26.02: the prune shapes now agree fast-vs-full on this stack (the #2062 closure fix removed the divergence), so #2062 drops that marker in a follow-up commit (tree otherwise identical; #2065 restacked).

@lmeyerov

lmeyerov commented Sep 6, 2026

Copy link
Copy Markdown
Contributor Author

CI receipt at 7d3c7dc (restacked on the #2055 rework, #2043 prune marker dropped): 80 check-runs success, 2 path-skipped. GPU receipt on the stack head above (the two prune pins pass on cuDF 26.02 at this tree).

@lmeyerov
lmeyerov force-pushed the fix/gfql-2058-general-path-int-dtypes branch from 730aadf to 952adde Compare September 6, 2026 15:33
@lmeyerov
lmeyerov changed the base branch from test/gfql-route-harness to refactor/gfql-route-admission-predicates September 6, 2026 15:34
@lmeyerov lmeyerov changed the title fix(gfql): every route keeps source dtypes: closure backfill, row-frame projection, 0-row joins, Cypher lanes (#2058) fix(gfql): fixes found by the route sweeps: source dtypes on every route, polars prune decline, polars-gpu fused lane (#2058, #2053, #2064) Sep 6, 2026
@lmeyerov
lmeyerov force-pushed the fix/gfql-2058-general-path-int-dtypes branch from 952adde to 04a446a Compare September 6, 2026 15:37
@lmeyerov
lmeyerov force-pushed the refactor/gfql-route-admission-predicates branch from c40b996 to b19ac3c Compare September 6, 2026 15:38
@lmeyerov
lmeyerov force-pushed the fix/gfql-2058-general-path-int-dtypes branch from 04a446a to 500265e Compare September 6, 2026 15:38
@lmeyerov

lmeyerov commented Sep 6, 2026

Copy link
Copy Markdown
Contributor Author

GPU receipt (dgx-spark, graphistry/test-rapids-official:26.02-gfql-polars, cudf 26.02.01 / polars 1.35.2, safe_run, TEST_CUDF=1; suites test_chain.py, test_hop.py, test_chain_alias_column_collision.py, tests/compute/gfql/**) on the folded stack top 500265e (#2062, containing #2056 and #2054): 13532 passed / 7 failed / 37 xfailed / 4 xpassed, versus master 86de0f5's 13227 / 93 in the same image. Every one of the 7 is inside master's set (5 polars-gpu strictness cases and the routing-contract / indexed-bypass pins tracked on #2064 and #1803); zero branch-specific failures, and the fused-lane fix folded from #2066 clears the other 86.

@lmeyerov

lmeyerov commented Sep 6, 2026

Copy link
Copy Markdown
Contributor Author

CI receipt at 500265e (folded: #2058 dtypes + #2053 polars prune decline + #2064 fused lane): 80 check-runs success, 2 path-skipped. Local on this head: broad CPU + cuDF 12939 / 0; tck-gfql 4143 / 689 xfail; GPU receipt above (13532 / 7, all inside master's set).

@lmeyerov
lmeyerov force-pushed the refactor/gfql-route-admission-predicates branch from b19ac3c to b5ba132 Compare September 7, 2026 02:03
@lmeyerov
lmeyerov force-pushed the fix/gfql-2058-general-path-int-dtypes branch from 500265e to c2443da Compare September 7, 2026 02:05
@lmeyerov

lmeyerov commented Sep 7, 2026

Copy link
Copy Markdown
Contributor Author

Rebased onto the new #2054 head b5ba132c2443da (clean rebase; own delta identical before and after, 244/244 lines). Gates on the new head: routes harness, specialization packages, collision matrix, #2058 dtype pins, fused polars lane, lane completeness, chain (788 pass / 10 xfail incl. cuDF). CI 80/80 (+2 filter-skips run locally in that set).

@lmeyerov
lmeyerov force-pushed the refactor/gfql-route-admission-predicates branch from b5ba132 to 9616f11 Compare September 7, 2026 06:06
@lmeyerov
lmeyerov force-pushed the fix/gfql-2058-general-path-int-dtypes branch from c2443da to ad6dc05 Compare September 7, 2026 06:06
@lmeyerov

lmeyerov commented Sep 7, 2026

Copy link
Copy Markdown
Contributor Author

CI on ad6dc05: 80/80 green (+2 filter-skips run locally). Head is final behind #2054.

@lmeyerov
lmeyerov force-pushed the fix/gfql-2058-general-path-int-dtypes branch 2 times, most recently from 3ec6424 to c2443da Compare September 7, 2026 06:51
@lmeyerov
lmeyerov force-pushed the refactor/gfql-route-admission-predicates branch from 2efeb04 to f5cee3b Compare September 7, 2026 06:53
@lmeyerov
lmeyerov force-pushed the fix/gfql-2058-general-path-int-dtypes branch from c2443da to 8af26b8 Compare September 7, 2026 06:55
Base automatically changed from refactor/gfql-route-admission-predicates to master September 7, 2026 07:02
lmeyerov and others added 4 commits September 7, 2026 00:04
…me projection, 0-row joins, Cypher lanes (#2058)

The general path's int64->float64 / bool->object came from one mechanism
(id-only stub rows concatenated onto a node frame and deduplicated
afterwards) that three other places reproduced on purpose. The endpoint
closure appends only endpoints missing from the node frame (dedup kept);
the bindings row frame restores an alias property's node-table dtype once
its bound rows hold no null; the 0-row connected-join path no longer mirrors
the widening; the Cypher seeded lanes keep source dtypes on every route and
_pivot_parity_casts is only the unsupported-dtype gate. Contract change:
pandas RETURN projections and 0-row aggregates of int/bool properties carry
the source dtypes; four pins restated. Pins: closed graph keeps int64/bool
with every route declined (pandas + cuDF); an edge to an absent id is dropped
with dtypes intact; duplicate ids still collapse; a Cypher destination
projection has identical dtypes on the general path and the lane.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QztW7jYsDd66e8rb8pJNQA
The polars chain declined prune_to_endpoints only on multi-hop edges; the
plain single-hop branch and the full polars traversal admitted it and
returned both endpoints where pandas keeps the arrival side (the flag prunes
by the hop labels polars defers). Now any hop with prune_to_endpoints is the
engine's typed NotImplementedError, and the plain single-hop predicate
declines it. Pins: polars raises on the seeded and unseeded single hop while
pandas answers [2, 3]; the predicate declines; the route corpus and the pandas
bypass table lose their #2053 known-divergence tags.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QztW7jYsDd66e8rb8pJNQA
…n the GPU target (#2064)

The low-cardinality count(*) alternative lowers to value_counts + unnest,
and cudf-polars has no unnest map function, so on engine='polars-gpu' every
fused count(*) shape raised inside the lane and declined to the generic
route. On the GPU target the lane now keeps the group_by formulation, which
cudf-polars executes. Pin: the alternative declines under the GPU target and
still produces the UNNEST plan on CPU.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QztW7jYsDd66e8rb8pJNQA
(cherry picked from commit e1eb737)
@lmeyerov
lmeyerov force-pushed the fix/gfql-2058-general-path-int-dtypes branch from 8af26b8 to 6910594 Compare September 7, 2026 07:05
@lmeyerov

lmeyerov commented Sep 7, 2026

Copy link
Copy Markdown
Contributor Author

Rebased onto master c2d3a07 (#2054 merged) → 6910594; clean rebase, own delta identical (0 drift lines). Gates on the new head: routes harness incl. frame variants, both specialization packages, #2058 dtype pins, fused polars lane, lane completeness: 745 pass / 37 xfail incl. cuDF; lint clean. CI re-fired.

@lmeyerov

lmeyerov commented Sep 7, 2026

Copy link
Copy Markdown
Contributor Author

CI on 6910594: 82/82 green (+2 filter-skips run locally). Ready to merge.

@lmeyerov
lmeyerov merged commit 1a41079 into master Sep 7, 2026
85 checks passed
@lmeyerov
lmeyerov deleted the fix/gfql-2058-general-path-int-dtypes branch September 7, 2026 18:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant