Skip to content

Conversation

gongxun0928
Copy link
Contributor

this pr is base on #1324

feat: pushdown bloom filter to pax table am

This optimization pushes down Bloom Filter conditions for runtime filters
to the Pax Table AM layer.

By applying the filter earlier than the SeqNext() function, it eliminates
the overhead of converting data from columnar format to TableTupleSlot,
resulting in faster query execution

CREATE TABLE t2(c1 int, c2 int, c3 int, c4 int, c5 int) with (appendonly=true, orientation=column) distributed REPLICATED;
INSERT INTO t2 VALUES (1,1,1,1,1), (2,2,2,2,2), (3,3,3,3,3), (4,4,4,4,4);
INSERT INTO t2 select * FROM t2;
INSERT INTO t2 select * FROM t2;
INSERT INTO t2 select * FROM t2;
CREATE TABLE t3(c1 int, c2 int, c3 int, c4 int, c5 int) using pax;


gpadmin=# insert into t3 select b,i,i,i,i from generate_series(1,5) b,generate_series(1,2000000) i;

INSERT 0 10000000
gpadmin=#
gpadmin=# analyze t3;
ANALYZE
gpadmin=# EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF)
SELECT t3.c3 FROM t3, t2 WHERE t3.c2 = t2.c4;
                                        QUERY PLAN
-------------------------------------------------------------------------------------------
 Gather Motion 3:1  (slice1; segments: 3) (actual time=355.000..1482.000 rows=160 loops=1)
   ->  Hash Join (actual time=0.000..1482.000 rows=96 loops=1)
         Hash Cond: (t3.c2 = t2.c4)
         Extra Text: (seg0)   Hash chain length 8.0 avg, 8 max, using 4 of 524288 buckets.
         ->  Seq Scan on t3 (actual time=0.000..691.000 rows=6000000 loops=1)
         ->  Hash (actual time=0.000..0.000 rows=32 loops=1)
               Buckets: 524288  Batches: 1  Memory Usage: 4098kB
               ->  Seq Scan on t2 (actual time=0.000..0.000 rows=32 loops=1)
 Optimizer: GPORCA
(9 rows)

gpadmin=# set gp_enable_runtime_filter_pushdown to on;
SET
gpadmin=# EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF)
SELECT t3.c3 FROM t3, t2 WHERE t3.c2 = t2.c4;
                                        QUERY PLAN
-------------------------------------------------------------------------------------------
 Gather Motion 3:1  (slice1; segments: 3) (actual time=99.000..303.000 rows=160 loops=1)
   ->  Hash Join (actual time=1.000..303.000 rows=96 loops=1)
         Hash Cond: (t3.c2 = t2.c4)
         Extra Text: (seg0)   Hash chain length 8.0 avg, 8 max, using 4 of 524288 buckets.
         ->  Seq Scan on t3 (actual time=1.000..301.000 rows=12 loops=1)
               Rows Removed by Pushdown Runtime Filter: 5999988
         ->  Hash (actual time=0.000..0.000 rows=32 loops=1)
               Buckets: 524288  Batches: 1  Memory Usage: 4098kB
               ->  Seq Scan on t2 (actual time=0.000..0.000 rows=32 loops=1)
 Optimizer: GPORCA
(10 rows)

gpadmin=# set pax_enable_row_filter to on;
SET
gpadmin=# EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF)
SELECT t3.c3 FROM t3, t2 WHERE t3.c2 = t2.c4;
                                        QUERY PLAN
-------------------------------------------------------------------------------------------
 Gather Motion 3:1  (slice1; segments: 3) (actual time=48.000..138.000 rows=160 loops=1)
   ->  Hash Join (actual time=0.000..138.000 rows=96 loops=1)
         Hash Cond: (t3.c2 = t2.c4)
         Extra Text: (seg0)   Hash chain length 8.0 avg, 8 max, using 4 of 524288 buckets.
         ->  Seq Scan on t3 (actual time=0.000..137.000 rows=12 loops=1)
               Rows Removed by Pushdown Runtime Filter: 5999988
         ->  Hash (actual time=0.000..0.000 rows=32 loops=1)
               Buckets: 524288  Batches: 1  Memory Usage: 4098kB
               ->  Seq Scan on t2 (actual time=0.000..0.000 rows=32 loops=1)
 Optimizer: GPORCA
(10 rows)

What does this PR do?

Type of Change

  • Bug fix (non-breaking change)
  • New feature (non-breaking change)
  • Breaking change (fix or feature with breaking changes)
  • Documentation update

Breaking Changes

Test Plan

  • Unit tests added/updated
  • Integration tests added/updated
  • Passed make installcheck
  • Passed make -C src/test installcheck-cbdb-parallel

Impact

Performance:

User-facing changes:

Dependencies:

Checklist

Additional Context

CI Skip Instructions


@gongxun0928 gongxun0928 force-pushed the pax/support-bloom-filter-pushdown branch from a02e2c3 to 1fb74ea Compare August 27, 2025 10:31
@gongxun0928 gongxun0928 force-pushed the pax/support-bloom-filter-pushdown branch from 1fb74ea to f83db6e Compare September 10, 2025 17:19
@gongxun0928 gongxun0928 force-pushed the pax/support-bloom-filter-pushdown branch from f83db6e to 52f7483 Compare October 10, 2025 02:39
This optimization pushes down Bloom Filter conditions for runtime filters
to the Pax Table AM layer.

By applying the filter earlier than the SeqNext() function, it eliminates
the overhead of converting data from columnar format to TableTupleSlot,
resulting in faster query execution
@gongxun0928 gongxun0928 force-pushed the pax/support-bloom-filter-pushdown branch from 52f7483 to 8b62803 Compare October 10, 2025 06:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant