PG17 compatibility: ruleutils - #7725
Merged
Merged
Conversation
(cherry picked from commit 728fe5c)
Relevant PG commit: 03734a7fed7d924679770adb78a7db8a37d14188 postgres/postgres@03734a7 (cherry picked from commit 978fdc2)
…CORD Vars. Relevant PG commit: e0e492e5a928e9c9eda01eeebadcfc36f9f8e7b7 postgres/postgres@e0e492e (cherry picked from commit 37ac804)
Relevant PG commit: 2075ba9dc9af8833b1147996c15246e44a1e4fab postgres/postgres@2075ba9 (cherry picked from commit 31def24)
Relevant PG commit: 97957fdbaa429c7c582d4753b108cb1e23e1b28a postgres/postgres@97957fd (cherry picked from commit 39ec86a)
Relevant PG commit: 29275b1d177096597675b5c6e7e7c9db2df8f4df postgres/postgres@29275b1 (cherry picked from commit fe3414f)
Relevant PG commit: dbbca2cf299b81299112ca6ada671a36235ec008 postgres/postgres@dbbca2c (cherry picked from commit 2e4bc1f)
Relevant PG commit: 33e729c5148c3a697abc552621b34bdc5fd497ed postgres/postgres@33e729c (cherry picked from commit 61b92ee)
Relevant PG commit: c649fa24a42ba89bf5460c7110e4fc8eeca65959 postgres/postgres@c649fa2 (cherry picked from commit c6c1ff2)
…rameters. Relevant PG commit: fd0398fcb099980fbedbb7750356ef234408c1c9 postgres/postgres@fd0398f (cherry picked from commit 6889367)
Relevant PG commit: 6185c9737cf48c9540782d88f12bd2912d6ca1cc postgres/postgres@6185c97 (cherry picked from commit d2648f4)
Relevant PG commit: 0294df2f1f842dfb0eed79007b21016f486a3c6c postgres/postgres@0294df2 (cherry picked from commit 6b4889f)
Relevant PG commit: de3600452b61d1bc3967e9e37e86db8956c8f577 postgres/postgres@de36004 (cherry picked from commit 662fc76)
Relevant PG commit: 87c21bb9412c8ba2727dec5ebcd74d44c2232d11 postgres/postgres@87c21bb (cherry picked from commit eefe0ae)
Relevant PG commit: f6a2529920cff76cb6e37ea840122574404dde8b postgres/postgres@f6a2529 (cherry picked from commit 661fcd6)
Relevant PG commit: bb766cde63b4f624d029b34c9cdd3d0a94fd5b46 postgres/postgres@bb766cd (cherry picked from commit 9c06754)
Relevant PG commit: 8461424fd717877ead0706984ef9b6440b2a97ad postgres/postgres@8461424 (cherry picked from commit c9f9d37)
Relevant PG commit: a42fc1c903b54ba94374e5e0c08905b9a1479f19 postgres/postgres@a42fc1c (cherry picked from commit 8e6dc39)
…ING. Relevant PG commit: 5c5bccef211cfc98e0d6c4bc1af40a33c8ac2488 postgres/postgres@5c5bcce (cherry picked from commit bd10f1a)
…LAIN. Relevant PG commit: postgres/postgres@81a12a4 81a12a4477533d7722bd6b6dac88b0e798f8e85b
…_var_field. Relevant PG commit: postgres/postgres@aed8813 aed881386aa6a6a542e46d14d3505e4e6f9310a0
…(S) commands Relevant PG commit: postgres/postgres@84f594d 84f594da358861cceeaeb7a97bb58f3765eeb284
… conflict. Relevant PG commit: postgres/postgres@a7eb633 a7eb633563c6ba8857cf59c7b43b593614537617
Relevant PG commit: postgres/postgres@c88ce38 c88ce386c4d7bfeb437ff31ec7c23c392c862e77
Relevant PG commit: postgres/postgres@5067c23 5067c230b8ee42a01cc77dc5745bc3a78f393af3
Reverts 8ad3592 def847f Relevant PG commit: postgres/postgres@eef5195 eef5195f300bb9cf2864d48761c0db2ad93842c1
Relevant PG commit: postgres/postgres@cd680b3 cd680b39211c5c3c88a143abcac576a22f996d7a
Relevant PG commit: postgres/postgres@446d5ad 446d5ad7ae7d3bf4fd08904ae54a6399cafb4e7d
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## release-13.0 #7725 +/- ##
===============================================
Coverage ? 89.64%
===============================================
Files ? 274
Lines ? 59583
Branches ? 7436
===============================================
Hits ? 53414
Misses ? 4038
Partials ? 2131 |
Onur Tirtir (onurctirtir)
approved these changes
Nov 11, 2024
Naisila Puka (naisila)
added a commit
that referenced
this pull request
Dec 24, 2024
This is the final commit that adds PG17 compatibility with Citus's current capabilities. You can use Citus community, release-13.0 branch, with PG17.1. --------- Specifically, this commit: - Enables PG17 in the configure script. - Adds PG17 tests to CI using test images that have 17.1 - Fixes an upgrade test: see below for details In `citus_prepare_upgrade()`, don't drop any_value when upgrading from PG16+, because PG16+ has its own any_value function. Attempting to do so results in the error seen in [pg16-pg17 upgrade](https://github.com/citusdata/citus/actions/runs/11768444117/job/32778340003?pr=7661): ``` ERROR: cannot drop function any_value(anyelement) because it is required by the database system CONTEXT: SQL statement "DROP AGGREGATE IF EXISTS pg_catalog.any_value(anyelement)" ``` When 16 becomes the minimum supported Postgres version, the drop statements can be removed. --------- Several PG17 Compatibility commits have been merged before this final one. All these subtasks are done #7653 See the list below: Compilation PR: #7699 Ruleutils PR: #7725 Sister PR for tests: citusdata/the-process#159 Helpful smaller PRs: - #7714 - #7726 - #7731 - #7732 - #7733 - #7738 - #7745 - #7747 - #7748 - #7749 - #7752 - #7755 - #7757 - #7759 - #7760 - #7761 - #7762 - #7765 - #7766 - #7768 - #7769 - #7771 - #7774 - #7776 - #7780 - #7781 - #7785 - #7788 - #7793 - #7796 --------- Co-authored-by: Colm <colmmchugh@microsoft.com>
Naisila Puka (naisila)
added a commit
that referenced
this pull request
Mar 11, 2025
PG17 compatibility - Part 2 #7699 was the first PG17 compatibility PR merged to main branch, which provided ONLY successful Citus compilation with PG17.0. This PR, consider it as Part 2, provides ruleutils changes for PG17. Ruleutils changes is the first thing we should merge, after successful build. It's the core for deparsing logic in Citus. # Question: How do we add ruleutils changes? - We add a new ruleutils file specific to PG17. - We keep track of the changes in Postgres's ruleutils file from here https://github.com/postgres/postgres/commits/REL_17_0/src/backend/utils/adt/ruleutils.c - Per each commit in that history that belongs only to 17.0, we add the relevant changes to static functions to our ruleutils file for PG17. It's like a manual commit copying. # Check the PR's commits for detailed steps https://github.com/citusdata/citus/pull/7725/commits
Naisila Puka (naisila)
added a commit
that referenced
this pull request
Mar 11, 2025
This is the final commit that adds PG17 compatibility with Citus's current capabilities. You can use Citus community, release-13.0 branch, with PG17.1. --------- Specifically, this commit: - Enables PG17 in the configure script. - Adds PG17 tests to CI using test images that have 17.1 - Fixes an upgrade test: see below for details In `citus_prepare_upgrade()`, don't drop any_value when upgrading from PG16+, because PG16+ has its own any_value function. Attempting to do so results in the error seen in [pg16-pg17 upgrade](https://github.com/citusdata/citus/actions/runs/11768444117/job/32778340003?pr=7661): ``` ERROR: cannot drop function any_value(anyelement) because it is required by the database system CONTEXT: SQL statement "DROP AGGREGATE IF EXISTS pg_catalog.any_value(anyelement)" ``` When 16 becomes the minimum supported Postgres version, the drop statements can be removed. --------- Several PG17 Compatibility commits have been merged before this final one. All these subtasks are done #7653 See the list below: Compilation PR: #7699 Ruleutils PR: #7725 Sister PR for tests: citusdata/the-process#159 Helpful smaller PRs: - #7714 - #7726 - #7731 - #7732 - #7733 - #7738 - #7745 - #7747 - #7748 - #7749 - #7752 - #7755 - #7757 - #7759 - #7760 - #7761 - #7762 - #7765 - #7766 - #7768 - #7769 - #7771 - #7774 - #7776 - #7780 - #7781 - #7785 - #7788 - #7793 - #7796 --------- Co-authored-by: Colm <colmmchugh@microsoft.com>
Naisila Puka (naisila)
added a commit
that referenced
this pull request
Mar 12, 2025
PG17 compatibility - Part 2 #7699 was the first PG17 compatibility PR merged to main branch, which provided ONLY successful Citus compilation with PG17.0. This PR, consider it as Part 2, provides ruleutils changes for PG17. Ruleutils changes is the first thing we should merge, after successful build. It's the core for deparsing logic in Citus. # Question: How do we add ruleutils changes? - We add a new ruleutils file specific to PG17. - We keep track of the changes in Postgres's ruleutils file from here https://github.com/postgres/postgres/commits/REL_17_0/src/backend/utils/adt/ruleutils.c - Per each commit in that history that belongs only to 17.0, we add the relevant changes to static functions to our ruleutils file for PG17. It's like a manual commit copying. # Check the PR's commits for detailed steps https://github.com/citusdata/citus/pull/7725/commits
Naisila Puka (naisila)
added a commit
that referenced
this pull request
Mar 12, 2025
This is the final commit that adds PG17 compatibility with Citus's current capabilities. You can use Citus community, release-13.0 branch, with PG17.1. --------- Specifically, this commit: - Enables PG17 in the configure script. - Adds PG17 tests to CI using test images that have 17.1 - Fixes an upgrade test: see below for details In `citus_prepare_upgrade()`, don't drop any_value when upgrading from PG16+, because PG16+ has its own any_value function. Attempting to do so results in the error seen in [pg16-pg17 upgrade](https://github.com/citusdata/citus/actions/runs/11768444117/job/32778340003?pr=7661): ``` ERROR: cannot drop function any_value(anyelement) because it is required by the database system CONTEXT: SQL statement "DROP AGGREGATE IF EXISTS pg_catalog.any_value(anyelement)" ``` When 16 becomes the minimum supported Postgres version, the drop statements can be removed. --------- Several PG17 Compatibility commits have been merged before this final one. All these subtasks are done #7653 See the list below: Compilation PR: #7699 Ruleutils PR: #7725 Sister PR for tests: citusdata/the-process#159 Helpful smaller PRs: - #7714 - #7726 - #7731 - #7732 - #7733 - #7738 - #7745 - #7747 - #7748 - #7749 - #7752 - #7755 - #7757 - #7759 - #7760 - #7761 - #7762 - #7765 - #7766 - #7768 - #7769 - #7771 - #7774 - #7776 - #7780 - #7781 - #7785 - #7788 - #7793 - #7796 --------- Co-authored-by: Colm <colmmchugh@microsoft.com>
Naisila Puka (naisila)
added a commit
that referenced
this pull request
Mar 12, 2025
This is the final commit that adds PG17 compatibility with Citus's current capabilities. You can use Citus community, release-13.0 branch, with PG17.1. --------- Specifically, this commit: - Enables PG17 in the configure script. - Adds PG17 tests to CI using test images that have 17.1 - Fixes an upgrade test: see below for details In `citus_prepare_upgrade()`, don't drop any_value when upgrading from PG16+, because PG16+ has its own any_value function. Attempting to do so results in the error seen in [pg16-pg17 upgrade](https://github.com/citusdata/citus/actions/runs/11768444117/job/32778340003?pr=7661): ``` ERROR: cannot drop function any_value(anyelement) because it is required by the database system CONTEXT: SQL statement "DROP AGGREGATE IF EXISTS pg_catalog.any_value(anyelement)" ``` When 16 becomes the minimum supported Postgres version, the drop statements can be removed. --------- Several PG17 Compatibility commits have been merged before this final one. All these subtasks are done #7653 See the list below: Compilation PR: #7699 Ruleutils PR: #7725 Sister PR for tests: citusdata/the-process#159 Helpful smaller PRs: - #7714 - #7726 - #7731 - #7732 - #7733 - #7738 - #7745 - #7747 - #7748 - #7749 - #7752 - #7755 - #7757 - #7759 - #7760 - #7761 - #7762 - #7765 - #7766 - #7768 - #7769 - #7771 - #7774 - #7776 - #7780 - #7781 - #7785 - #7788 - #7793 - #7796 --------- Co-authored-by: Colm <colmmchugh@microsoft.com>
ibrahim halatci (ihalatci)
added a commit
that referenced
this pull request
May 29, 2026
Scaffold only, cloned from ruleutils_18.c retargeted at the PG19 version guard so the binary links (per the #7725/#8010 precedent; the real upstream-derived port lands in a follow-up PR). Exclude the file from citus-style in .gitattributes: it is an upstream-derived clone exceeding the CI uncrustify 10,000-line limit.
ibrahim halatci (ihalatci)
added a commit
that referenced
this pull request
May 29, 2026
Scaffold cloned from ruleutils_18.c retargeted at the PG19 version guard, plus the minimal source changes required for it to compile under PG19's -Werror (the real upstream-derived deparse port lands in a follow-up PR, per the #7725/#8010 precedent): - get_name_for_var_field(): handle the new PG19 RTE_GRAPH_TABLE rtekind in the switch (grouped with the subquery cases, as upstream does). - isSimpleNode(): replace the comment-style /* FALLTHROUGH */ with pg_fallthrough; (PG19 builds with -Wimplicit-fallthrough=5, which no longer accepts comment markers). - generate_function_name(): pass the new &fgc_flags argument to func_get_detail(), which gained a parameter in PG19. Exclude the file from citus-style in .gitattributes: it is an upstream-derived clone exceeding the CI uncrustify 10,000-line limit.
ibrahim halatci (ihalatci)
added a commit
that referenced
this pull request
May 29, 2026
DESCRIPTION: Port upstream PG19 ruleutils.c into Citus deparser tree. Mirrors #8010 (PG18) and #7725 (PG17). Replaces the ruleutils_19.c placeholder from the build-foundation PR with a proper port of upstream PG19's src/backend/utils/adt/ruleutils.c, produced by a 3-way merge (git merge-file: ruleutils_18.c as base, upstream PG18 and upstream PG19 as the two sides). All merge conflicts fell in blocks Citus had already stripped from _18.c or in shard-aware deparse variants, and were resolved by keeping the Citus side. The file keeps Citus' existing curation (deparse hooks, shard-aware UPDATE/DELETE branches) while picking up real upstream PG18->PG19 deparse changes. Without this, Citus on a PG19 backend would deparse worker-bound SQL using PG18 semantics -- silent corruption the moment a PG19-only node, clause, or formatting decision appears in the tree. The trailing #endif guard text is corrected to "(PG_VERSION_NUM >= PG_VERSION_19) && (PG_VERSION_NUM < PG_VERSION_20)" (the _17.c/_18.c snapshots carry stale guard comments; not propagated). The generate_function_name fgc_flags fix is absorbed naturally by the merge -- upstream PG19 already passes &fgc_flags at that call site. Excluded from citus-style via .gitattributes (added in the build-foundation PR): the file exceeds the CI uncrustify 10,000-line limit and is upstream-derived. Refs: #8597
ibrahim halatci (ihalatci)
added a commit
that referenced
this pull request
Jun 6, 2026
… CI image bump) (#8601) DESCRIPTION: Add PG19 build support; extension compiles and loads. First PR of the PG19 enablement series tracked in #8597. Achieves source-level compatibility with PG19 (`pg19devel`) and a clean `CREATE EXTENSION citus` while keeping PG17 and PG18 builds green. Regression-test correctness and PG19 feature integration land in follow-up PRs against the `pg19-support` development branch. ## Commits 1. **ci** — bump `image_suffix` to `-dev-e11d99c` (PG19-capable test images) and PG minors to 16.14/17.10/18.4; add `pg19_version` and wire `19devel` into the **build matrix only**. 2. **source-level compatibility + shmem sizing** — accept PG19 in `configure` / `pg_version_constants.h`; centralised compat shims in `pg_version_compat.h`; distributed/columnar header adaptations for renamed/retyped PG19 APIs; 128 KB shmem slack on PG19 for the new `ShmemInitHash` up-front allocation, plus the `maintenanced.c` double-count fix that change exposes. 3. **-Werror source fixes** — warning fixes across 56 files (old-style prototypes, strict forward declarations, implicit-fallthrough, new switch arms, API renames). **No compiler-flag demotion** — the build keeps the same warning/error flags as PG17/PG18. 4. **ruleutils_19.c scaffold** — clone of `ruleutils_18.c` retargeted at the PG19 version guard so the binary links (per the #7725 / #8010 precedent; the real upstream-derived port lands in a follow-up PR). Excluded from citus-style via `.gitattributes` — it is an upstream-derived clone exceeding the CI uncrustify 10,000-line limit. 5. **test** — add PG19 to the oldest-supported Citus version matrix. ## Scope / non-goals - **Build path only.** PG19 is intentionally NOT added to `test-citus`, `test-citus-failure`, `test-citus-cdc`, or the upgrade jobs: the regression suite needs the upstream-derived `ruleutils_19.c` from the follow-up PR, and no released Citus supports PG19 yet. Running the suite on this PR alone produces silent PG18-on-PG19 deparse corruption. - `multi_explain.c` is touched only for the `Instrumentation` / `NodeInstrumentation` rename; a separate `TupleDescFinalize` runtime fix lands in a later PR. ## Validation - PG19devel / PG17.10 / PG18.4: build exits 0 (validated in CI images). - `CREATE EXTENSION citus` succeeds on PG19. - No regression-suite changes here. Refs: #8597
ibrahim halatci (ihalatci)
added a commit
that referenced
this pull request
Jun 13, 2026
DESCRIPTION: Port upstream PG19 ruleutils.c into Citus deparser tree. Mirrors #8010 (PG18) and #7725 (PG17). Replaces the ruleutils_19.c placeholder from the build-foundation PR with a proper port of upstream PG19's src/backend/utils/adt/ruleutils.c, produced by a 3-way merge (git merge-file: ruleutils_18.c as base, upstream PG18 and upstream PG19 as the two sides). All merge conflicts fell in blocks Citus had already stripped from _18.c or in shard-aware deparse variants, and were resolved by keeping the Citus side. The file keeps Citus' existing curation (deparse hooks, shard-aware UPDATE/DELETE branches) while picking up real upstream PG18->PG19 deparse changes. Without this, Citus on a PG19 backend would deparse worker-bound SQL using PG18 semantics -- silent corruption the moment a PG19-only node, clause, or formatting decision appears in the tree. The trailing #endif guard text is corrected to "(PG_VERSION_NUM >= PG_VERSION_19) && (PG_VERSION_NUM < PG_VERSION_20)" (the _17.c/_18.c snapshots carry stale guard comments; not propagated). The generate_function_name fgc_flags fix is absorbed naturally by the merge -- upstream PG19 already passes &fgc_flags at that call site. Excluded from citus-style via .gitattributes (added in the build-foundation PR): the file exceeds the CI uncrustify 10,000-line limit and is upstream-derived. Refs: #8597
ibrahim halatci (ihalatci)
added a commit
that referenced
this pull request
Jun 16, 2026
DESCRIPTION: Port upstream PG19 ruleutils.c into Citus deparser tree. Mirrors #8010 (PG18) and #7725 (PG17). Replaces the ruleutils_19.c placeholder from the build-foundation PR with a proper port of upstream PG19's src/backend/utils/adt/ruleutils.c, produced by a 3-way merge (git merge-file: ruleutils_18.c as base, upstream PG18 and upstream PG19 as the two sides). All merge conflicts fell in blocks Citus had already stripped from _18.c or in shard-aware deparse variants, and were resolved by keeping the Citus side. The file keeps Citus' existing curation (deparse hooks, shard-aware UPDATE/DELETE branches) while picking up real upstream PG18->PG19 deparse changes. Without this, Citus on a PG19 backend would deparse worker-bound SQL using PG18 semantics -- silent corruption the moment a PG19-only node, clause, or formatting decision appears in the tree. The trailing #endif guard text is corrected to "(PG_VERSION_NUM >= PG_VERSION_19) && (PG_VERSION_NUM < PG_VERSION_20)" (the _17.c/_18.c snapshots carry stale guard comments; not propagated). The generate_function_name fgc_flags fix is absorbed naturally by the merge -- upstream PG19 already passes &fgc_flags at that call site. Excluded from citus-style via .gitattributes (added in the build-foundation PR): the file exceeds the CI uncrustify 10,000-line limit and is upstream-derived. Refs: #8597 --- Stacked on #8601 (`pg19-build-foundation`).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
PG17 compatibility - Part 2
#7699 was the first PG17 compatibility PR merged to main branch, which provided ONLY successful Citus compilation with PG17.0.
This PR, consider it as Part 2, provides ruleutils changes for PG17. Ruleutils changes is the first thing we should merge, after successful build. It's the core for deparsing logic in Citus.
Question: How do we add ruleutils changes?
Answer:
Notes for reviewer