Fix HideCitusDependentObjectsOnQueriesOfPgMetaTables function - #7796
Merged
Naisila Puka (naisila) merged 1 commit intoDec 20, 2024
Merged
Conversation
Naisila Puka (naisila)
requested review from
Onur Tirtir (onurctirtir) and
Teja Mupparti (tejeswarm)
December 20, 2024 13:23
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## release-13.0 #7796 +/- ##
===============================================
Coverage ? 89.60%
===============================================
Files ? 274
Lines ? 59730
Branches ? 7451
===============================================
Hits ? 53520
Misses ? 4078
Partials ? 2132 |
Onur Tirtir (onurctirtir)
approved these changes
Dec 20, 2024
Naisila Puka (naisila)
marked this pull request as ready for review
December 20, 2024 14:43
The function should take into account the new entry for MERGE joins that is introduced in PG17 -> mergeJoinCondition
Naisila Puka (naisila)
force-pushed
the
naisila/fix-mergeJoinCondition
branch
from
December 20, 2024 14:44
f6a8223 to
bd1fbff
Compare
Naisila Puka (naisila)
changed the base branch from
naisila/pg17_support
to
release-13.0
December 20, 2024 14:58
This was referenced Dec 20, 2024
Naisila Puka (naisila)
added a commit
that referenced
this pull request
Dec 24, 2024
This is the final commit that adds PG17 compatibility with Citus's current capabilities. You can use Citus community, release-13.0 branch, with PG17.1. --------- Specifically, this commit: - Enables PG17 in the configure script. - Adds PG17 tests to CI using test images that have 17.1 - Fixes an upgrade test: see below for details In `citus_prepare_upgrade()`, don't drop any_value when upgrading from PG16+, because PG16+ has its own any_value function. Attempting to do so results in the error seen in [pg16-pg17 upgrade](https://github.com/citusdata/citus/actions/runs/11768444117/job/32778340003?pr=7661): ``` ERROR: cannot drop function any_value(anyelement) because it is required by the database system CONTEXT: SQL statement "DROP AGGREGATE IF EXISTS pg_catalog.any_value(anyelement)" ``` When 16 becomes the minimum supported Postgres version, the drop statements can be removed. --------- Several PG17 Compatibility commits have been merged before this final one. All these subtasks are done #7653 See the list below: Compilation PR: #7699 Ruleutils PR: #7725 Sister PR for tests: citusdata/the-process#159 Helpful smaller PRs: - #7714 - #7726 - #7731 - #7732 - #7733 - #7738 - #7745 - #7747 - #7748 - #7749 - #7752 - #7755 - #7757 - #7759 - #7760 - #7761 - #7762 - #7765 - #7766 - #7768 - #7769 - #7771 - #7774 - #7776 - #7780 - #7781 - #7785 - #7788 - #7793 - #7796 --------- Co-authored-by: Colm <colmmchugh@microsoft.com>
Naisila Puka (naisila)
added a commit
that referenced
this pull request
Mar 11, 2025
There is a crash when running vanilla tests because of the `citus.hide_citus_dependent_objects` GUC. We turn on this GUC only for the pg vanilla tests. This GUC runs the following function `HideCitusDependentObjectsOnQueriesOfPgMetaTables`. This function doesn't take into account the new `mergeJoinCondition`. I rewrote the function such that it checks for merge join conditions as well. Relevant PG commit: postgres/postgres@0294df2f1 The crash could be reproduced locally like the following: ```SQL SET citus.hide_citus_dependent_objects TO on; CREATE OR REPLACE FUNCTION pg_catalog.is_citus_depended_object(oid,oid) RETURNS bool LANGUAGE C AS 'citus', $$is_citus_depended_object$$; -- try a system catalog MERGE INTO pg_class c USING (SELECT 'pg_depend'::regclass AS oid) AS j ON j.oid = c.oid WHEN MATCHED THEN UPDATE SET reltuples = reltuples + 1 RETURNING j.oid; CREATE VIEW classv AS SELECT * FROM pg_class; MERGE INTO classv c USING pg_namespace n ON n.oid = c.relnamespace WHEN MATCHED AND c.oid = 'pg_depend'::regclass THEN UPDATE SET reltuples = reltuples - 1 RETURNING c.oid; -- crash happens here ```
Naisila Puka (naisila)
added a commit
that referenced
this pull request
Mar 11, 2025
This is the final commit that adds PG17 compatibility with Citus's current capabilities. You can use Citus community, release-13.0 branch, with PG17.1. --------- Specifically, this commit: - Enables PG17 in the configure script. - Adds PG17 tests to CI using test images that have 17.1 - Fixes an upgrade test: see below for details In `citus_prepare_upgrade()`, don't drop any_value when upgrading from PG16+, because PG16+ has its own any_value function. Attempting to do so results in the error seen in [pg16-pg17 upgrade](https://github.com/citusdata/citus/actions/runs/11768444117/job/32778340003?pr=7661): ``` ERROR: cannot drop function any_value(anyelement) because it is required by the database system CONTEXT: SQL statement "DROP AGGREGATE IF EXISTS pg_catalog.any_value(anyelement)" ``` When 16 becomes the minimum supported Postgres version, the drop statements can be removed. --------- Several PG17 Compatibility commits have been merged before this final one. All these subtasks are done #7653 See the list below: Compilation PR: #7699 Ruleutils PR: #7725 Sister PR for tests: citusdata/the-process#159 Helpful smaller PRs: - #7714 - #7726 - #7731 - #7732 - #7733 - #7738 - #7745 - #7747 - #7748 - #7749 - #7752 - #7755 - #7757 - #7759 - #7760 - #7761 - #7762 - #7765 - #7766 - #7768 - #7769 - #7771 - #7774 - #7776 - #7780 - #7781 - #7785 - #7788 - #7793 - #7796 --------- Co-authored-by: Colm <colmmchugh@microsoft.com>
Naisila Puka (naisila)
added a commit
that referenced
this pull request
Mar 12, 2025
There is a crash when running vanilla tests because of the `citus.hide_citus_dependent_objects` GUC. We turn on this GUC only for the pg vanilla tests. This GUC runs the following function `HideCitusDependentObjectsOnQueriesOfPgMetaTables`. This function doesn't take into account the new `mergeJoinCondition`. I rewrote the function such that it checks for merge join conditions as well. Relevant PG commit: postgres/postgres@0294df2f1 The crash could be reproduced locally like the following: ```SQL SET citus.hide_citus_dependent_objects TO on; CREATE OR REPLACE FUNCTION pg_catalog.is_citus_depended_object(oid,oid) RETURNS bool LANGUAGE C AS 'citus', $$is_citus_depended_object$$; -- try a system catalog MERGE INTO pg_class c USING (SELECT 'pg_depend'::regclass AS oid) AS j ON j.oid = c.oid WHEN MATCHED THEN UPDATE SET reltuples = reltuples + 1 RETURNING j.oid; CREATE VIEW classv AS SELECT * FROM pg_class; MERGE INTO classv c USING pg_namespace n ON n.oid = c.relnamespace WHEN MATCHED AND c.oid = 'pg_depend'::regclass THEN UPDATE SET reltuples = reltuples - 1 RETURNING c.oid; -- crash happens here ```
Naisila Puka (naisila)
added a commit
that referenced
this pull request
Mar 12, 2025
This is the final commit that adds PG17 compatibility with Citus's current capabilities. You can use Citus community, release-13.0 branch, with PG17.1. --------- Specifically, this commit: - Enables PG17 in the configure script. - Adds PG17 tests to CI using test images that have 17.1 - Fixes an upgrade test: see below for details In `citus_prepare_upgrade()`, don't drop any_value when upgrading from PG16+, because PG16+ has its own any_value function. Attempting to do so results in the error seen in [pg16-pg17 upgrade](https://github.com/citusdata/citus/actions/runs/11768444117/job/32778340003?pr=7661): ``` ERROR: cannot drop function any_value(anyelement) because it is required by the database system CONTEXT: SQL statement "DROP AGGREGATE IF EXISTS pg_catalog.any_value(anyelement)" ``` When 16 becomes the minimum supported Postgres version, the drop statements can be removed. --------- Several PG17 Compatibility commits have been merged before this final one. All these subtasks are done #7653 See the list below: Compilation PR: #7699 Ruleutils PR: #7725 Sister PR for tests: citusdata/the-process#159 Helpful smaller PRs: - #7714 - #7726 - #7731 - #7732 - #7733 - #7738 - #7745 - #7747 - #7748 - #7749 - #7752 - #7755 - #7757 - #7759 - #7760 - #7761 - #7762 - #7765 - #7766 - #7768 - #7769 - #7771 - #7774 - #7776 - #7780 - #7781 - #7785 - #7788 - #7793 - #7796 --------- Co-authored-by: Colm <colmmchugh@microsoft.com>
Naisila Puka (naisila)
added a commit
that referenced
this pull request
Mar 12, 2025
There is a crash when running vanilla tests because of the `citus.hide_citus_dependent_objects` GUC. We turn on this GUC only for the pg vanilla tests. This GUC runs the following function `HideCitusDependentObjectsOnQueriesOfPgMetaTables`. This function doesn't take into account the new `mergeJoinCondition`. I rewrote the function such that it checks for merge join conditions as well. Relevant PG commit: postgres/postgres@0294df2f1 The crash could be reproduced locally like the following: ```SQL SET citus.hide_citus_dependent_objects TO on; CREATE OR REPLACE FUNCTION pg_catalog.is_citus_depended_object(oid,oid) RETURNS bool LANGUAGE C AS 'citus', $$is_citus_depended_object$$; -- try a system catalog MERGE INTO pg_class c USING (SELECT 'pg_depend'::regclass AS oid) AS j ON j.oid = c.oid WHEN MATCHED THEN UPDATE SET reltuples = reltuples + 1 RETURNING j.oid; CREATE VIEW classv AS SELECT * FROM pg_class; MERGE INTO classv c USING pg_namespace n ON n.oid = c.relnamespace WHEN MATCHED AND c.oid = 'pg_depend'::regclass THEN UPDATE SET reltuples = reltuples - 1 RETURNING c.oid; -- crash happens here ```
Naisila Puka (naisila)
added a commit
that referenced
this pull request
Mar 12, 2025
This is the final commit that adds PG17 compatibility with Citus's current capabilities. You can use Citus community, release-13.0 branch, with PG17.1. --------- Specifically, this commit: - Enables PG17 in the configure script. - Adds PG17 tests to CI using test images that have 17.1 - Fixes an upgrade test: see below for details In `citus_prepare_upgrade()`, don't drop any_value when upgrading from PG16+, because PG16+ has its own any_value function. Attempting to do so results in the error seen in [pg16-pg17 upgrade](https://github.com/citusdata/citus/actions/runs/11768444117/job/32778340003?pr=7661): ``` ERROR: cannot drop function any_value(anyelement) because it is required by the database system CONTEXT: SQL statement "DROP AGGREGATE IF EXISTS pg_catalog.any_value(anyelement)" ``` When 16 becomes the minimum supported Postgres version, the drop statements can be removed. --------- Several PG17 Compatibility commits have been merged before this final one. All these subtasks are done #7653 See the list below: Compilation PR: #7699 Ruleutils PR: #7725 Sister PR for tests: citusdata/the-process#159 Helpful smaller PRs: - #7714 - #7726 - #7731 - #7732 - #7733 - #7738 - #7745 - #7747 - #7748 - #7749 - #7752 - #7755 - #7757 - #7759 - #7760 - #7761 - #7762 - #7765 - #7766 - #7768 - #7769 - #7771 - #7774 - #7776 - #7780 - #7781 - #7785 - #7788 - #7793 - #7796 --------- Co-authored-by: Colm <colmmchugh@microsoft.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
There is a crash when running vanilla tests because of the
citus.hide_citus_dependent_objectsGUC. We turn on this GUC only for the pg vanilla tests. This GUC runs the following functionHideCitusDependentObjectsOnQueriesOfPgMetaTables. This function doesn't take into account the newmergeJoinCondition. I rewrote the function such that it checks for merge join conditions as well.Relevant PG commit: postgres/postgres@0294df2f1
The crash can be reproduced locally like the following: