Skip to content

Add PG 18Beta1 compatibility (Build + RuleUtils) - #7981

Merged
Mehmet YILMAZ (m3hm3t) merged 29 commits into
mainfrom
m3hm3t/pg18_support
Jul 16, 2025
Merged

Add PG 18Beta1 compatibility (Build + RuleUtils)#7981
Mehmet YILMAZ (m3hm3t) merged 29 commits into
mainfrom
m3hm3t/pg18_support

Conversation

@m3hm3t

@m3hm3t Mehmet YILMAZ (m3hm3t) commented May 9, 2025

Copy link
Copy Markdown
Contributor

This PR provides successful build against PG18Beta1. RuleUtils PR was reviewed separately: #8010

PG 18Beta1–related changes for building Citus

TupleDesc / Attr layout

What changed in PG: Postgres consolidated the TupleDescData.attrs[] array into a more compact representation. Direct field access (tupdesc->attrs[i]) was replaced by the new TupleDescAttr() API.

Citus adaptation: Everywhere we previously used tupdesc->attrs[...], we now call TupleDescAttr(tupdesc, idx) (or our own Attr() macro) under a compatibility guard.

General Logic:

  • Use Attr(...) in places where columnar_version_compat.h is included. This avoids the need to sprinkle #if PG_VERSION_NUM guards around each attribute access.

  • Use TupleDescAttr(tupdesc, i) when the relevant PostgreSQL header is already included and the additional macro indirection is unnecessary.

Collation‐aware LIKE

What changed in PG: The textlike operator now requires an explicit collation, to avoid ambiguous‐collation errors. Core code switched from DirectFunctionCall2(textlike, ...) to DirectFunctionCall2Coll(textlike, DEFAULT_COLLATION_OID, ...).

Citus adaptation: In remote_commands.c and any other LIKE call, we now use DirectFunctionCall2Coll(textlike, DEFAULT_COLLATION_OID, ...) and #include <utils/pg_collation.h>.

Columnar storage API

  • Adapt columnar_relation_set_new_filelocator (and related init routines) for PG 18’s revised SMGR and storage-initialization hooks.
  • Pull in the new headers (explain_format.h, columnar_version_compat.h) so the columnar module compiles cleanly against PG 18.
  • heap_modify_tuple + heap_inplace_update only exist on PG < 18; on PG18 the in-place helper was removed upstream

  • postgres/postgres@a07e03f

OpenSSL / TLS integration

What changed in PG: Moved from the legacy SSL_library_init() to OPENSSL_init_ssl(OPENSSL_INIT_LOAD_CONFIG, NULL), updated certificate API calls (X509_getm_notBefore, X509_getm_notAfter), and standardized on TLS_method().

Citus adaptation: We now #include <openssl/opensslv.h> and use #if OPENSSL_VERSION_NUMBER >= 0x10100000L to choose between OPENSSL_init_ssl() or SSL_library_init(), and wrap X509_gmtime_adj() calls around the new accessor functions.

Adapt ExtractColumns() to the new PG-18 expandRTE() signature

PostgreSQL 18 postgres/postgres@80feb72 added a fourth argument of type VarReturningType to expandRTE(), so calls that used the old 7-parameter form no longer compile. This patch:

  • Wraps the expandRTE(...) call in a #if PG_VERSION_NUM >= 180000 guard.
  • On PG 18+ passes the new VAR_RETURNING_DEFAULT argument before location.
  • On PG 15–17 continues to call the original 7-arg form.
  • Adds the necessary includes (parser/parse_relation.h for expandRTE and VarReturningType, and pg_version_constants.h for PG_VERSION_NUM).

Adapt ExecutorStart/ExecutorRun hooks to PG-18’s new signatures

PostgreSQL 18 postgres/postgres@525392d changed the signatures of the executor hooks:

  • ExecutorStart_hook now returns bool instead of void, and
  • ExecutorRun_hook drops its old run_once argument.

This patch preserves Citus’s existing hook logic by:

  1. Adding two adapter functions under #if PG_VERSION_NUM >= PG_VERSION_18:

    • citus_executor_start_adapter(QueryDesc *queryDesc, int eflags)
      Calls the old CitusExecutorStart(queryDesc, eflags) and then returns true to satisfy the new hook’s bool return type.
    • citus_executor_run_adapter(QueryDesc *queryDesc, ScanDirection direction, uint64 count)
      Calls the old CitusExecutorRun(queryDesc, direction, count, true) (passing true for the dropped run_once argument), and returns void.
  2. Installing the adapters in _PG_init() instead of the original hooks when building against PG 18+:

    #if PG_VERSION_NUM >= PG_VERSION_18
        ExecutorStart_hook = citus_executor_start_adapter;
        ExecutorRun_hook   = citus_executor_run_adapter;
    #else
        ExecutorStart_hook = CitusExecutorStart;
        ExecutorRun_hook   = CitusExecutorRun;
    #endif

Adapt to PG-18’s removal of the “run_once” flag from ExecutorRun/PortalRun

PostgreSQL commit [3eea7a0](postgres/postgres@3eea7a0) rationalized the executor’s parallelism logic by moving the “execute a plan only once” check into ExecutePlan() itself and dropping the old bool run_once argument from the public APIs:

- void ExecutorRun(QueryDesc *queryDesc,
-                  ScanDirection direction,
-                  uint64 count,
-                  bool run_once);
+ void ExecutorRun(QueryDesc *queryDesc,
+                  ScanDirection direction,
+                  uint64 count);

(and similarly for PortalRun()).

To stay compatible across PG 15–18, Citus now:

  1. Updates all internal calls to ExecutorRun(...) and PortalRun(...):

    • On PG 18+, use the new three-argument form (ExecutorRun(qd, dir, count)).
    • On PG 15–17, keep the old four-arg form (ExecutorRun(qd, dir, count, true)) under a #if PG_VERSION_NUM < 180000 guard.
  2. Guards the dispatcher hooks via the adapter functions (from the earlier patch) so that Citus’s executor hooks continue to work under both the old and new signatures.

Adapt to PG-18’s shortened PortalRun signature

PostgreSQL 18’s refactoring (see commit 3eea7a0) also removed the old run_once and alternate‐dest arguments from the public PortalRun() API. The signature changed from:

- bool PortalRun(Portal portal,
-                long count,
-                bool isTopLevel,
-                bool run_once,
-                DestReceiver *dest,
-                DestReceiver *altdest,
-                QueryCompletion *qc);
+ bool PortalRun(Portal portal,
+                long count,
+                bool isTopLevel,
+                DestReceiver *dest,
+                DestReceiver *altdest,
+                QueryCompletion *qc);

To support both versions in Citus, we:

  1. Version-guard each call to PortalRun():

    • On PG 18+ invoke the new 6-argument form.
    • On PG 15–17 fall back to the legacy 7-argument form, passing true for run_once.

Add support for PG-18’s new plansource argument in PortalDefineQuery**

PostgreSQL 18 extended the PortalDefineQuery API to carry a CachedPlanSource *plansource pointer so that the portal machinery can track cached‐plan invalidation (as introduced alongside deferred-locking in commit postgres/postgres@525392d. To remain compatible across PG 15–18, Citus now wraps its calls under a version guard:

-   PortalDefineQuery(portal, NULL, sql, commandTag, plantree_list, NULL);
+#if PG_VERSION_NUM >= 180000
+   /* PG 18+: seven-arg signature (adds plansource) */
+   PortalDefineQuery(
+       portal,
+       NULL,            /* no prepared-stmt name */
+       sql,             /* the query text */
+       commandTag,      /* the CommandTag */
+       plantree_list,   /* List of PlannedStmt* */
+       NULL,            /* no CachedPlan */
+       NULL             /* no CachedPlanSource */
+   );
+#else
+   /* PG 15–17: six-arg signature */
+   PortalDefineQuery(
+       portal,
+       NULL,            /* no prepared-stmt name */
+       sql,             /* the query text */
+       commandTag,      /* the CommandTag */
+       plantree_list,   /* List of PlannedStmt* */
+       NULL             /* no CachedPlan */
+   );
+#endif

Adapt ExecInitRangeTable() calls to PG-18’s new signature

PostgreSQL commit cbc127917e04a978a788b8bc9d35a70244396d5b overhauled the planner API for range‐table initialization:

PG 18+: added a fourth Bitmapset *unpruned_relids argument to support deferred partition pruning

In Citus’s create_estate_for_relation() (in columnar_metadata.c), we now wrap the call in a compile‐time guard so that the code compiles correctly on all supported PostgreSQL versions:

/* Prepare permission info on PG 16+ */
#if PG_VERSION_NUM >= PG_VERSION_16
    List *perminfos = NIL;
    addRTEPermissionInfo(&perminfos, rte);
#else
    List *perminfos = NIL;  /* unused on PG 15 */
#endif

/* Initialize the range table, with the right signature for each PG version */
#if PG_VERSION_NUM >= PG_VERSION_18
    /* PG 18+: four‐arg signature (adds unpruned_relids) */
    ExecInitRangeTable(
        estate,
        list_make1(rte),
        perminfos,
        NULL        /* unpruned_relids: not used by columnar */
    );
#elif PG_VERSION_NUM >= PG_VERSION_16
    /* PG 16–17: three‐arg signature (permInfos) */
    ExecInitRangeTable(
        estate,
        list_make1(rte),
        perminfos
    );
#else
    /* PG 15: two‐arg signature */
    ExecInitRangeTable(
        estate,
        list_make1(rte)
    );
#endif

estate->es_output_cid = GetCurrentCommandId(true);

Adapt pgstat_report_vacuum() to PG-18’s new timestamp argument

PostgreSQL commit [30a6ed0ce4bb18212ec38cdb537ea4b43bc99b83](postgres/postgres@30a6ed0) extended the pgstat_report_vacuum() API by adding a TimestampTz start_time parameter at the end so that the VACUUM statistics collector can record when the operation began:

/* PG ≤17: four-arg signature */
- void pgstat_report_vacuum(Oid tableoid,
-                           bool shared,
-                           double num_live_tuples,
-                           double num_dead_tuples);
+/* PG ≥18: five-arg signature adds a start_time */
+ void pgstat_report_vacuum(Oid tableoid,
+                           bool shared,
+                           double num_live_tuples,
+                           double num_dead_tuples,
+                           TimestampTz start_time);

To support both versions, we now wrap the call in columnar_tableam.c with a version guard, supplying GetCurrentTimestamp() for PG-18+:

#if PG_VERSION_NUM >= 180000
    /* PG 18+: include start_timestamp */
    pgstat_report_vacuum(
        RelationGetRelid(rel),
        rel->rd_rel->relisshared,
        Max(new_live_tuples, 0),  /* live tuples */
        0,                        /* dead tuples */
        GetCurrentTimestamp()     /* start time */
    );
#else
    /* PG 15–17: original signature */
    pgstat_report_vacuum(
        RelationGetRelid(rel),
        rel->rd_rel->relisshared,
        Max(new_live_tuples, 0),  /* live tuples */
        0                         /* dead tuples */
    );
#endif

Adapt ExecuteTaskPlan() to PG-18’s expanded CreateQueryDesc() signature

PostgreSQL 18 changed CreateQueryDesc() from an eight-argument to a nine-argument call by inserting a CachedPlan *cplan parameter immediately after the PlannedStmt *plannedstmt argument (see commit postgres/postgres@525392d). To remain compatible with PG 15–17, Citus now wraps its invocation in local_executor.c with a version guard:

-    /* PG15–17: eight-arg CreateQueryDesc without cached plan */
-    QueryDesc *queryDesc = CreateQueryDesc(
-        taskPlan,           /* PlannedStmt *plannedstmt */
-        queryString,        /* const char *sourceText */
-        GetActiveSnapshot(),/* Snapshot snapshot */
-        InvalidSnapshot,    /* Snapshot crosscheck_snapshot */
-        destReceiver,       /* DestReceiver *dest */
-        paramListInfo,      /* ParamListInfo params */
-        queryEnv,           /* QueryEnvironment *queryEnv */
-        0                   /* int instrument_options */
-    );
+#if PG_VERSION_NUM >= 180000
+    /* PG18+: nine-arg CreateQueryDesc with a CachedPlan slot */
+    QueryDesc *queryDesc = CreateQueryDesc(
+        taskPlan,           /* PlannedStmt *plannedstmt */
+        NULL,               /* CachedPlan *cplan (none) */
+        queryString,        /* const char *sourceText */
+        GetActiveSnapshot(),/* Snapshot snapshot */
+        InvalidSnapshot,    /* Snapshot crosscheck_snapshot */
+        destReceiver,       /* DestReceiver *dest */
+        paramListInfo,      /* ParamListInfo params */
+        queryEnv,           /* QueryEnvironment *queryEnv */
+        0                   /* int instrument_options */
+    );
+#else
+    /* PG15–17: eight-arg CreateQueryDesc without cached plan */
+    QueryDesc *queryDesc = CreateQueryDesc(
+        taskPlan,           /* PlannedStmt *plannedstmt */
+        queryString,        /* const char *sourceText */
+        GetActiveSnapshot(),/* Snapshot snapshot */
+        InvalidSnapshot,    /* Snapshot crosscheck_snapshot */
+        destReceiver,       /* DestReceiver *dest */
+        paramListInfo,      /* ParamListInfo params */
+        queryEnv,           /* QueryEnvironment *queryEnv */
+        0                   /* int instrument_options */
+    );
+#endif

Adapt RelationGetPrimaryKeyIndex() to PG-18’s new “deferrable_ok” flag

PostgreSQL commit postgres/postgres@14e87ff added a new Boolean deferrable_ok parameter to RelationGetPrimaryKeyIndex() so that the lock manager can defer unique‐constraint locks when requested. The API changed from:

RelationGetPrimaryKeyIndex(Relation relation)

to:

RelationGetPrimaryKeyIndex(Relation relation, bool deferrable_ok)
diff --git a/src/backend/distributed/metadata/node_metadata.c b/src/backend/distributed/metadata/node_metadata.c
index e3a1b2c..f4d5e6f 100644
--- a/src/backend/distributed/metadata/node_metadata.c
+++ b/src/backend/distributed/metadata/node_metadata.c
@@ -2965,8 +2965,18 @@
     */
-    Relation replicaIndex = index_open(RelationGetPrimaryKeyIndex(pgDistNode),
-                                      AccessShareLock);
+    #if PG_VERSION_NUM >= PG_VERSION_18
+        /* PG 18+ adds a bool "deferrable_ok" parameter */
+        Relation replicaIndex =
+            index_open(
+                RelationGetPrimaryKeyIndex(pgDistNode, false),
+                AccessShareLock);
+    #else
+        Relation replicaIndex =
+            index_open(
+                RelationGetPrimaryKeyIndex(pgDistNode),
+                AccessShareLock);
+    #endif

     ScanKeyInit(&scanKey[0], Anum_pg_dist_node_nodename,
                  BTEqualStrategyNumber, F_TEXTEQ, CStringGetTextDatum(nodeName));
diff --git a/src/backend/distributed/operations/node_protocol.c b/src/backend/distributed/operations/node_protocol.c
index e3a1b2c..f4d5e6f 100644
--- a/src/backend/distributed/operations/node_protocol.c
+++ b/src/backend/distributed/operations/node_protocol.c
@@ -746,7 +746,12 @@
   if (!OidIsValid(idxoid))
   {
-        idxoid = RelationGetPrimaryKeyIndex(rel);
+        /* Determine the index OID of the primary key (PG18 adds a second parameter) */
+#if PG_VERSION_NUM >= PG_VERSION_18
+        idxoid = RelationGetPrimaryKeyIndex(rel, false);
+#else
+        idxoid = RelationGetPrimaryKeyIndex(rel);
+#endif
   }

   return idxoid;

Because Citus has always taken the lock immediately—just as the old two-arg call did—we pass false to keep that same immediate-lock behavior. Passing true would switch to deferred locking, which we don’t want.

Adapt ExplainOnePlan() to PG-18’s expanded API

PostgreSQL 18 extended postgres/postgres@525392d the ExplainOnePlan() function to carry the CachedPlan * and CachedPlanSource * pointers plus an explicit query_index, letting the EXPLAIN machinery track plan‐source invalidation. The old signature:

/* PG ≤17 */
void
ExplainOnePlan(PlannedStmt *plannedstmt,
               IntoClause *into,
               struct ExplainState *es,
               const char *queryString,
               ParamListInfo params,
               QueryEnvironment *queryEnv,
               const instr_time *planduration,
               const BufferUsage *bufusage);

became, in PG 18:

/* PG ≥18 */
void
ExplainOnePlan(PlannedStmt *plannedstmt,
               CachedPlan   *cplan,
               CachedPlanSource *plansource,
               int            query_index,
               IntoClause    *into,
               struct ExplainState *es,
               const char   *queryString,
               ParamListInfo params,
               QueryEnvironment *queryEnv,
               const instr_time *planduration,
               const BufferUsage *bufusage,
               const MemoryContextCounters *mem_counters);

To compile under both versions, Citus now wraps each call in multi_explain.c with:

#if PG_VERSION_NUM >= PG_VERSION_18
    /* PG 18+: pass NULL for the new cached‐plan fields and zero for query_index */
    ExplainOnePlan(
        plan,         /* PlannedStmt *plannedstmt */
        NULL,         /* CachedPlan *cplan */
        NULL,         /* CachedPlanSource *plansource */
        0,            /* query_index */
        into,         /* IntoClause *into */
        es,           /* ExplainState *es */
        queryString,  /* const char *queryString */
        params,       /* ParamListInfo params */
        NULL,         /* QueryEnvironment *queryEnv */
        &planduration,/* const instr_time *planduration */
        (es->buffers ? &bufusage : NULL),
        (es->memory  ? &mem_counters : NULL)
    );
#elif PG_VERSION_NUM >= PG_VERSION_17
    /* PG 17: same as before, plus passing mem_counters if enabled */
    ExplainOnePlan(
        plan,
        into,
        es,
        queryString,
        params,
        queryEnv,
        &planduration,
        (es->buffers ? &bufusage : NULL),
        (es->memory ? &mem_counters : NULL)
    );
#else
    /* PG 15–16: original seven-arg form */
    ExplainOnePlan(
        plan,
        into,
        es,
        queryString,
        params,
        queryEnv,
        &planduration,
        (es->buffers ? &bufusage : NULL)
    );
#endif

Adapt to the unified “index interpretation” API in PG 18 (commit a8025f544854)

PostgreSQL commit postgres/postgres@a8025f5 generalized the old btree‐specific operator‐interpretation API into a single “index interpretation” interface:

  • Renamed type:
    OpBtreeInterpretationOpIndexInterpretation
  • Renamed function:
    get_op_btree_interpretation(opno)get_op_index_interpretation(opno)
  • Unified field:
    Each interpretation now carries cmptype instead of strategy.

To build cleanly on PG 18 while still supporting PG 15–17, Citus’s shard‐pruning code now wraps these changes:

#include "pg_version_constants.h"

#if PG_VERSION_NUM >= PG_VERSION_18
/* On PG 18+ the btree‐only APIs vanished; alias them to the new generic versions */
typedef OpIndexInterpretation OpBtreeInterpretation;
#define get_op_btree_interpretation(opno)  get_op_index_interpretation(opno)
#define ROWCOMPARE_NE  COMPARE_NE
#endif

/* … later, when checking an interpretation … */
OpBtreeInterpretation *interp =
    (OpBtreeInterpretation *) lfirst(cell);

#if PG_VERSION_NUM >= PG_VERSION_18
    /* use cmptype on PG 18+ */
    if (interp->cmptype == ROWCOMPARE_NE)
#else
    /* use strategy on PG 15–17 */
    if (interp->strategy == ROWCOMPARE_NE)
#endif
{
    /* … */
}

Adapt create_foreignscan_path() for PG-18’s revised signature

PostgreSQL commit postgres/postgres@e222534 reordered and removed a couple of parameters in the FDW‐path builder:

  • PG 15–17 signature (11 args)

    create_foreignscan_path(PlannerInfo   *root,
                            RelOptInfo    *rel,
                            PathTarget    *target,
                            double         rows,
                            Cost           startup_cost,
                            Cost           total_cost,
                            List          *pathkeys,
                            Relids         required_outer,
                            Path          *fdw_outerpath,
                            List          *fdw_restrictinfo,
                            List          *fdw_private);
  • PG 18+ signature (9 args)

    create_foreignscan_path(PlannerInfo   *root,
                            RelOptInfo    *rel,
                            PathTarget    *target,
                            double         rows,
                            int            disabled_nodes,
                            Cost           startup_cost,
                            Cost           total_cost,
                            Relids         required_outer,
                            Path          *fdw_outerpath,
                            List          *fdw_private);

To support both, Citus now defines a compatibility macro in pg_version_compat.h:

#include "nodes/bitmapset.h"   /* for Relids */
#include "nodes/pg_list.h"     /* for List */
#include "optimizer/pathnode.h" /* for create_foreignscan_path() */

#if PG_VERSION_NUM >= PG_VERSION_18

/* PG18+: drop pathkeys & fdw_restrictinfo, add disabled_nodes */
#define create_foreignscan_path_compat(a, b, c, d, e, f, g, h, i, j, k) \
    create_foreignscan_path(                                            \
        (a),          /* root */                                       \
        (b),          /* rel */                                        \
        (c),          /* target */                                     \
        (d),          /* rows */                                       \
        (0),          /* disabled_nodes (unused by Citus) */           \
        (e),          /* startup_cost */                              \
        (f),          /* total_cost */                                \
        (g),          /* required_outer */                            \
        (h),          /* fdw_outerpath */                             \
        (k)           /* fdw_private */                               \
    )

#else

/* PG15–17: original signature */
#define create_foreignscan_path_compat(a, b, c, d, e, f, g, h, i, j, k) \
    create_foreignscan_path(                                            \
        (a), (b), (c), (d),                                            \
        (e), (f),                                                      \
        (g), (h), (i), (j), (k)                                        \
    )
#endif

Now every call to create_foreignscan_path_compat(...)—even in tests like fake_fdw.c—automatically picks the correct argument list for PG 15 through PG 18.

Drop the obsolete bitmap‐scan hooks on PG 18+

PostgreSQL commit postgres/postgres@c395322 cleaned up the TableAmRoutine API by removing the two bitmap‐scan callback slots:

  • scan_bitmap_next_block
  • scan_bitmap_next_tuple

Since those hook‐slots no longer exist in PG 18, Citus now wraps their NULL‐initialization in a #if PG_VERSION_NUM < PG_VERSION_18 guard. On PG 15–17 we still explicitly set them to NULL (to satisfy the old struct layout), and on PG 18+ we omit them entirely:

#if PG_VERSION_NUM < PG_VERSION_18
    /* PG 15–17 only: these fields were removed upstream in PG 18 */
    .scan_bitmap_next_block = NULL,
    .scan_bitmap_next_tuple = NULL,
#endif

Adapt vac_update_relstats() invocation to PG-18’s new “all_frozen” argument

PostgreSQL commit postgres/postgres@99f8f3f extended the vac_update_relstats() API by inserting a num_all_frozen_pages parameter between the existing num_all_visible_pages and hasindex arguments:

- /* PG ≤17: */
- void
- vac_update_relstats(Relation relation,
-                    BlockNumber num_pages,
-                    double     num_tuples,
-                    BlockNumber num_all_visible_pages,
-                    bool       hasindex,
-                    TransactionId frozenxid,
-                    MultiXactId  minmulti,
-                    bool      *frozenxid_updated,
-                    bool      *minmulti_updated,
-                    bool       in_outer_xact);
+ /* PG ≥18: adds num_all_frozen_pages */
+ void
+ vac_update_relstats(Relation    relation,
+                    BlockNumber num_pages,
+                    double      num_tuples,
+                    BlockNumber num_all_visible_pages,
+                    BlockNumber num_all_frozen_pages,
+                    bool        hasindex,
+                    TransactionId frozenxid,
+                    MultiXactId  minmulti,
+                    bool      *frozenxid_updated,
+                    bool      *minmulti_updated,
+                    bool       in_outer_xact);

To compile cleanly on both PG 15–17 and PG 18+, Citus wraps its call in a version guard and supplies a zero placeholder for the new field:

#if PG_VERSION_NUM >= 180000
    /* PG 18+: supply explicit “all_frozen” count */
    vac_update_relstats(
        rel,
        new_rel_pages,
        new_live_tuples,
        new_rel_allvisible,    /* allvisible */
        0,                     /* all_frozen */
        nindexes > 0,
        newRelFrozenXid,
        newRelminMxid,
        &frozenxid_updated,
        &minmulti_updated,
        false                  /* in_outer_xact */
    );
#else
    /* PG 15–17: original signature */
    vac_update_relstats(
        rel,
        new_rel_pages,
        new_live_tuples,
        new_rel_allvisible,
        nindexes > 0,
        newRelFrozenXid,
        newRelminMxid,
        &frozenxid_updated,
        &minmulti_updated,
        false                  /* in_outer_xact */
    );
#endif

Why all_frozen = 0?
Columnar storage never embeds transaction IDs in its pages, so it never needs to track “all‐frozen” pages the way a heap does. Setting both allvisible and allfrozen to zero simply tells Postgres “there are no pages with the visibility or frozen‐status bits set,” matching our existing behavior.

This change ensures Citus’s VACUUM‐statistic updates work unmodified across all supported Postgres versions.

@m3hm3t Mehmet YILMAZ (m3hm3t) self-assigned this May 9, 2025
@codecov

codecov Bot commented May 16, 2025

Copy link
Copy Markdown

Codecov Report

Attention: Patch coverage is 89.83051% with 6 lines in your changes missing coverage. Please review.

Project coverage is 89.19%. Comparing base (5deaf9a) to head (abd2c38).
Report is 1 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #7981      +/-   ##
==========================================
- Coverage   89.19%   89.19%   -0.01%     
==========================================
  Files         284      284              
  Lines       61515    61523       +8     
  Branches     7696     7696              
==========================================
+ Hits        54871    54878       +7     
- Misses       4440     4443       +3     
+ Partials     2204     2202       -2     
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@m3hm3t Mehmet YILMAZ (m3hm3t) changed the title M3hm3t/pg18 support PG18 Support Jun 12, 2025
@m3hm3t
Mehmet YILMAZ (m3hm3t) marked this pull request as ready for review June 20, 2025 09:47
@m3hm3t Mehmet YILMAZ (m3hm3t) changed the title PG18 Support Add PostgreSQL 18 compatibility across Citus codebase Jun 20, 2025
@m3hm3t Mehmet YILMAZ (m3hm3t) changed the title Add PostgreSQL 18 compatibility across Citus codebase Add PostgreSQL 18 compatibility across Citus codebase ( Build + RuleUtils ) Jul 1, 2025
@m3hm3t
Mehmet YILMAZ (m3hm3t) force-pushed the m3hm3t/pg18_support branch 5 times, most recently from d4c66aa to 172782a Compare July 1, 2025 12:55
@m3hm3t
Mehmet YILMAZ (m3hm3t) marked this pull request as draft July 1, 2025 12:56
@m3hm3t
Mehmet YILMAZ (m3hm3t) force-pushed the m3hm3t/pg18_support branch 6 times, most recently from b95047d to 6129856 Compare July 1, 2025 13:21
@m3hm3t

Mehmet YILMAZ (m3hm3t) commented Jul 2, 2025

Copy link
Copy Markdown
Contributor Author

moved to PR desc

1 similar comment
@m3hm3t

Mehmet YILMAZ (m3hm3t) commented Jul 2, 2025

Copy link
Copy Markdown
Contributor Author

moved to PR desc

Add support for PostgreSQL version 18 in build and test workflow

pipeline isn’t picking up the 18 line because your regex for the full field only allows digits and dots ([0-9.]+), so it skips the "full": "18beta1" entry.

Fix image suffix formatting in build parameters
@m3hm3t

Mehmet YILMAZ (m3hm3t) commented Jul 2, 2025

Copy link
Copy Markdown
Contributor Author

moved to PR desc

2 similar comments
@m3hm3t

Mehmet YILMAZ (m3hm3t) commented Jul 2, 2025

Copy link
Copy Markdown
Contributor Author

moved to PR desc

@m3hm3t

Mehmet YILMAZ (m3hm3t) commented Jul 2, 2025

Copy link
Copy Markdown
Contributor Author

moved to PR desc

@m3hm3t
Mehmet YILMAZ (m3hm3t) marked this pull request as ready for review July 2, 2025 11:59

@naisila Naisila Puka (naisila) left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The 16 commits in this PR do not correspond to relevant PG18 commit changes, but rather commits fixing your development work (refactoring etc). The main commit that contains a bunch of PG18 related changes all in one place is pg18 compile work cont. Although the changes are minor, it is difficult to review them when they are all in one place, and the comments are scattered throughout the PR description.

I think it is more readable to follow the practice of previous years, and keep the changes separated in commits, while linking the relevant PG18 commit for each change. PG18 comes with a lot of minor changes, and to maintain them we need a tracking system. Links below:

I would suggest you to rearrange your PR similarly, after addressing the review as well. If you have a different idea from the previous years, I'm definitely open to that, as long as it is systematic and readable.

Comment thread .github/workflows/packaging-test-pipelines.yml
Comment thread citus-tools Outdated
Comment thread src/include/columnar/columnar_version_compat.h Outdated
Comment thread src/backend/columnar/columnar_tableam.c
Comment thread src/backend/columnar/columnar_tableam.c Outdated
#if PG_VERSION_NUM >= PG_VERSION_18
vac_update_relstats(rel, new_rel_pages, new_live_tuples,
new_rel_allvisible, /* allvisible */
0, /* all_frozen */

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why all_frozen is 0?
I think it would be more consistent to define it similar to BlockNumber new_rel_allvisible = 0;, together with the explanation why all_frozen is 0.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I hard-code all_frozen = 0 for columnar storage doesn’t maintain a visibility map or “all-frozen” state the way heap tables do.

I agree on the refactor you suggested. If you are okay on all_frozen = 0, I will update the code.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Makes sense, you can update the code

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated

Comment thread src/backend/distributed/planner/shard_pruning.c Outdated
Comment thread src/backend/distributed/planner/shard_pruning.c Outdated
Comment thread src/backend/distributed/shared_library_init.c Outdated
Comment thread src/backend/distributed/utils/aggregate_utils.c
Comment thread src/include/pg_version_compat.h Outdated
Implement code changes to enhance functionality and improve performance

Revert unwanted edits to configure scripts
@m3hm3t

Mehmet YILMAZ (m3hm3t) commented Jul 9, 2025

Copy link
Copy Markdown
Contributor Author

moved to PR desc

@naisila Naisila Puka (naisila) left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

At this point rearranging into separate commits would take some time.
One thing you could do is: gather every change, along with its relevant PG commit, in the PR description. It will be a long PR description, but it is readable and I believe we can understand the reason for every change in the PR.

Comment thread .github/workflows/packaging-test-pipelines.yml
Comment thread src/backend/columnar/columnar_metadata.c
Comment thread src/backend/columnar/columnar_tableam.c Outdated
#if PG_VERSION_NUM >= PG_VERSION_18
vac_update_relstats(rel, new_rel_pages, new_live_tuples,
new_rel_allvisible, /* allvisible */
0, /* all_frozen */

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Makes sense, you can update the code

Comment thread src/backend/columnar/columnar_tableam.c
Comment thread src/backend/distributed/commands/multi_copy.c
* If new object classes are added and none of them are node-wide, then update
* this assertion check based on latest supported major Postgres version.
*/
StaticAssertStmt(PG_MAJORVERSION_NUM <= 17,

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually ObjectClass was removed in PG17, so we need to check for new catalog OIDs. postgres/postgres@89e5ef7

* If new object classes are added and none of them are node-wide, then update
* this assertion check based on latest supported major Postgres version.
*/
StaticAssertStmt(PG_MAJORVERSION_NUM <= 17,

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@naisila Naisila Puka (naisila) left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reminder to revert changes in Configure script and .yml files, and anywhere else that 18beta1 version is included.

@naisila Naisila Puka (naisila) changed the title Add PostgreSQL 18 compatibility across Citus codebase ( Build + RuleUtils ) Add PG 18Beta1 compatibility (Build + RuleUtils) Jul 16, 2025
Comment thread citus-tools Outdated
@m3hm3t
Mehmet YILMAZ (m3hm3t) merged commit 9e42f3f into main Jul 16, 2025
119 checks passed
@m3hm3t
Mehmet YILMAZ (m3hm3t) deleted the m3hm3t/pg18_support branch July 16, 2025 12:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants