PG19: evaluate async I/O and pg_stat_io adoption for columnar

[ihalatci]

Part of #8597 (Phase 3 — later feature integration).

Covers two of the seven Phase 3 areas, because they are the same surface: columnar's buffer-read path. pg_stat_io only becomes meaningful for Citus if columnar adopts read streams.

Async I/O — Citus currently gains nothing

PG19 makes async I/O the default: DEFAULT_IO_METHOD is IOMETHOD_WORKER, and READ_STREAM_USE_BATCHING is available. AIO benefits are delivered through the read stream API — there is no separate extension-facing AIO API to call.

Columnar does not use it on the path that matters:

• src/backend/columnar/columnar_storage.c:673 — the genuine data-path read, via plain ReadBuffer(), inside ReadFromBlock(). Correction (verified 2026-08-15): this issue originally cited :447,673,701 as read sites. That was wrong. :447 is inside ColumnarStorageReserveData() and is ReadBuffer(rel, P_NEW) — a relation extend; :701 is inside WriteToBlock(). Both are write path. Only :673 is a read, so a read-stream conversion is one third the scope originally implied here.
• src/backend/columnar/columnar_tableam.c:1470 — the only read_stream_next_buffer() call in the tree, and it is consumer-side only; columnar never calls read_stream_begin_relation().

So columnar reads stay synchronous under PG19 defaults. This is not a correctness problem — nothing breaks, nothing is silently wrong. It is a throughput opportunity that heap access methods get for free and columnar does not.

Work would be: convert columnar's stripe/chunk read path to read_stream_begin_relation() + read_stream_next_buffer(), with READ_STREAM_USE_BATCHING where the access pattern is sequential. Needs measurement before commitment — columnar's access pattern may not benefit enough to justify the churn.

pg_stat_io — no surface today

PG19 extends pg_stat_io with read_bytes / write_bytes / extend_bytes, adds IOOBJECT_WAL, and adds pg_stat_get_backend_io(). pgstat_count_io_op*() gained a bytes argument.

Citus has zero references to any of this in src/ and in src/test/ (verified with a recursive scan; a non-recursive glob gives a false negative here). Nothing to update and nothing to fix — columnar's I/O is invisible to pg_stat_io because it goes through the buffer manager as ordinary reads.

If and only if the read-stream conversion above happens, revisit whether columnar should report I/O with a distinguishable IOObject / IOContext so operators can attribute columnar I/O separately.

Priority

Lowest of the Phase 3 items. Pure optimisation, no correctness dependency, does not gate the pg19-support → main merge. Should be measured before it is scheduled.

[ihalatci]

Correction to this issue body (author self-correction, post-measurement)

I opened this issue with two premises. Measurement disproved one of them, and found two of my three code citations were wrong. Recording that here rather than editing the body, so the error stays visible.

1. The pg_stat_io premise is FALSE. Columnar I/O is already fully visible.

I claimed columnar reads are not attributable in pg_stat_io. They are. A cold full-column scan of a 1972 MB columnar table reported:

reads = 252877   read_bytes = 2071568384   (= 1976 MB)

cross-validated against an independent cold EXPLAIN to 0.02%. The probe was also shown capable of printing zero (warm re-scan: reads=0, both AMs), so the non-zero reading is meaningful rather than an artifact of a probe that always prints something.

A finer check: sum(val) over one 8-byte column of 3M rows attributed 5,447,680 bytes, i.e. ~4.4x zstd compression on a monotonic sequence. The counter tracks only the column the query needed - column pruning is visible in pg_stat_io. It is not merely counting "something."

What is genuinely not possible is distinguishing columnar from heap inside pg_stat_io: the view is keyed only on backend_type, object, context (system_views.sql:1261), with no relation or AM dimension. That would require a core PostgreSQL change, not a Citus one.

This half should be closed as already satisfied.

2. Two of the three lines I cited are on the write path

I cited columnar_storage.c :447, :673, :701 as read sites. Only :673 (ReadFromBlock) is a read. :447 is extend and :701 is write. The force parameter on :673 matters and I did not mention it.

3. AIO is reached, not absent - it is starved

I implied columnar bypasses async I/O. It does not. ReadBuffer() reaches AsyncReadBuffers() unconditionally for a relation read:

ReadBuffer bufmgr.c:879 -> ReadBuffer_common :1276 -> StartReadBuffer :1637
  -> StartReadBuffersImpl :1371 -> AsyncReadBuffers :1938 (called :1557)

There is no non-AIO path. The real problem is that columnar issues strictly single-block reads, so there is nothing for AIO to combine or overlap. Starved, not absent. That also makes the two findings sequential rather than independent: I/O combining is a prerequisite for AIO to pay off, and read_stream_begin_relation() takes the BufferAccessStrategy in the same constructor.

4. Recommended disposition

• pg_stat_io half: close, already satisfied (premise false).
• AIO conversion: defer. Measured 22.0% I/O vs 78% CPU on the target workload, giving a ceiling of ~1.28x, on a rig that demonstrably detects AIO at 1.60-1.74x on heap.
• The finding actually worth acting on is neither, and will be filed separately: columnar passes no BufferAccessStrategy, so a single columnar scan evicts ~99.99% of a co-resident hot set while a heap scan reading 2.4x more bytes evicts 0%. That is a 4+ year old defect, unrelated to PG19.

[ihalatci]

Status: no local implementation/PR; AIO adoption is deferred after measurement showed a limited ~1.28x ceiling on the sampled columnar workload. pg_stat_io already observes columnar reads but cannot distinguish access methods. Needs: decide whether to close/defer this feature item; any future work requires a benchmark-backed design. The separate BufferAccessStrategy cache-pollution defect should be tracked independently, not implemented here.