Skip to content

Commit

Permalink
mirrorless: Enable WAL optimization for COPY FROM
Browse files Browse the repository at this point in the history
CopyFrom has an optimization where WAL can be avoided if the COPY is in
the same transaction as the CREATE and the data is being written to the
same relfilenode created in this transaction.

Unfortunately, this optimization was ifdefed out, due to legacy
assumptions about our inability to support wal_level = minimal.

Here is an example in a mirrorless demo cluster with wal_level =
minimal:

BEGIN;
CREATE TABLE foo(i int) DISTRIBUTED REPLICATED;
COPY foo FROM PROGRAM 'seq 1 3';
COMMIT;

Without patch, WAL for this table:
rmgr: Storage     len (rec/tot):     46/    46, tx:          0, lsn: 0/0C010D30, prev 0/0C010D10, desc: CREATE base/13720/32768; smgr: heap
rmgr: Heap2       len (rec/tot):     86/    86, tx:        540, lsn: 0/0C023C20, prev 0/0C023B88, desc: MULTI_INSERT+INIT 3 tuples flags 0x02, blkref #0: rel 1663/13720/32768 blk 0

With patch, WAL for this table (MULTI_INSERT record not emitted):
rmgr: Storage     len (rec/tot):     46/    46, tx:          0, lsn: 0/0C004908, prev 0/0C0048E8, desc: CREATE base/13720/24576; smgr: heap

PS: AO/CO tables avoid writing WAL for all inserts in a more general
way, and this change doesn't affect them.
  • Loading branch information
soumyadeep2007 committed May 17, 2024
1 parent 603a381 commit 3abcf7c
Showing 1 changed file with 0 additions and 7 deletions.
7 changes: 0 additions & 7 deletions src/backend/commands/copy.c
Original file line number Diff line number Diff line change
Expand Up @@ -3970,15 +3970,8 @@ CopyFrom(CopyState cstate)
cstate->rel->rd_newRelfilenodeSubid != InvalidSubTransactionId))
{
ti_options |= TABLE_INSERT_SKIP_FSM;
/*
* The optimization to skip WAL has been disabled in GPDB. wal_level
* is hardcoded to 'archive' in GPDB, so it wouldn't have any effect
* anyway.
*/
#if 0
if (!XLogIsNeeded())
ti_options |= TABLE_INSERT_SKIP_WAL;
#endif
}

/*
Expand Down

0 comments on commit 3abcf7c

Please sign in to comment.