Skip to content

Add --exclude-tmpdir option, which makes exclude pgsql_tmp and snapshots dirs #271

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 1 commit into from
Closed

Add --exclude-tmpdir option, which makes exclude pgsql_tmp and snapshots dirs #271

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions dir.c
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,8 @@ const char *pgdata_exclude[] =
NULL, /* arclog_path will be set later */
NULL, /* srvlog_path will be set later */
NULL, /* 'pg_tblspc' will be set later */
NULL, /* 'pg_logical' will be set later */
NULL, /* 'pgsql_tmp' will be set later */
NULL, /* sentinel */
};

Expand Down
16 changes: 15 additions & 1 deletion docs/index-ja.html
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -660,6 +660,12 @@ <h2>バックアップ・オプション</h2>
<li>バックアップ済みのサーバログの保持期限を指定します。<code>--keep-srvlog-files</code> では保存ファイル数を、<code>--keep-srvlog-days</code> では保存日数を指定します。これらのオプションを明示的に指定した場合、閾値を超えたファイルがサーバログが保存されるディレクトリ (log_directory) から削除されます。 本オプションは、<code>--with-serverlog</code>、<code>--srvlog-path</code> を指定してバックアップを取得した場合に限ります。 これらのオプションを同時に指定した場合は両方の閾値を超えたファイルが削除されます。</li>
</ul>
</li>
<li><strong><code>--exclude-tmpdir</code></strong>

<ul>
<li>一時ファイル用に使われる <code>pgsql_tmp</code> ディレクトリ内と、ロジカルレプリケーションで使われる <code>snapshots</code> ディレクトリ内をバックアップ対象から除外します。</li>
</ul>
</li>
</ul>


Expand Down Expand Up @@ -1071,6 +1077,14 @@ <h1 id="waytopassoptions">パラメータの指定方法</h1>
<td>アーカイブWALのリストア方法</td>
<td>環境変数、設定ファイルにはboolean型で指定</td>
</tr>
<tr>
<td></td>
<td>&ndash;exclude-tmpdir</td>
<td>EXCLUDE_TMPDIR</td>
<td>指定可</td>
<td>一時ファイルとロジカルスナップショットをバックアップから除外</td>
<td>環境変数、設定ファイルにはboolean型で指定</td>
</tr>
</tbody>
</table>

Expand Down Expand Up @@ -1472,4 +1486,4 @@ <h1 id="seealso">関連項目</h2>
<p class="footer">Copyright (c) 2009-2023, NIPPON TELEGRAPH AND TELEPHONE CORPORATION</p>

</body>
</html>
</html>
16 changes: 15 additions & 1 deletion docs/index.html
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -651,6 +651,12 @@ <h2>Backup options</h2>
<li>Specify how long backuped serverlog files will be kept. <code>--keep-srvlog-files</code> means number of files. <code>--keep-srvlog-days</code> means days to be kept. When you do backup, only files exceeded specified condition are deleted from server log directory (log_directory). This option works when you specify <code>--with-serverlog</code> and <code>--srvlog-path</code> options in backup command. If these two options are given toghether, pg_rman deletes files which are old enough against both conditions.</li>
</ul>
</li>
<li><strong><code>--exclude-tmpdir</code></strong>

<ul>
<li>Exclude the <code>pgsql_tmp</code> directory, which is used for temporary files, and the <code>snapshots</code> directory, which is used for logical replication.</li>
</ul>
</li>
</ul>


Expand Down Expand Up @@ -1062,6 +1068,14 @@ <h1 id="waytopassoptions">Way to pass options</h1>
<td>how to restore archive WAL</td>
<td>specify boolean type in environmental variable or configuration file</td>
</tr>
<tr>
<td></td>
<td>&ndash;exclude-tmpdir</td>
<td>EXCLUDE_TMPDIR</td>
<td>Yes</td>
<td>Exclude temporary files and logical snapshots from backup</td>
<td>specify boolean type in environmental variable or configuration file</td>
</tr>
</tbody>
</table>

Expand Down Expand Up @@ -1462,4 +1476,4 @@ <h1 id="seealso">See Also</h2>
<p class="footer">Copyright (c) 2009-2023, NIPPON TELEGRAPH AND TELEPHONE CORPORATION</p>

</body>
</html>
</html>
4 changes: 4 additions & 0 deletions expected/backup.out
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -74,3 +74,7 @@ INFO: Please execute 'pg_rman validate' to verify the files are correctly copied
ERROR: could not start backup
DETAIL: system identifier of target database is different from the one of initially configured database
10
###### BACKUP COMMAND TEST-0011 ######
###### full backup with --exclude-tmpdir option ######
0
3
1 change: 1 addition & 0 deletions expected/option.out
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -40,6 +40,7 @@ Backup options:
--keep-srvlog-days=DAY keep serverlog modified in DAY days
--standby-host=HOSTNAME standby host when taking backup from standby
--standby-port=PORT standby port when taking backup from standby
--exclude-tmpdir exclude pgsql_tmp and pg_logical/snapshots

Restore options:
--recovery-target-time time stamp up to which recovery will proceed
Expand Down
10 changes: 10 additions & 0 deletions pg_rman.c
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,8 @@
const char *PROGRAM_VERSION = "1.3.16";
const char *PROGRAM_URL = "http://github.com/ossc-db/pg_rman";
const char *PROGRAM_ISSUES = "http://github.com/ossc-db/pg_rman/issues";
const char *SNAPSHOTS_DIR = "snapshots";
const char *PGSQL_TMP_DIR = "pgsql_tmp";

/* path configuration */
char *backup_path;
Expand All @@ -32,6 +34,7 @@ bool check = false;

/* directory configuration */
pgBackup current;
static bool exclude_tmpdir = false;

/* backup configuration */
static bool smooth_checkpoint;
Expand Down Expand Up @@ -97,6 +100,7 @@ static pgut_option options[] =
{ 's', 10, "recovery-target-timeline" , &target_tli_string, SOURCE_ENV },
{ 's', 11, "recovery-target-action" , &target_action , SOURCE_ENV },
{ 'b', 12, "hard-copy" , &is_hard_copy , SOURCE_ENV },
{ 'b', 15, "exclude-tmpdir", &exclude_tmpdir, SOURCE_ENV },
/* catalog options */
{ 'b', 'a', "show-all" , &show_all },
{ 0 }
Expand Down Expand Up @@ -209,6 +213,11 @@ main(int argc, char *argv[])
pgdata_exclude[i++] = arclog_path;
if (srvlog_path)
pgdata_exclude[i++] = srvlog_path;
if (exclude_tmpdir)
{
pgdata_exclude[i++] = SNAPSHOTS_DIR;
pgdata_exclude[i++] = PGSQL_TMP_DIR;
}

/* do actual operation */
if (pg_strcasecmp(cmd, "init") == 0)
Expand Down Expand Up @@ -289,6 +298,7 @@ pgut_help(bool details)
printf(_(" --keep-srvlog-days=DAY keep serverlog modified in DAY days\n"));
printf(_(" --standby-host=HOSTNAME standby host when taking backup from standby\n"));
printf(_(" --standby-port=PORT standby port when taking backup from standby\n"));
printf(_(" --exclude-tmpdir exclude pgsql_tmp and pg_logical/snapshots\n"));
printf(_("\nRestore options:\n"));
printf(_(" --recovery-target-time time stamp up to which recovery will proceed\n"));
printf(_(" --recovery-target-xid transaction ID up to which recovery will proceed\n"));
Expand Down
18 changes: 15 additions & 3 deletions sql/backup.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -195,9 +195,9 @@ sleep 1
echo 'archive backup in the same situation but with --full-backup-on-error option'
pg_rman backup -B ${BACKUP_PATH} -b archive -F -s -Z -p ${TEST_PGPORT} -d postgres;echo $?
pg_rman validate -B ${BACKUP_PATH} --quiet
pg_rman show detail -B ${BACKUP_PATH} > ${TEST_BASE}/TEST-0011.log 2>&1
grep OK ${TEST_BASE}/TEST-0011.log | grep FULL | wc -l
grep ERROR ${TEST_BASE}/TEST-0011.log | grep ARCH | wc -l
pg_rman show detail -B ${BACKUP_PATH} > ${TEST_BASE}/TEST-0009.log 2>&1
grep OK ${TEST_BASE}/TEST-0009.log | grep FULL | wc -l
grep ERROR ${TEST_BASE}/TEST-0009.log | grep ARCH | wc -l

echo '###### BACKUP COMMAND TEST-0010 ######'
echo '###### failure in backup with different system identifier database ######'
Expand All @@ -206,6 +206,18 @@ pg_ctl stop -m immediate > /dev/null 2>&1
init_database
pg_rman backup -B ${BACKUP_PATH} -b full -p ${TEST_PGPORT} -d postgres --quiet;echo $?

echo '###### BACKUP COMMAND TEST-0011 ######'
echo '###### full backup with --exclude-tmpdir option ######'
init_catalog
mkdir -p $PGDATA_PATH/base/pgsql_tmp
touch $PGDATA_PATH/base/pgsql_tmp/012345
touch $PGDATA_PATH/pg_logical/snapshots/012345
pg_rman backup -B ${BACKUP_PATH} -b full -s --exclude-tmpdir -p ${TEST_PGPORT} -d postgres --quiet;echo $?
pg_rman validate -B ${BACKUP_PATH} --quiet
pg_rman show detail -B ${BACKUP_PATH} > ${TEST_BASE}/TEST-0011.log 2>&1
test -f ${BACKUP_PATH}/????????/??????/database/base/pgsql_tmp/012345 || echo OK >> ${TEST_BASE}/TEST-0011.log
test -f ${BACKUP_PATH}/????????/??????/database/pg_logical/snapshots/012345 || echo OK >> ${TEST_BASE}/TEST-0011.log
grep -c OK ${TEST_BASE}/TEST-0011.log

# cleanup
## clean up the temporal test data
Expand Down