Skip to content

Enable pre- and post-scp commands on remote host #43

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 7 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -243,6 +243,8 @@ Variable | Default | Notes
`SCP_HOST` | | When provided, the resulting backup file will be uploaded by means of `scp` to the host stated.
`SCP_USER` | | User name to log into `SCP_HOST`.
`SCP_DIRECTORY` | | Directory on `SCP_HOST` where backup file is stored.
`PRE_SCP_COMMAND` | | Commands that is executed on `SCP_HOST` before the backup is transferred.
`POST_SCP_COMMAND` | | Commands that is executed on `SCP_HOST` after the backup has been transferred.
`GPG_PASSPHRASE` | | When provided, the backup will be encrypted with gpg using this `passphrase`.
`INFLUXDB_URL` | | When provided, backup metrics will be sent to an InfluxDB instance at this URL, e.g. `https://influxdb.example.com`.
`INFLUXDB_DB` | | Required when using `INFLUXDB_URL`; e.g. `my_database`.
Expand Down Expand Up @@ -272,7 +274,7 @@ If so configured, they can also be shipped to an InfluxDB instance. This allows

## Automatic backup rotation

You probably don't want to keep all backups forever. A more common strategy is to hold onto a few recent ones, and remove older ones as they become irrelevant. There's no built-in support for this in `docker-volume-backup`, but it's simple enough to set up externally.
You probably don't want to keep all backups forever. A more common strategy is to hold onto a few recent ones, and remove older ones as they become irrelevant. There's no built-in support for this in `docker-volume-backup`, but if you transfer your backups via SCP to a remote host, you can trigger the rotate-backups script by means of setting the environmental variable `POST_SCP_COMMAND`.

### Rotation for local backups

Expand All @@ -281,6 +283,10 @@ Check out these utilities, for example:
* https://rotate-backups.readthedocs.io/en/latest/
* https://github.com/xolox/python-rotate-backups

### Rotation for backups tranferred via SCP

If you like to trigger `rotate-backups` on a remote host, install `rotate-backups` on the remote host (i.e., by means of `sudo pip install rotate-backups`). Then, follow the instructions for [backing up to remote host by means of SCP](#backing-up-to-remote-host-by-means-of-scp). Finally, set the environmental variable `POST_SCP_COMMAND: rotate-backups --daily 7 --weekly 4 --monthly 12 --yearly always /backup-directory` (where `/backup-directory` is the directory on the remote host where your backups has been transferred to). The suggested configuration preserves zero hourly, seven daily, four weekly, twelve monthly and unlimited yearly backups.

### Rotation for S3 backups

Amazon S3 has [Versioning](https://docs.aws.amazon.com/AmazonS3/latest/dev/Versioning.html) and [Object Lifecycle Management](https://docs.aws.amazon.com/AmazonS3/latest/dev/object-lifecycle-mgmt.html) features that can be useful for backups.
Expand Down
11 changes: 10 additions & 1 deletion src/backup.sh
Original file line number Diff line number Diff line change
Expand Up @@ -121,11 +121,20 @@ fi

if [ ! -z "$SCP_HOST" ]; then
info "Uploading backup by means of SCP"
SSH_CONFIG="-o StrictHostKeyChecking=no -i /ssh/id_rsa"
if [ ! -z "$PRE_SCP_COMMAND" ]; then
echo "Pre-scp command: $PRE_SCP_COMMAND"
ssh $SSH_CONFIG $SCP_USER@$SCP_HOST $PRE_SCP_COMMAND
fi
echo "Will upload to $SCP_HOST:$SCP_DIRECTORY"
TIME_UPLOAD="$(date +%s.%N)"
scp -ro StrictHostKeyChecking=no -i /ssh/id_rsa $BACKUP_FILENAME $SCP_USER@$SCP_HOST:$SCP_DIRECTORY
scp $SSH_CONFIG $BACKUP_FILENAME $SCP_USER@$SCP_HOST:$SCP_DIRECTORY
echo "Upload finished"
TIME_UPLOADED="$(date +%s.%N)"
if [ ! -z "$POST_SCP_COMMAND" ]; then
echo "Post-scp command: $POST_SCP_COMMAND"
ssh $SSH_CONFIG $SCP_USER@$SCP_HOST $POST_SCP_COMMAND
fi
fi

if [ -d "$BACKUP_ARCHIVE" ]; then
Expand Down
2 changes: 2 additions & 0 deletions src/entrypoint.sh
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,8 @@ AWS_EXTRA_ARGS="${AWS_EXTRA_ARGS:-}"
SCP_HOST="${SCP_HOST:-}"
SCP_USER="${SCP_USER:-}"
SCP_DIRECTORY="${SCP_DIRECTORY:-}"
PRE_SCP_COMMAND="${PRE_SCP_COMMAND:-}"
POST_SCP_COMMAND="${POST_SCP_COMMAND:-}"
BACKUP_FILENAME=${BACKUP_FILENAME:-"backup-%Y-%m-%dT%H-%M-%S.tar.gz"}
BACKUP_ARCHIVE="${BACKUP_ARCHIVE:-/archive}"
BACKUP_UID=${BACKUP_UID:-0}
Expand Down
25 changes: 25 additions & 0 deletions test/pre-post-scp-command/docker-compose.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
version: "3"

services:

dashboard:
image: grafana/grafana:7.4.5
volumes:
- grafana-data:/var/lib/grafana # This is where Grafana keeps its data

backup:
build: ../..
environment:
SCP_HOST: 192.168.0.42 # Remote host IP address
SCP_USER: pi # Remote host user to log in
SCP_DIRECTORY: /home/pi/backups # Remote host directory
# Commands that is executed before the backup is transferred by means of scp:
PRE_SCP_COMMAND: "ls -la /home/pi/backups"
# Command that is executed after the backup has been transferred by means of scp:
POST_SCP_COMMAND: "rotate-backups --daily 7 --weekly 4 --monthly 12 --yearly always /home/pi/backups"
volumes:
- grafana-data:/backup/grafana-data:ro # Mount the Grafana data volume (as read-only)
- ~/.ssh/id_rsa:/ssh/id_rsa:ro # Mount the SSH private key (as read-only)

volumes:
grafana-data: