Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ ADD . /opt/grafana-backup-tool
RUN chmod -R a+r /opt/grafana-backup-tool \
&& find /opt/grafana-backup-tool -type d -print0 | xargs -0 chmod a+rx

RUN pip3 --no-cache-dir install .
RUN pip3 --no-cache-dir install --break-system-packages .

RUN chown -R 1337:1337 /opt/grafana-backup-tool
USER 1337
Expand Down
66 changes: 44 additions & 22 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,13 +3,15 @@
A Python-based application to backup Grafana settings using the [Grafana API](https://grafana.com/docs/grafana/latest/http_api/).

The aim of this tool is to:

1. Easily backup and restore Grafana.
2. Have versioned backups`(date and time in file name)` for restoring and saving to cloud storage providers. Currently support
1. `Amazon S3`
2. `Azure Storage`
3. `GCP Cloud Storage` (Use service-account's credential file, [see here](https://cloud.google.com/storage/docs/reference/libraries#linux-or-macos))

## Supported components

* Folder
* Folder Permissions
* Library Elements (doesn't work with Grafana 8.0.0 but 8.4.3)
Expand All @@ -19,34 +21,37 @@ The aim of this tool is to:
* Alert Rules (Supported in version 9.4.0 of grafana and up.)
* Teams
* Team Members (Needs Basic Authentication (username and password, see [grafana doc](https://grafana.com/docs/grafana/latest/http_api/org/#admin-organizations-api))
* You need to set `Admin's account and password` in `grafanaSettings.json`, or set the base64 encoded `admin account and password` in ENV `GRAFANA_BASIC_AUTH`. E.g `export GRAFANA_BASIC_AUTH=YWRtaW46YWRtaW4=`
* Or Sets this ENV of the Grafana server `GF_USERS_ALLOW_ORG_CREATE=true`. see [grafana doc](https://grafana.com/docs/grafana/latest/http_api/org/#create-organization)
* You need to set `Admin's account and password` in `grafanaSettings.json`, or set the base64 encoded `admin account and password` in ENV `GRAFANA_BASIC_AUTH`. E.g `export GRAFANA_BASIC_AUTH=[Generated-Base64-Auth-Token]`
* Or Sets this ENV of the Grafana server `GF_USERS_ALLOW_ORG_CREATE=true`. see [grafana doc](https://grafana.com/docs/grafana/latest/http_api/org/#create-organization)
* Organization (Needs Basic Authentication (username and password, see [grafana doc](https://grafana.com/docs/grafana/latest/http_api/org/#admin-organizations-api))
* You need to set `Admin's account and password` in `grafanaSettings.json`, or set the base64 encoded `admin account and password` in ENV `GRAFANA_BASIC_AUTH`. E.g `export GRAFANA_BASIC_AUTH=YWRtaW46YWRtaW4=`
* Or Sets this ENV of the Grafana server `GF_USERS_ALLOW_ORG_CREATE=true`. see [grafana doc](https://grafana.com/docs/grafana/latest/http_api/org/#create-organization)
* You need to set `Admin's account and password` in `grafanaSettings.json`, or set the base64 encoded `admin account and password` in ENV `GRAFANA_BASIC_AUTH`. E.g `export GRAFANA_BASIC_AUTH=[Generated-Base64-Auth-Token]`
* Or Sets this ENV of the Grafana server `GF_USERS_ALLOW_ORG_CREATE=true`. see [grafana doc](https://grafana.com/docs/grafana/latest/http_api/org/#create-organization)
* User (Needs Basic Authentication (username and password, see [grafana doc](https://grafana.com/docs/grafana/latest/http_api/org/#admin-organizations-api))
* You need to set `Admin's account and password` in `grafanaSettings.json`, or set the base64 encoded `admin account and password` in ENV `GRAFANA_BASIC_AUTH`. E.g `export GRAFANA_BASIC_AUTH=YWRtaW46YWRtaW4=`
* Grafana's api doesn't provide user's password when backing up, so the `default_user_password` in `grafanaSettings.json`, or in ENV `DEFAULT_USER_PASSWORD`, E.g `export DEFAULT_USER_PASSWORD=supersecret` will be used when restoring.
* You need to set `Admin's account and password` in `grafanaSettings.json`, or set the base64 encoded `admin account and password` in ENV `GRAFANA_BASIC_AUTH`. E.g `export GRAFANA_BASIC_AUTH=[Generated-Base64-Auth-Token]`
* Grafana's api doesn't provide user's password when backing up, so the `default_user_password` in `grafanaSettings.json`, or in ENV `DEFAULT_USER_PASSWORD`, E.g `export DEFAULT_USER_PASSWORD=supersecret` will be used when restoring.
* Snapshots
* Dashboard Versions (only backup, no restore)
* Annotations

**NOTE** The only supported `orgId` right now is `1`, the default organization will be backed up only!

## Requirements

* Bash
* Python 2.7 or Python 3.x
* Access to a Grafana API server.
* A `Token` of an `Admin` role (see `Configuration` section below for more info)

## Configuration

There are three ways to setup the configuration:

1. Use `environment variables` to define the variables for connecting to a Grafana server.
2. Use `hard-coded settings` in `conf/grafanaSettings.json` (this is the default settings file if not specified otherwise).
3. Use `~/.grafana-backup.json` to define variables in json format.

### Example Config

* Copy [grafanaSettings.example.json](examples/grafanaSettings.example.json) and modify it for you to use, remove `azure`, `aws`, `gcp`, `influxdb` blocks (but keep the ones you used).
* Check out the [examples](examples) folder for more configuration details.

Expand All @@ -57,7 +62,7 @@ There are three ways to setup the configuration:
```bash
### Do not use a trailing slash on GRAFANA_URL
export GRAFANA_URL=http://some.host.org:3000
export GRAFANA_TOKEN=eyJrIjoidUhaU2ZQQndrWFN3RRVkUnVfrT56a1JoaG9KWFFObEgiLCJuIjoiYWRtaW4iLCJpZCI6MX0=
export GRAFANA_TOKEN=[Grafana-Access-Token]

# GRAFANA_HEADERS is optional
export GRAFANA_HEADERS=Host:some.host.org
Expand All @@ -68,30 +73,39 @@ To create and obtain a `Token` for your Grafana server, please refer to the [off
**NOTE** that you need to generate a `Token` with an `Admin` role for the backup to succeed, otherwise you will have potential permission issues.

## Installation

### Virtual environment (optional but recommended)

Create a virtualenv, you could using something like `pyenv` if you'd prefer
```

```bash
virtualenv -p $(which python3) venv
source venv/bin/activate
```

### Installation using pypi
```

```bash
pip install grafana-backup
```

### Installation using this repo

First clone this repo
```

```bash
git clone https://github.com/ysde/grafana-backup-tool.git
cd grafana-backup-tool
```

Installation works best using `pip`
```

```bash
pip install .
```

## How to Use

* First perform the **Configuration** and **Installation** sections as described above.
* Use the `grafana-backup save` command to backup all your folders, dashboards, datasources and alert channels to the `_OUTPUT_` subdirectory of the current directory.

Expand All @@ -115,24 +129,28 @@ $ grafana-backup restore _OUTPUT_/202006272027.tar.gz
```

## Docker

Replace variables below to use the Docker version of this tool

* `{YOUR_GRAFANA_TOKEN}`: Your Grafana site `Token`.
* `{YOUR_GRAFANA_URL}`: Your Grafana site `URL`.
* `{YOUR_BACKUP_FOLDER_ON_THE_HOST}`: The `backup folder` on the Grafana host machine.

## Kubernetes

Check out the CronJob in [examples](examples) for a simple example of how grafana-backup-tool
can be ran within a Kubernetes environment

### Backup

If you decide to use a volume (-v) then you'll need to create the volume first with 1337 uid/gid ownership first, example:
```

```bash
mkdir /tmp/backup
sudo chown 1337:1337 /tmp/backup
```

```
```bash
docker run --user $(id -u):$(id -g) --rm --name grafana-backup-tool \
-e GRAFANA_TOKEN={YOUR_GRAFANA_TOKEN} \
-e GRAFANA_URL={YOUR_GRAFANA_URL} \
Expand All @@ -145,9 +163,9 @@ docker run --user $(id -u):$(id -g) --rm --name grafana-backup-tool \

***Example:***

```
```bash
docker run --user $(id -u):$(id -g) --rm --name grafana-backup-tool \
-e GRAFANA_TOKEN="eyJrIjoiNGZqTDEyeXNaY0RsMXNhbkNTSnlKN2M3bE1VeHdqVTEiLCJuIjoiZ3JhZmFuYS1iYWNrdXAiLCJpZCI6MX0=" \
-e GRAFANA_TOKEN="[Grafana-Access-Token]" \
-e GRAFANA_URL=http://192.168.0.79:3000 \
-e GRAFANA_ADMIN_ACCOUNT=admin \
-e GRAFANA_ADMIN_PASSWORD=adminpassword \
Expand All @@ -157,7 +175,8 @@ docker run --user $(id -u):$(id -g) --rm --name grafana-backup-tool \
```

***S3 Example:*** Set S3 configurations in `-e` or `grafanaSettings.json`([example](https://github.com/ysde/grafana-backup-tool/blob/master/examples/grafana-backup.example.json))
```

```bash
-e AWS_S3_BUCKET_NAME="my-backups-bucket" \
-e AWS_S3_BUCKET_KEY="grafana-backup-folder" \
-e AWS_DEFAULT_REGION="us-east-1" \
Expand All @@ -166,22 +185,23 @@ docker run --user $(id -u):$(id -g) --rm --name grafana-backup-tool \
```

***Azure Example:*** Set Azure configurations in `-e` or `grafanaSettings.json`([example](https://github.com/ysde/grafana-backup-tool/blob/master/examples/grafana-backup.example.json))
```

```bash
-e AZURE_STORAGE_CONTAINER_NAME="azure-storage-container-name" \
-e AZURE_STORAGE_CONNECTION_STRING="azure-storage-connection-string"
```

***GCS Example:*** Set GCS configurations in `-e` or `grafanaSettings.json`([example](https://github.com/ysde/grafana-backup-tool/blob/master/examples/grafana-backup.example.json))
```

```bash
-e GCS_BUCKET_NAME="bucket-name" \
-e GCLOUD_PROJECT="gcp-project-name" \
-e GOOGLE_APPLICATION_CREDENTIALS="credential-file-path"
```


### Restore

```
```bash
docker run --user $(id -u):$(id -g) --rm --name grafana-backup-tool \
-e GRAFANA_TOKEN={YOUR_GRAFANA_TOKEN} \
-e GRAFANA_URL={YOUR_GRAFANA_URL} \
Expand All @@ -196,9 +216,9 @@ docker run --user $(id -u):$(id -g) --rm --name grafana-backup-tool \

***Example:***

```
```bash
docker run --user $(id -u):$(id -g) --rm --name grafana-backup-tool \
-e GRAFANA_TOKEN="eyJrIjoiNGZqTDEyeXNaY0RsMXNhbkNTSnlKN2M3bE1VeHdqVTEiLCJuIjoiZ3JhZmFuYS1iYWNrdXAiLCJpZCI6MX0=" \
-e GRAFANA_TOKEN="[Grafana-Access-Token]" \
-e GRAFANA_URL=http://192.168.0.79:3000 \
-e GRAFANA_ADMIN_ACCOUNT=admin \
-e GRAFANA_ADMIN_PASSWORD=adminpassword \
Expand All @@ -210,9 +230,11 @@ docker run --user $(id -u):$(id -g) --rm --name grafana-backup-tool \
```

### Building

You can build the docker image simply by executing `make` in the root of this repo. The image will get tagged as `ysde:grafana-backup`

### Monitoring

InfluxDB support has been added and Prometheus push gateway support will be added in the future.

In order to monitor successful backups with InfluxDB simply configure `grafana-backup` InfluxDB settings using this [example](examples) configuration.
Expand Down
2 changes: 1 addition & 1 deletion grafana_backup/create_alert_rule.py
Original file line number Diff line number Diff line change
Expand Up @@ -32,8 +32,8 @@ def main(args, settings, file_path):
get_response= get_alert_rule(uid, grafana_url, http_get_headers, verify_ssl, client_cert, debug)
status_code=get_response[0]
print("Got a code: {0}", status_code)
http_post_headers['x-disable-provenance']='*'
if status_code == 404:
http_post_headers['x-disable-provenance']='*'
result = create_alert_rule(json.dumps(alert_rule), grafana_url, http_post_headers, verify_ssl, client_cert, debug)
else:
result = update_alert_rule(alert_rule['uid'], json.dumps(alert_rule), grafana_url, http_post_headers, verify_ssl, client_cert, debug)
Expand Down