Skip to content

Commit

Permalink
reorganize and edit docs for upcoming release
Browse files Browse the repository at this point in the history 
Signed-off-by: JENNIFER RONDEAU <jrondeau@heptio.com>
  • Loading branch information
JENNIFER RONDEAU committed Jul 6, 2018
1 parent 3646fcc commit cfdcd65
Show file tree
Hide file tree
Showing 4 changed files with 226 additions and 238 deletions.
174 changes: 9 additions & 165 deletions README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -19,155 +19,7 @@ Ark consists of:

## More information

[The documentation][29] provides detailed information about building from source, architecture, extending Ark, and more.

## Getting started

The following example sets up the Ark server and client, then backs up and restores a sample application.

For simplicity, the example uses Minio, an S3-compatible storage service that runs locally on your cluster. See [Set up Ark with your cloud provider][3] for how to run on a cloud provider.

### Prerequisites

* Access to a Kubernetes cluster, version 1.7 or later. Version 1.7.5 or later is required to run `ark backup delete`.
* A DNS server on the cluster
* `kubectl` installed

### Download

Clone or fork the Ark repository:

```
git clone git@github.com:heptio/ark.git
```

NOTE: Make sure to check out the appropriate version. We recommend that you check out the latest tagged version. The master branch is under active development and might not be stable.

### Set up server

1. Start the server and the local storage service. In the root directory of Ark, run:

```bash
kubectl apply -f examples/common/00-prereqs.yaml
kubectl apply -f examples/minio/
```

NOTE: If you get an error about Config creation, wait for a minute, then run the commands again.

1. Deploy the example nginx application:

```bash
kubectl apply -f examples/nginx-app/base.yaml
```

1. Check to see that both the Ark and nginx deployments are successfully created:

```
kubectl get deployments -l component=ark --namespace=heptio-ark
kubectl get deployments --namespace=nginx-example
```

### Install client

For this example, we recommend that you [download a pre-built release][26].

You can also [build from source][7].

Make sure that you install somewhere in your `$PATH`.

### Back up

1. Create a backup for any object that matches the `app=nginx` label selector:

```
ark backup create nginx-backup --selector app=nginx
```

Alternatively if you want to backup all objects *except* those matching the label `backup=ignore`:

```
ark backup create nginx-backup --selector 'backup notin (ignore)'
```

1. Simulate a disaster:

```
kubectl delete namespace nginx-example
```

1. To check that the nginx deployment and service are gone, run:

```
kubectl get deployments --namespace=nginx-example
kubectl get services --namespace=nginx-example
kubectl get namespace/nginx-example
```

You should get no results.

NOTE: You might need to wait for a few minutes for the namespace to be fully cleaned up.

### Restore

1. Run:

```
ark restore create --from-backup nginx-backup
```

1. Run:

```
ark restore get
```

After the restore finishes, the output looks like the following:

```
NAME BACKUP STATUS WARNINGS ERRORS CREATED SELECTOR
nginx-backup-20170727200524 nginx-backup Completed 0 0 2017-07-27 20:05:24 +0000 UTC <none>
```

NOTE: The restore can take a few moments to finish. During this time, the `STATUS` column reads `InProgress`.

After a successful restore, the `STATUS` column is `Completed`, and `WARNINGS` and `ERRORS` are 0. All objects in the `nginx-example` namespace should be just as they were before you deleted them.

If there are errors or warnings, you can look at them in detail:

```
ark restore describe <RESTORE_NAME>
```

For more information, see [the debugging information][18].

### Clean up

If you want to delete any backups you created, including data in object storage and persistent
volume snapshots, you can run:

```
ark backup delete BACKUP_NAME
```

This asks the Ark server to delete all backup data associated with `BACKUP_NAME`. You need to do
this for each backup you want to permanently delete. A future version of Ark will allow you to
delete multiple backups by name or label selector.

Once fully removed, the backup is no longer visible when you run:

```
ark backup get BACKUP_NAME
```

If you want to uninstall Ark but preserve the backup data in object storage and persistent volume
snapshots, it is safe to remove the `heptio-ark` namespace and everything else created for this
example:

```
kubectl delete -f examples/common/
kubectl delete -f examples/minio/
kubectl delete -f examples/nginx-app/base.yaml
```
[The documentation][29] provides a getting started guide, plus information about building from source, architecture, extending Ark, and more.

## Troubleshooting

Expand All @@ -179,12 +31,12 @@ Thanks for taking the time to join our community and start contributing!

Feedback and discussion is available on [the mailing list][24].

#### Before you start
### Before you start

* Please familiarize yourself with the [Code of Conduct][8] before contributing.
* See [CONTRIBUTING.md][5] for instructions on the developer certificate of origin that we require.

#### Pull requests
### Pull requests

* We welcome pull requests. Feel free to dig through the [issues][4] and jump in.

Expand All @@ -195,30 +47,22 @@ See [the list of releases][6] to find out about feature changes.
[0]: https://github.com/heptio
[1]: https://travis-ci.org/heptio/ark.svg?branch=master
[2]: https://travis-ci.org/heptio/ark
[3]: /docs/cloud-common.md

[4]: https://github.com/heptio/ark/issues
[5]: https://github.com/heptio/ark/blob/master/CONTRIBUTING.md
[6]: https://github.com/heptio/ark/releases
[7]: /docs/build-from-scratch.md

[8]: https://github.com/heptio/ark/blob/master/CODE_OF_CONDUCT.md
[9]: https://kubernetes.io/docs/setup/
[10]: https://kubernetes.io/docs/tasks/tools/install-kubectl/#install-with-homebrew-on-macos
[11]: https://kubernetes.io/docs/tasks/tools/install-kubectl/#tabset-1
[12]: https://github.com/kubernetes/kubernetes/blob/master/cluster/addons/dns/README.md
[13]: /docs/output-file-format.md
[14]: https://github.com/kubernetes/kubernetes
[15]: https://aws.amazon.com/
[16]: https://cloud.google.com/
[17]: https://azure.microsoft.com/
[18]: /docs/debugging-restores.md
[19]: /docs/img/backup-process.png
[20]: https://kubernetes.io/docs/concepts/api-extension/custom-resources/#customresourcedefinitions
[21]: https://kubernetes.io/docs/concepts/api-extension/custom-resources/#custom-controllers
[22]: https://github.com/coreos/etcd


[24]: http://j.hept.io/ark-list
[25]: https://kubernetes.slack.com/messages/ark-dr
[26]: https://github.com/heptio/ark/releases
[27]: /docs/hooks.md
[28]: /docs/plugins.md


[29]: https://heptio.github.io/ark/
[30]: /docs/troubleshooting.md
45 changes: 18 additions & 27 deletions docs/about.md
View file
Original file line number Diff line number Diff line change
@@ -1,22 +1,12 @@
# About Heptio Ark
# How Ark Works

Heptio Ark provides customizable degrees of recovery for all Kubernetes objects (Pods, Deployments, Jobs, Custom Resource Definitions, etc.), as well as for persistent volumes. This recovery can be cluster-wide, or fine-tuned according to object type, namespace, or labels.
Each Ark operation -- on-demand backup, scheduled backup, restore -- is a custom resource, defined with a Kubernetes [Custom Resource Definition (CRD)][20] and stored in [etcd][22]. The config custom resource specifies core information and options such as cloud provider settings. Ark also includes controllers that process the custom resources to perform backups, restores, and all related operations.

Ark is ideal for the disaster recovery use case, as well as for snapshotting your application state, prior to performing system operations on your cluster (e.g. upgrades).

## Features

Ark provides the following operations:

* On-demand backups
* Scheduled backups
* Restores
You can back up or restore all objects in your cluster, or you can filter objects by type, namespace, and/or label.

Each operation is a custom resource, defined with a Kubernetes [Custom Resource Definition (CRD)][20] and stored in [etcd][22]. An additional custom resource, Config, specifies required information and customized options, such as cloud provider settings. These resources are handled by [custom controllers][21] when their corresponding requests are submitted to the Kubernetes API server.

Each controller watches its custom resource for API requests (Ark operations), performs validations, and handles the logic for interacting with the cloud provider API -- for example, managing object storage and persistent volumes.
Ark is ideal for the disaster recovery use case, as well as for snapshotting your application state, prior to performing system operations on your cluster (e.g. upgrades).

### On-demand backups
## On-demand backups

The **backup** operation:

Expand All @@ -29,25 +19,23 @@ need to tell a database to flush its in-memory buffers to disk before taking a s

Note that cluster backups are not strictly atomic. If Kubernetes objects are being created or edited at the time of backup, they might not be included in the backup. The odds of capturing inconsistent information are low, but it is possible.

### Scheduled backups
## Scheduled backups

The **schedule** operation allows you to back up your data at recurring intervals. The first backup is performed when the schedule is first created, and subsequent backups happen at the schedule's specified interval. These intervals are specified by a Cron expression.

A Schedule acts as a wrapper for Backups; when triggered, it creates them behind the scenes.

Scheduled backups are saved with the name `<SCHEDULE NAME>-<TIMESTAMP>`, where `<TIMESTAMP>` is formatted as *YYYYMMDDhhmmss*.

### Restores
## Restores

The **restore** operation allows you to restore all of the objects and persistent volumes from a previously created Backup. Heptio Ark supports multiple namespace remapping--for example, in a single restore, objects in namespace "abc" can be recreated under namespace "def", and the ones in "123" under "456".
The **restore** operation allows you to restore all of the objects and persistent volumes from a previously created backup. You can also restore only a filtered subset of objects and persistent volumes. Ark supports multiple namespace remapping--for example, in a single restore, objects in namespace "abc" can be recreated under namespace "def", and the objects in namespace "123" under "456".

Kubernetes objects that have been restored can be identified with a label that looks like `ark-restore=<BACKUP NAME>-<TIMESTAMP>`, where `<TIMESTAMP>` is formatted as *YYYYMMDDhhmmss*.
The default name of a restore is `<BACKUP NAME>-<TIMESTAMP>`, where `<TIMESTAMP>` is formatted as *YYYYMMDDhhmmss*. You can also specify a custom name. A restored object also includes a label with key `ark-restore` and value `<RESTORE NAME>`.

You can also run the Ark server in restore-only mode, which disables backup, schedule, and garbage collection functionality during disaster recovery.

## Backup workflow

Here's what happens when you run `ark backup create test-backup`:
When you run `ark backup create test-backup`:

1. The Ark client makes a call to the Kubernetes API server to create a `Backup` object.

Expand All @@ -57,24 +45,27 @@ Here's what happens when you run `ark backup create test-backup`:

1. The `BackupController` makes a call to the object storage service -- for example, AWS S3 -- to upload the backup file.

By default `ark backup create` makes disk snapshots of any persistent volumes. You can adjust the snapshots by specifying additional flags. See [the CLI help][30] for more information. Snapshots can be disabled with the option `--snapshot-volumes=false`.
By default, `ark backup create` makes disk snapshots of any persistent volumes. You can adjust the snapshots by specifying additional flags. See [the CLI help][30] for more information. Snapshots can be disabled with the option `--snapshot-volumes=false`.

![19]

## Set a backup to expire

When you create a backup, you can specify a TTL by adding the flag `--ttl <DURATION>`. If Ark sees that an existing Backup resource is expired, it removes:
When you create a backup, you can specify a TTL by adding the flag `--ttl <DURATION>`. If Ark sees that an existing backup resource is expired, it removes:

* The Backup resource
* The backup resource
* The backup file from cloud object storage
* All PersistentVolume snapshots
* All associated Restores

## Object storage sync

Heptio Ark treats object storage as the source of truth. It continuously checks to see that the correct Backup resources are always present. If there is a properly formatted backup file in the storage bucket, but no corresponding Backup resources in the Kubernetes API, Ark synchronizes the information from object storage to Kubernetes.
Heptio Ark treats object storage as the source of truth. It continuously checks to see that the correct backup resources are always present. If there is a properly formatted backup file in the storage bucket, but no corresponding backup resource in the Kubernetes API, Ark synchronizes the information from object storage to Kubernetes.

This allows restore functionality to work in a cluster migration scenario, where the original Backup objects do not exist in the new cluster. See the tutorials for details.
This allows restore functionality to work in a cluster migration scenario, where the original backup objects do not exist in the new cluster.

[19]: /img/backup-process.png
[20]: https://kubernetes.io/docs/concepts/api-extension/custom-resources/#customresourcedefinitions
[21]: https://kubernetes.io/docs/concepts/api-extension/custom-resources/#custom-controllers
[22]: https://github.com/coreos/etcd
[30]: https://github.com/heptio/ark/blob/master/docs/cli-reference/ark_create_backup.md
Loading

0 comments on commit cfdcd65

Please sign in to comment.