Skip to content

Commit

Permalink
First stub of documentation
Browse files Browse the repository at this point in the history 
Co-authored-by: Gabriele Bartolini <gabriele.bartolini@2ndquadrant.it>
  • Loading branch information
@gbartolini
gbartolini and gbartolini committed Mar 6, 2020
1 parent ebde943 commit 1a51a6f
Show file tree
Hide file tree
Showing 13 changed files with 476 additions and 2 deletions.
Binary file added docs/graffle/architecture-r.graffle
View file
Binary file not shown.
Binary file added docs/graffle/architecture-rw.graffle
View file
Binary file not shown.
7 changes: 7 additions & 0 deletions docs/mkdocs.yml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -18,3 +18,10 @@ markdown_extensions:

nav:
- Cloud Native PostgreSQL: 'index.md'
- Before you start: 'before_you_start.md'
- Architecture: 'architecture.md'
- Quickstart: 'quickstart.md'
- Custom Resource Definitions: 'crd.md'
- Configuration samples: 'samples.md'
- End-to-end tests: 'e2e.md'
- Credits and license: 'credits.md'
114 changes: 114 additions & 0 deletions docs/src/architecture.md
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,114 @@
For High Availability goals, the PostgreSQL database management system provides administrators with built-in **physical replication** capabilities based on **Write Ahead Log (WAL) shipping**.

PostgreSQL supports both asynchronous and synchronous streaming replication, as well as asynchronous file-based log shipping (normally used as fallback option, for example to store WAL files in an object store). Replicas are normally called *standby servers* and can also be used for read-only workloads thanks to the *Hot Standby* feature.

Cloud Native PostgreSQL currently supports clusters based on asynchronous streaming replication to manage multiple hot standby replicas, with the following specifications:

* One primary, with optional multiple hot standby replicas for High Availability
* Available services for applications:
* `-rw`: applications connect to the only primary instance of the cluster
* `-r`: applications connect to any of the instances for read-only workloads
* Shared-nothing architecture recommended for better resilience of the PostgreSQL cluster:
* PostgreSQL instances should reside on different Kubernetes worker nodes and share only the network
* PostgreSQL instances can reside in different availability zones in the same region
* All nodes of a PostgreSQL cluster should reside in the same region

### Read-write workloads

Applications can decide to connect to the PostgreSQL instance elected as *current primary*
by the Kubernetes operator, as depicted in the following diagram:

![Applications writing to the single master](./images/architecture-rw.png)

Applications can simply use the `-rw` suffix service.

In case of temporary or permanent unavailability of the master, Kubernetes
will move the `-rw` to another instance of the cluster for high availability
purposes.

### Read-only workloads

!!! Important
Applications must be aware of the limitations that [Hot Standby](https://www.postgresql.org/docs/current/hot-standby.html)
presents and familiar with the way PostgreSQL operates when dealing with these workloads.

Applications can access to any PostgreSQL instance at any time through the `-r`
service made available by the operator at connection time.

The following diagram shows the architecture:

![Applications reading from any instance in round robin](./images/architecture-r.png)

## Application deployments

Applications are supposed to work with the services created by Cloud Native PostgreSQL
in the same Kubernetes cluster:

* `[cluster name]-rw`
* `[cluster name]-r`

Those services are entirely managed by the Kubernetes cluster and
implement a form of Virtual IP as described in the
["Service" page of the Kubernetes Documentation](https://kubernetes.io/docs/concepts/services-networking/service/#virtual-ips-and-service-proxies).

!!! Hint
It is highly recommended to use those services in your applications,
and avoid connecting directly to a specific PostgreSQL instance, as the latter
can change during the cluster lifetime.

You can use these services in your applications through:

* DNS resolution
* environment variables

As far as the credentials to connect to PostgreSQL are concerned, you can
use the secrets generated by the operator.

!!! Warning
The operator will create another service, named `[cluster name]-any`. That
service is used internally to manage PostgreSQL instance discovery.
It's not supposed to be used directly by applications.

### DNS resolution

You can use the Kubernetes DNS service, which is required by this operator,
to point to a given server.
You can do that by just using the name of the service if the application is
deployed in the same namespace as the PostgreSQL cluster.
In case the PostgreSQL cluster resides in a different namespace, you can use the
full qualifier: `service-name.namespace-name`.

DNS is the preferred and recommended discovery method.

### Environment variables

If you deploy your application in the same namespace that contains the
PostgreSQL cluster, you can also use environment variables to connect to the database.

For example, suppose that your PostgreSQL cluster is called `pg-database`,
you can use the following environment variables in your applications:

* `PG_DATABASE_R_SERVICE_HOST`: the IP address of the service
pointing to all the PostgreSQL instances for read-only workloads

* `PG_DATABASE_RW_SERVICE_HOST`: the IP address of the
service pointing to the *primary* instance of the cluster

### Secrets

The PostgreSQL operator will generate a secret for every PostgreSQL cluster it deploys.
That secret contains the passwords for the `postgres` user and also for
the *owner* of the database.

The secret generated has the same name as the `Cluster` resource created
in Kubernetes, and contains two entries:

* `postgresPassword` - containing the password of the `postgres` user, which
is the superuser defined in the instance;

* `ownerPassword` - containing the password of the user owning the database.

The latter credentials are the ones which should be by used the applications
connecting to the PostgreSQL cluster.

The first one is supposed to be used only for administrative purposes.
43 changes: 43 additions & 0 deletions docs/src/before_you_start.md
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@
Before we get started, it is important to go over some terminology that is
specific to Kubernetes and PostgreSQL.

## Kubernetes terminology

| Resource | Description |
|-------------------------------------------------------------------------------------------------------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| [Node](https://kubernetes.io/docs/concepts/architecture/nodes/) | A *node* is a worker machine in Kubernetes, either virtual or physical, where all services necessary to run pods are managed by the master(s). |
| [Pod](https://kubernetes.io/docs/concepts/workloads/pods/pod/) | A *pod* is the smallest computing unit that can be deployed in a Kubernetes cluster, and is composed by one or more containers that share network and storage. |
| [Service](https://kubernetes.io/docs/concepts/services-networking/service/) | A *service* is an abstraction that exposes as a network service an application that runs on a group of pods, and standardizes important features such as service discovery across applications, load balancing, failover, and so on. |
| [Secret](https://kubernetes.io/docs/concepts/configuration/secret/) | A *secret* is an object that is designed to store small amounts of sensitive data such as passwords, access keys or tokens, and use them in pods. |
| [Storage Class](https://kubernetes.io/docs/concepts/storage/storage-classes/) | A *storage class* allows an administrator to define the classes of storage in a cluster, including provisioner (such as AWS EBS), reclaim policies, mount options, volume expansion, and so on. |
| [Persistent Volume](https://kubernetes.io/docs/concepts/storage/persistent-volumes/) | A *persistent volume* (PV) is a resource in a Kubernetes cluster that represents storage that has been either manually provisioned by an administrator or dynamically provisioned by a *storage class* controller. A PV is associated to a pod using a *persistent volume claim* and its lifecycle is independent of any pod that uses it. Normally, a PV is a network volume, especially in the Public Cloud. A [*local persistent volume* (LPV)](https://kubernetes.io/docs/concepts/storage/volumes/#local) is a persistent volume that exist only on the particular node where the pod that uses it is running. |
| [Persistent Volume Claim](https://kubernetes.io/docs/concepts/storage/persistent-volumes/#persistentvolumeclaims) | A *persistent volume claim* (PVC) represents a request for storage which might include size, access mode, or a particular storage class. Similarly to how a pod consumes node resources, a PVC consumes resources of a PV. |
| [Namespace](https://kubernetes.io/docs/concepts/overview/working-with-objects/namespaces/) | A *namespace* is a logical and isolated subset of a Kubernetes cluster and can be seen as a *virtual cluster* within the wider physical cluster. Namespaces allow administrators to create separate environments, based on projects, departments, teams, and so on. |
| [RBAC](https://kubernetes.io/docs/reference/access-authn-authz/rbac/) | *Role Based Access Control* (RBAC), known also as *role-based security*, is a method used in computer systems security to restrict access to the network and resources of a system to authorized users only. Kubernetes has a native API to control roles at namespace and cluster level and associate them to specific resources and individuals. |
| [CRD](https://kubernetes.io/docs/concepts/extend-kubernetes/api-extension/custom-resources/) | A *custom resource definition* (CRD) is an extension of the Kubernetes API and allows developers to create new data types and objects, *called custom resources*. |
| [Operator](https://kubernetes.io/docs/concepts/extend-kubernetes/operator/) | An *operator* is a custom resource that automates those steps that are normally performed by a human operator when managing one or more applications or given services. An operator assists Kubernetes in making sure that the defined state of the resource always matches the observed one. |
| [kubectl](https://kubernetes.io/docs/reference/kubectl/overview/) | `kubectl` is the command line tool used to manage a Kubernetes cluster. |

Cloud Native PostgreSQL requires Kubernetes 1.15 or higher.

## PostgreSQL terminology

| Resource | Description |
|-------------------------------------------------------------------------------------------------------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| Instance | A Postgres server process running and listening on a pair "IP address(es)" and "TCP port" (usually 5432). |
| Primary | A PostgreSQL instance that can accept both read and write operations. |
| Replica | A PostgreSQL instance that is replicating from the only primary instance in a cluster and is kept updated by reading a stream of Write-Ahead Log (WAL) records. A replica is also known as *standby* or *secondary* server. PostgreSQL relies on both physical streaming replication (async/sync) and file-based log shipping (async). |
| Hot Standby | PostgreSQL feature that allows a *replica* to accept read-only workloads. |
| Cluster | To be intended as High Availability (HA) Cluster: a set of PostgreSQL instances made up by a single primary and an optional arbitrary number of replicas. |

## Cloud terminology

| Resource | Description |
|-------------------------------------------------------------------------------------------------------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| Region | A *region* in the Cloud is an isolated and independent geographic area organised in *availability zones*. Zones within a region have very little round-trip network latency. |
| Zone | An *availability zone* in the Cloud (also known as *zone*) is an area in a region where resources can be deployed. Usually, an availability zone corresponds to a data centre or an isolated building of the same data centre. |

## What to do next

Now that you have familiarized with terminology, you can decide to
[test Cloud Native PostgreSQL on your laptop using a local cluster](quickstart.md) before you deploy the operator in your selected cloud environment.
Loading

0 comments on commit 1a51a6f

Please sign in to comment.