Skip to content

Commit

Permalink
file flexvolume.md moved to the new folder /devel/sig-storage - tombs…
Browse files Browse the repository at this point in the history
…tome file created - URLs updated
  • Loading branch information
eduartua committed Jan 29, 2019
1 parent ca79d65 commit ab55d85
Show file tree
Hide file tree
Showing 5 changed files with 169 additions and 158 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@ Kubernetes volume plugins are currently “in-tree” meaning they are linked, c
4. Volume plugins get full privileges of kubernetes components (kubelet and kube-controller-manager).
5. Plugin developers are forced to make plugin source code available, and can not choose to release just a binary.

The existing [Flex Volume](/contributors/devel/flexvolume.md) plugin attempted to address this by exposing an exec based API for mount/unmount/attach/detach. Although it enables third party storage vendors to write drivers out-of-tree, it requires access to the root filesystem of node and master machines in order to deploy the third party driver files.
The existing [Flex Volume] plugin attempted to address this by exposing an exec based API for mount/unmount/attach/detach. Although it enables third party storage vendors to write drivers out-of-tree, it requires access to the root filesystem of node and master machines in order to deploy the third party driver files.

Additionally, it doesn’t address another pain of in-tree volumes plugins: dependencies. Volume plugins tend to have many external requirements: dependencies on mount and filesystem tools, for example. These dependencies are assumed to be available on the underlying host OS, which often is not the case, and installing them requires direct machine access. There are efforts underway, for example https://github.com/kubernetes/community/pull/589, that are hoping to address this for in-tree volume plugins. But, enabling volume plugins to be completely containerized will make dependency management much easier.

Expand All @@ -56,7 +56,7 @@ The objective of this document is to document all the requirements for enabling
* Recommend deployment process for Kubernetes compatible, third-party CSI Volume drivers on a Kubernetes cluster.

## Non-Goals
* Replace [Flex Volume plugin](/contributors/devel/flexvolume.md)
* Replace [Flex Volume plugin]
* The Flex volume plugin exists as an exec based mechanism to create “out-of-tree” volume plugins.
* Because Flex drivers exist and depend on the Flex interface, it will continue to be supported with a stable API.
* The CSI Volume plugin will co-exist with Flex volume plugin.
Expand Down Expand Up @@ -777,3 +777,7 @@ Instead of creating a new `VolumeAttachment` object, another option we considere
* List of nodes the volume was successfully attached to.

We dismissed this approach because having attach/detach triggered by the creation/deletion of an object is much easier to manage (for both external-attacher and Kubernetes) and more robust (fewer corner cases to worry about).


[Flex Volume]: /contributors/devel/sig-storage/flexvolume.md
[Flex Volume plugin]: /contributors/devel/sig-storage/flexvolume.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ Beginning in version 1.8, the Kubernetes Storage SIG is putting a stop to accept

[CSI](https://github.com/container-storage-interface/spec/blob/master/spec.md) provides a single interface that storage vendors can implement in order for their storage solutions to work across many different container orchestrators, and volume plugins are out-of-tree by design. This is a large effort, the full implementation of CSI is several quarters away, and there is a need for an immediate solution for storage vendors to continue adding volume plugins.

[Flexvolume](/contributors/devel/flexvolume.md) is an in-tree plugin that has the ability to run any storage solution by executing volume commands against a user-provided driver on the Kubernetes host, and this currently exists today. However, the process of setting up Flexvolume is very manual, pushing it out of consideration for many users. Problems include having to copy the driver to a specific location in each node, manually restarting kubelet, and user's limited access to machines.
[Flexvolume] is an in-tree plugin that has the ability to run any storage solution by executing volume commands against a user-provided driver on the Kubernetes host, and this currently exists today. However, the process of setting up Flexvolume is very manual, pushing it out of consideration for many users. Problems include having to copy the driver to a specific location in each node, manually restarting kubelet, and user's limited access to machines.

An automated deployment technique is discussed in [Recommended Driver Deployment Method](#recommended-driver-deployment-method). The crucial change required to enable this method is allowing kubelet and controller manager to dynamically discover plugin changes.

Expand Down Expand Up @@ -164,3 +164,5 @@ Cons: Does not guarantee every node has a pod running. Pod anti-affinity can be
* How does this system work with containerized kubelet?
* Are there any SELinux implications?
[Flexvolume]: /contributors/devel/sig-storage/flexvolume.md
156 changes: 2 additions & 154 deletions contributors/devel/flexvolume.md
Original file line number Diff line number Diff line change
@@ -1,155 +1,3 @@
# Flexvolume
This file has moved to https://git.k8s.io/community/contributors/devel/sig-storage/flexvolume.md.

Flexvolume enables users to write their own drivers and add support for their volumes in Kubernetes. Vendor drivers should be installed in the volume plugin path on every node, and on master if the driver requires attach capability (unless `--enable-controller-attach-detach` Kubelet option is set to false, but this is highly discouraged because it is a legacy mode of operation).

Flexvolume is a GA feature from Kubernetes 1.8 release onwards.

## Prerequisites

Install the vendor driver on all nodes (also on master nodes if "--enable-controller-attach-detach" Kubelet option is enabled) in the plugin path. Path for installing the plugin: `<plugindir>/<vendor~driver>/<driver>`. The default plugin directory is `/usr/libexec/kubernetes/kubelet-plugins/volume/exec/`. It can be changed in kubelet via the `--volume-plugin-dir` flag, and in controller manager via the `--flex-volume-plugin-dir` flag.

For example to add a `cifs` driver, by vendor `foo` install the driver at: `/usr/libexec/kubernetes/kubelet-plugins/volume/exec/foo~cifs/cifs`

The vendor and driver names must match flexVolume.driver in the volume spec, with '~' replaced with '/'. For example, if `flexVolume.driver` is set to `foo/cifs`, then the vendor is `foo`, and driver is `cifs`.

## Dynamic Plugin Discovery
Beginning in v1.8, Flexvolume supports the ability to detect drivers on the fly. Instead of requiring drivers to exist at system initialization time or having to restart kubelet or controller manager, drivers can be installed, upgraded/downgraded, and uninstalled while the system is running.
For more information, please refer to the [design document](/contributors/design-proposals/storage/flexvolume-deployment.md).

## Automated Plugin Installation/Upgrade
One possible way to install and upgrade your Flexvolume drivers is by using a DaemonSet. See [Recommended Driver Deployment Method](/contributors/design-proposals/storage/flexvolume-deployment.md#recommended-driver-deployment-method) for details, and see [here](https://git.k8s.io/examples/staging/volumes/flexvolume/deploy/) for an example.

## Plugin details
The plugin expects the following call-outs are implemented for the backend drivers. Some call-outs are optional. Call-outs are invoked from Kubelet and Controller Manager.

### Driver invocation model:

#### Init:
Initializes the driver. Called during Kubelet & Controller manager initialization. On success, the function returns a capabilities map showing whether each Flexvolume capability is supported by the driver.
Current capabilities:
* `attach` - a boolean field indicating whether the driver requires attach and detach operations. This field is *required*, although for backward-compatibility the default value is set to `true`, i.e. requires attach and detach.
See [Driver output](#driver-output) for the capabilities map format.
```
<driver executable> init
```

#### Attach:
Attach the volume specified by the given spec on the given node. On success, returns the device path where the device is attached on the node. Called from Controller Manager.

This call-out does not pass "secrets" specified in Flexvolume spec. If your driver requires secrets, do not implement this call-out and instead use "mount" call-out and implement attach and mount in that call-out.

```
<driver executable> attach <json options> <node name>
```

#### Detach:
Detach the volume from the node. Called from Controller Manager.
```
<driver executable> detach <mount device> <node name>
```

#### Wait for attach:
Wait for the volume to be attached on the remote node. On success, the path to the device is returned. Called from Controller Manager. The timeout should be 10m (based on https://git.k8s.io/kubernetes/pkg/kubelet/volumemanager/volume_manager.go#L88 )

```
<driver executable> waitforattach <mount device> <json options>
```

#### Volume is Attached:
Check the volume is attached on the node. Called from Controller Manager.

```
<driver executable> isattached <json options> <node name>
```

#### Mount device:
Mount device mounts the device to a global path which individual pods can then bind mount. Called only from Kubelet.

This call-out does not pass "secrets" specified in Flexvolume spec. If your driver requires secrets, do not implement this call-out and instead use "mount" call-out and implement attach and mount in that call-out.

```
<driver executable> mountdevice <mount dir> <mount device> <json options>
```

#### Unmount device:
Unmounts the global mount for the device. This is called once all bind mounts have been unmounted. Called only from Kubelet.

```
<driver executable> unmountdevice <mount device>
```
In addition to the user-specified options and [default JSON options](#default-json-options), the following options capturing information about the pod are passed through and generated automatically.

```
kubernetes.io/pod.name
kubernetes.io/pod.namespace
kubernetes.io/pod.uid
kubernetes.io/serviceAccount.name
```

#### Mount:
Mount the volume at the mount dir. This call-out defaults to bind mount for drivers which implement attach & mount-device call-outs. Called only from Kubelet.

```
<driver executable> mount <mount dir> <json options>
```

#### Unmount:
Unmount the volume. This call-out defaults to bind mount for drivers which implement attach & mount-device call-outs. Called only from Kubelet.

```
<driver executable> unmount <mount dir>
```

See [lvm] & [nfs] for a quick example on how to write a simple flexvolume driver.

### Driver output:

Flexvolume expects the driver to reply with the status of the operation in the
following format.

```
{
"status": "<Success/Failure/Not supported>",
"message": "<Reason for success/failure>",
"device": "<Path to the device attached. This field is valid only for attach & waitforattach call-outs>"
"volumeName": "<Cluster wide unique name of the volume. Valid only for getvolumename call-out>"
"attached": <True/False (Return true if volume is attached on the node. Valid only for isattached call-out)>
"capabilities": <Only included as part of the Init response>
{
"attach": <True/False (Return true if the driver implements attach and detach)>
}
}
```

### Default Json options

In addition to the flags specified by the user in the Options field of the FlexVolumeSource, the following flags (set through their corresponding FlexVolumeSource fields) are also passed to the executable.
Note: Secrets are passed only to "mount/unmount" call-outs.

```
"kubernetes.io/fsType":"<FS type>",
"kubernetes.io/readwrite":"<rw>",
"kubernetes.io/fsGroup":"<FS group>",
"kubernetes.io/mountsDir":"<string>",
"kubernetes.io/pvOrVolumeName":"<Volume name if the volume is in-line in the pod spec; PV name if the volume is a PV>"
"kubernetes.io/pod.name":"<string>",
"kubernetes.io/pod.namespace":"<string>",
"kubernetes.io/pod.uid":"<string>",
"kubernetes.io/serviceAccount.name":"<string>",
"kubernetes.io/secret/key1":"<secret1>"
...
"kubernetes.io/secret/keyN":"<secretN>"
```

### Example of Flexvolume

Please refer to the [Flexvolume example directory]. See [nginx-lvm.yaml] & [nginx-nfs.yaml] for a quick example on how to use Flexvolume in a pod.


[lvm]: https://git.k8s.io/examples/staging/volumes/flexvolume/lvm
[nfs]: https://git.k8s.io/examples/staging/volumes/flexvolume/nfs
[nginx-lvm.yaml]: https://git.k8s.io/examples/staging/volumes/flexvolume/nginx-lvm.yaml
[nginx-nfs.yaml]: https://git.k8s.io/examples/staging/volumes/flexvolume/nginx-nfs.yaml
[Flexvolume example directory]: https://git.k8s.io/examples/staging/volumes/flexvolume/
This file is a placeholder to preserve links. Please remove by April 29, 2019 or the release of kubernetes 1.13, whichever comes first.
Loading

0 comments on commit ab55d85

Please sign in to comment.