Use static PV based provisioning for pre-provisioned CephFS volume support

[ShyamsundarR]

OR, Title: Use CephFS CSI storage class for dynamic provisioning only

Current implementation of CephFS CSI plugin, has the option to use pre-provisioned CephFS subvolumes. This is done by crafting a StorageClass that contains provisionVolume as false and giving the rootPath a value of the actual path on the CephFS volume to mount and use. Further, when mentioning such pre-provisioned storage classes the secrets should contain a userID and userKey combination for use with the mount operation. (see [1])

The implementation for the above, makes the CreateVolume and DeleteVolume close to NO-OPs, just crafting and returning a VolumeID for use to identify the volume during mount operations.

An alternative to using pre-provisioned volumes is to create static PVs instead, and use the CSI nodeplugin to mount these PVs. The PV can have a claimRef, with required name and namespace keys and values, that can help match a PVC claim, and the PVC can have a volumeName specification to match the PV. Thus, retaining a PVC to PV 1:1 relationship.

I tried the above alternative, and can see that it functions as required, thus eliminating the need for the controller service to support the notion of detecting and faking Create/Delete for pre-provisioned CephFS volumes. A sample pod spec, all the way from defining the userID and key secret, to the pod using a PVC is provided here [2].

The CSI spec does not preclude using a nodeplugin exclusively, or rather does not mandate that only dynamically provisioned volumes must be honored by the nodeplugin. Thus, this is in line with the CSI specification.

This issue hence, is to remove the notion of static provisioning using a storage class, and instead using static PV provisioning, thus removing support for static provisioning or pre-provisioned CephFS volumes using a storage class from the 1.1.0 version of the plugin.

NOTE: 1.1.0 will retain backward compatibility with pre-provisioned volumes created prior to the 1.1.0 version

[1] StorageClass provisionVolume details:

ceph-csi/examples/cephfs/storageclass.yaml

Lines 12 to 19 in d02e50a

	# For provisionVolume: "true":
	# A new volume will be created along with a new Ceph user.
	# Requires admin credentials (adminID, adminKey).
	# For provisionVolume: "false":
	# It is assumed the volume already exists and the user is expected
	# to provide path to that volume (rootPath) and user credentials
	# (userID, userKey).
	provisionVolume: "true"

[2] YAML manifest for static PV with PVC and pod using the same: https://gist.github.com/ShyamsundarR/1009b79966992470f588a277429ae2e1

[ShyamsundarR]

@gman0 PTAL, I plan on removing the pre-provisioned CephFS support in 1.1.0 in favor of using the same via static PV provisioning.

[gman0]

@ShyamsundarR yes this is fine, thanks!