Merge pull request #9 from shapeblue/enhancements-csi

Enhancements to CloudStack CSI driver

Author: Pearl1594

.github/workflows/release.yaml

@@ -90,6 +90,8 @@ jobs:
           sed -E "s|image: +cloudstack-csi-driver|image: ${REGISTRY_NAME}/cloudstack-csi-driver:${VERSION}|" deploy/k8s/controller-deployment.yaml >> manifest.yaml
           echo "---" >> manifest.yaml
           sed -E "s|image: +cloudstack-csi-driver|image: ${REGISTRY_NAME}/cloudstack-csi-driver:${VERSION}|" deploy/k8s/node-daemonset.yaml >> manifest.yaml
+          echo "---" >> manifest.yaml
+          cat deploy/k8s/volume-snapshot-class.yaml >> manifest.yaml
 
       - name: Create Release
         id: create_release
@@ -102,6 +104,16 @@ jobs:
           draft: false
           prerelease: false
 
+      - name: Upload Snapshot CRDs Asset
+        uses: actions/upload-release-asset@v1
+        env:
+          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+        with:
+          upload_url: ${{ steps.create_release.outputs.upload_url }}
+          asset_path: deploy/k8s/00-snapshot-crds.yaml
+          asset_name: snapshot-crds.yaml
+          asset_content_type: application/x-yaml
+
       - name: Upload Release Asset
         uses: actions/upload-release-asset@v1
         env:

README.md

@@ -1,3 +1,7 @@
+**Fork Notice:** 
+
+This repo is a fork of the [Leaseweb's] (https://github.com/leaseweb/cloudstack-csi-driver) maitained cloudstack-csi-driver, which is in-turn a fork of [Apalia's](https://github.com/apalia/cloudstack-csi-driver) cloudstack-csi-driver
+
 # CloudStack CSI Driver
 
 [![Go Reference](https://pkg.go.dev/badge/github.com/shapeblue/cloudstack-csi-driver.svg)](https://pkg.go.dev/github.com/shapeblue/cloudstack-csi-driver)
@@ -76,13 +80,28 @@ The storage class must also have a parameter named
 `csi.cloudstack.apache.org/disk-offering-id` whose value is the CloudStack disk
 offering ID.
 
+**Reclaim Policy**: Storage classes can have a `reclaimPolicy` of either `Delete` or `Retain`. If no `reclaimPolicy` is specified, it defaults to `Delete`. 
+
+- `Delete`: When a PVC is deleted or a CKS cluster (Managed Kubernetes Cluster in CloudStack) is deleted, the associated persistent volumes and their underlying CloudStack disk volumes will be automatically removed.
+- `Retain`: Persistent volumes and their underlying CloudStack disk volumes will be preserved even after PVC deletion or cluster deletion, allowing for manual recovery or data preservation.
+
 #### Using cloudstack-csi-sc-syncer
 
 The tool `cloudstack-csi-sc-syncer` may also be used to synchronize CloudStack
 disk offerings to Kubernetes storage classes.
 
 [More info...](./cmd/cloudstack-csi-sc-syncer/README.md)
 
+> **Note:** The VolumeSnapshot CRDs (CustomResourceDefinitions) of version 8.3.0 are installed in this deployment. If you use a different version, please ensure compatibility with your Kubernetes cluster and CSI sidecars.
+
+
+```
+kubectl apply -f https://raw.githubusercontent.com/kubernetes-csi/external-snapshotter/v8.3.0/client/config/crd/snapshot.storage.k8s.io_volumesnapshotclasses.yaml
+kubectl apply -f https://raw.githubusercontent.com/kubernetes-csi/external-snapshotter/v8.3.0/client/config/crd/snapshot.storage.k8s.io_volumesnapshots.yaml
+kubectl apply -f https://raw.githubusercontent.com/kubernetes-csi/external-snapshotter/v8.3.0/client/config/crd/snapshot.storage.k8s.io_volumesnapshotcontents.yaml
+
+```
+
 ### Usage
 
 Example:
@@ -106,6 +125,115 @@ To build the container images:
 make container
 ```
 
+
+## Volume Snapshots
+
+**NOTE:** To create volume snapshots in KVM, make sure to set the `kvm.snapshot.enabled` global setting to true and restart the Management Server
+
+### Volume snapshot creation
+For Volume snapshots to be created, the following configurations need to be applied:
+
+```
+kubectl apply -f deploy/k8s/00-snapshot-crds.yaml        # Installs the VolumeSnapshotClass, VolumeSnapshotContent and VolumeSnapshtot CRDs
+kubectl apply -f deploy/k8s/volume-snapshot-class.yaml   # Defines VolumeSnapshotClass for CloudStack CSI driver
+```
+
+Once the CRDs are installed, the snapshot can be taken by applying:
+```
+kubectl apply ./examples/k8s/snapshot/snapshot.yaml
+```
+
+In order to take the snapshot of a volume, `persistentVolumeClaimName` should be set to the right PVC name that is bound to the volume whose snapshot is to be taken.
+
+You can check CloudStack volume snapshots if the snapshot was successfully created. If for any reason there was an issue, it can be investgated by checking the logs of the cloudstack-csi-controller pods: cloudstack-csi-controller, csi-snapshotter and snapshot-controller containers
+
+```
+kubectl logs -f <cloudstack-csi-controller pod_name> -n kube-system # defaults to tailing logs of cloudstack-csi-controller
+kubectl logs -f <cloudstack-csi-controller pod_name> -n kube-system -c csi-snapshotter
+kubectl logs -f <cloudstack-csi-controller pod_name> -n kube-system -c snapshot-controller
+```
+
+### Restoring a Volume snapshot
+
+To restore a volume snapshot:
+1. Restore a snapshot and Use it in a pod
+* Create a PVC from the snapshot - for example ./examples/k8s/snapshot/pvc-from-snapshot.yaml
+* Apply the configuration:
+```
+kubectl apply -f ./examples/k8s/snapshot/pvc-from-snapshot.yaml
+```
+* Create a pod that uses the restored PVC; example pod config ./examples/k8s/snapshot/restore-pod.yaml
+```
+kubectl apply -f ./examples/k8s/snapshot/restore-pod.yaml
+```
+2. To restore a snapshot when using a deployment
+Update the deployment to point to the restored PVC
+
+```
+spec:
+  volumes:
+    - name: app-volume
+      persistentVolumeClaim:
+        claimName: pvc-from-snapshot
+```
+
+
+### Deletion of a volume snapshot
+
+To delete a volume snapshot
+One can simlpy delete the volume snapshot created in kubernetes using
+
+```
+kubectl delete volumesnapshot snapshot-1       # here, snapshot-1 is the name of the snapshot created
+```
+
+#### Troubleshooting issues with volume snapshot deletion
+If for whatever reason, snapshot deletion gets stuck, one can troubleshoot the issue doing the following:
+
+* Inspect the snapshot
+
+```
+kubectl get volumesnapshot <snapshot-name> [-n <namespace>] -o yaml
+```
+
+Look for the following section:
+```
+metadata:
+  finalizers:
+    - snapshot.storage.kubernetes.io/volumesnapshot-as-source
+```
+
+If finalizers are present, Kubernetes will not delete the resource until they are removed or resolved.
+
+* Patch to Remove Finalizers
+
+```
+kubectl patch volumesnapshot <snapshot-name> [-n <namespace>] --type=merge -p '{"metadata":{"finalizers":[]}}'
+```
+
+**Caution:** This bypasses cleanup logic. Use only if you're certain the snapshot is no longer needed at the CSI/backend level
+
+### What happens when you restore a volume from a snapshot
+* The CSI external-provisioner (a container in the cloudstack-csi-controller pod) sees the new PVC and notices it references a snapshot
+* The CSI driver's `CreateVolume` method is called with a `VolumeContentSource` that contains the snapshot ID
+* The CSI driver creates a new volume from the snapshot (using the CloudStack's createVolume API)
+* The new volume is now available as a PV (persistent volume) and is bound to the new PVC
+* The volume is NOT attached to any node just by restoring from a snapshot, the volume is only attached to a node when a Pod that uses the new PVC is scheduled on a node
+* The CSI driver's `ControllerPublishVolume` and `NodePublishVolume` methods are called to attach and mount the volume to the node where the Pod is running
+
+Hence to debug any issues during restoring a snapshot, check the logs of the cloudstack-csi-controller, external-provisioner containers 
+
+```
+kubectl logs -f <cloudstack-csi-controller pod_name> -n kube-system # defaults to tailing logs of cloudstack-csi-controller
+kubectl logs -f <cloudstack-csi-controller pod_name> -n kube-system -c external-provisioner
+```
+
+## Additional General Notes:
+
+**Node Scheduling Best Practices**: When deploying applications that require specific node placement, use `nodeSelector` or `nodeAffinity` instead of `nodeName`. The `nodeName` field bypasses the Kubernetes scheduler, which can cause issues with storage provisioning. When a StorageClass has `volumeBindingMode: WaitForFirstConsumer`, the CSI controller relies on scheduler decisions to properly bind PVCs. Using `nodeName` prevents this scheduling integration, potentially causing PVC binding failures.
+
+**Network CIDR Considerations**: When deploying CKS (CloudStack Kubernetes Service) clusters on pre-existing networks, avoid using the `10.0.0.0/16` CIDR range as it conflicts with Calico's default pod network configuration. This overlap can prevent proper CSI driver initialization and may cause networking issues within the cluster.
+
 ## See also
 
 - [CloudStack Kubernetes Provider](https://github.com/apache/cloudstack-kubernetes-provider) - Kubernetes Cloud Controller Manager for Apache CloudStack

cmd/cloudstack-csi-driver/Dockerfile

@@ -14,7 +14,9 @@ RUN apk add --no-cache \
     # blkid, mount and umount are required by k8s.io/mount-utils \
     blkid \
     mount \
-    umount
+    umount \
+    # Provides udevadm for device path detection \
+    udev
 
 COPY ./bin/cloudstack-csi-driver /cloudstack-csi-driver
-ENTRYPOINT ["/cloudstack-csi-driver"]
+ENTRYPOINT ["/cloudstack-csi-driver"]