Describe the feature you'd like to have

i want some global global ext4 mount default options, not those mentioned in storageclass - there is couple problems current situation has

• errors=remount-ro is NOT default, default is errors=continue - this is dangerous, when some network happens, rbd looses some writes, reports it back to filesystem, but it swallows it and does nothing (just reports something to logs, but introduces inconsistency in filesystem)

• mount options could be provided via storageclass, but this is not userfriendly for some options you want to always have present - when i introduce the option mentioned earlier (errors=remount-ro) this is applied only to new pvc/pv, because pv definition get these options in its definition - this is not looked up from the storageclass at the mount moment. this means patching all existing PVs with this new option.

• running e2fsck on the volumes is complicated. there is no way to just ask for full check by some annotation or pv option. this takes us to manual e2fsck, getting workload down and ... this not nice to do.

• when security measures are in place (i mean PSA, limited possibility to use privileged containers), you could end up with cluster, where only specific namespaces could have privileged pods) its not easy to to just run privileged container that in namespace where the pvc is defined, use block mode for some debug pod, that will overtake the pvc (get the exclusive access to it for RWO volumes) and block the original workload to get in place because of some automation tries do its work and start the workload again.

• in the end, errors=remount-ro should be the default, data safety in first place, better to get the workload chance to recognize volume readonly problem, than let it run in inconsistent state

What is the value to the end user? (why is it a priority?)

• better safety for the ext4 filesystems in case of some network connection problems, not introducing corrupted volumes

• in case of support for hint to run e2fsck - possibility to get full check of pvc filesystem in case something is wrong.

How will we know we have a good solution? (acceptance criteria)

when filesystem detects some consistency problem, it stops writes that could do even more harm

Additional context

maybe some documentation how to safely do filesystem checks would be nice

i could share some parts of my current state of steps to do the e2fsck, needs to be adopted for use by someone else...
(im used to these oldstyle grep things and not using fancy label/jq queries, but this is detail for now ;)

(i do not mention couple of snippets to get the rbd volume name based on workload pod or pvc)

get ceph-csi-rbd nodeplugin pod on the same node as the workload already using the pv
... doing it on the same node was some part of my previous workflow and maybe is not the most important here when descalilng the workload to 0

kubectl -n ls-system get pod -o wide | grep $POD_NODE | grep ceph-csi-ceph-csi-rbd-nodeplugin-

... you get NODEPLUGIN_POD for later use

when you exec into ceph-csi-ceph-csi-rbd-nodeplugin-xxx...

kubectl -n ls-system exec -ti $NODEPLUGIN_POD csi-rbdplugin -- /bin/bash
...

you could prepare env for runing rbd commands requiring some auth...

CACERT="/run/secrets/kubernetes.io/serviceaccount/ca.crt"
TOKEN=$(cat /run/secrets/kubernetes.io/serviceaccount/token)
APISERVER="https://${KUBERNETES_SERVICE_HOST}:${KUBERNETES_SERVICE_PORT}"
CEPH_USER=$(curl -s --cacert $CACERT   -H "Authorization: Bearer $TOKEN"   "$APISERVER/api/v1/namespaces/ls-system/secrets/ceph-csi-credentials" | jq '.data["userID"]' -r | base64 -d)
echo $CEPH_USER 
CEPH_TOKEN=$(curl -s --cacert $CACERT   -H "Authorization: Bearer $TOKEN"   "$APISERVER/api/v1/namespaces/ls-system/secrets/ceph-csi-credentials" | jq '.data["userKey"]' -r | base64 -d)
CEPH_MONS=$(jq -r '.[0].monitors | join(",")' /etc/ceph-csi-config/config.json )
echo $CEPH_MONS 
CEPH_MON=$( echo $CEPH_MONS | sed -e 's/,.*/:6789/g' )
echo $CEPH_MON
RBD_ARGS="--id $CEPH_USER --key $CEPH_TOKEN --mon-host $CEPH_MON"
rbd showmapped

then you could do the downscale of the workload, and later do the e2fsck, using first part with showmounted serves as safety to see the volume was realy unmapped (and you accidentaly do not hit some other volume ;o)

C=mycluster
V=csi-vol-$V

###

RBDDEV=""
rbd showmapped | grep -i "$V"

rbd showmapped
rbd showmapped | grep -i "$V"
rbd status $C/$V $RBD_ARGS
rbd info $C/$V $RBD_ARGS

RBDDEV=$(rbd map $C/$V $RBD_ARGS)
echo $RBDDEV

e2fsck -f -y -v $RBDDEV

rbd unmap $RBDDEV
rbd showmapped | grep -i "$V"