As an admin, I want to enable fuse overlay storage driver for all workspaces

[dkwon17]

Is your enhancement related to a problem? Please describe

As an admin, I want to enable fuse overlay storage driver for Podman in all workspaces by setting a field in the CheCluster CR.

Describe the solution you'd like

For example:

spec:
  devEnvironments:
    disableFuseOverlay: true/false

For using the fuse overlay storage driver for Podman in workspaces, we need:

1. The OpenShift cluster's CRI-O engine should allow pods to access the /dev/fuse device.

   • For OpenShift < 4.15, a MachineConfig needs to be created to edit a CRI-O config file to enable this. (See blog post)

2. Workspaces must have the io.kubernetes.crio-o.Devices: "/dev/fuse" annotation in order to access /dev/fuse

   • For OpenShift < 4.15, an additional annotation is needed. (See blog post)

3. For images based on UDI, Podman storage driver must be set to overlay

   • This is because the UDI image has the vfs storage driver set in the dockerfile:
     https://github.com/devfile/developer-images/blob/7e57123d72a7dcf183ce973a46fb6087f4dbeba2/universal/ubi8/Dockerfile#L233-L234
   • The /home/user/.config/containers/storage.conf file must be changed to:

[storage]
- driver = "vfs"
+ driver = "overlay"

+ [storage.options.overlay]
+ mount_program="/usr/bin/fuse-overlayfs"

Describe alternatives you've considered

No response

Additional context

No response

[dkwon17]

To enable fuse for a workspace, there are three requirements:

1. For OpenShift versions older than 4.15, the administrator has enabled /dev/fuse access on the cluster by following Configuring fuse-overlayfs
2. The storage.conf file in the workspace container has been configured to use fuse-overlayfs. See Enabling fuse-overlayfs with a Configmap.
3. The workspace has the necessary annotations for using the /dev/fuse device. See Accessing /dev/fuse from a workspace.

For this feature I believe we can assume that the admin has already completed the first requirement.

The idea I have for how this feature can be implemented is:

If spec.devEnvironments.disableFuseOverlay = false, have the che-operator create this configmap in the Eclipse Che namespace to mount the new storage.conf file to all user workspaces:

kind: ConfigMap
apiVersion: v1
metadata:
  name: overlayfs-storageconf
  namespace: eclipse-che
  labels:
    app.kubernetes.io/part-of: che.eclipse.org
    app.kubernetes.io/component: workspaces-config
  annotations:
    controller.devfile.io/mount-as: file
    controller.devfile.io/mount-path: /home/user/.config/containers
data:
  storage.conf: |
    [storage]
    driver = "overlay"

    [storage.options.overlay]
    mount_program="/usr/bin/fuse-overlayfs"

This configmap satisfies the second requirement.

There is no easy way at the moment to satisfy the third requirement, because there is no easy way for an admin specify necessary spec.template.attributes for all workspaces in the cluster.

A possible solution would be to introduce a field in the DWOC for adding annotations (or maybe even pod or container overrides) for all workspaces, and have che-operator edit the DWOC (similar to what's happening in dev_workspace_config.go.

Any thoughts @AObuchow @tolusha @ibuziuk ?

[tolusha]

Having this feature [1], admin is able to mount storage.conf for all users workspaces.
Maybe we can mention this in the doc only.

[1] https://eclipse.dev/che/docs/stable/administration-guide/configuring-a-user-namespace/

[dkwon17]

@tolusha

Sounds good, I think it makes most sense for the admin to create the configmap for tor the storage.conf file.

So, to enable fuse for all workspaces, instead of introducing:

spec:
  devEnvironments:
    disableFuseOverlay: true/false

we can introduce:

spec:
  devEnvironments:
    annotations: (extra annotations for all workspaces)

And document that the admin must create the storage.conf configmap and set:

spec.devEnvironments.annotations = ["io.kubernetes.crio-o.Devices: "/dev/fuse"]

This method would allow admins to make additional changes to storage.conf if needed, by just editing the configmap

[ibuziuk]

Closing since it is in 7.88.0 RN and I believe all PRs have been merged