Add container resource caps enforcement for workspace containers

[tolusha]

What does this PR do?

Implements ContainerResourceCaps configuration to enforce maximum resource limits and requests for workspace containers. When configured, container resource requirements that exceed the caps will be limited to the maximum values. This feature does not apply to initContainers or projectClone containers.

Assisted-by: Claude.

What issues does this PR fix or reference?

https://issues.redhat.com/browse/CRW-9546

Is it tested? How?

1. Create DevWorkspaceOperatorConfig

apiVersion: controller.devfile.io/v1alpha1
config:
  workspace:
    containerResourceCaps:
      limits:
        cpu: "1"
        memory: 2Gi
      requests:
        cpu: "0.1"
        memory: 100Mi
kind: DevWorkspaceOperatorConfig
metadata:
  name: devworkspace-operator-config
  namespace: devworkspace-controller

2. Start a workspace from devifile

3. Check containers resources in a workspace deployment:

• caps applied for tools-1 and tools-2
• no changes for tools-3

PR Checklist

• E2E tests pass (when PR is ready, comment /test v8-devworkspace-operator-e2e, v8-che-happy-path to trigger)
  • v8-devworkspace-operator-e2e: DevWorkspace e2e test
  • v8-che-happy-path: Happy path for verification integration with Che

[openshift-ci]

Skipping CI for Draft Pull Request.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with /test all

[dkwon17]

@tolusha thank you for the PR,

I noticed that if I set for example:

apiVersion: controller.devfile.io/v1alpha1
config:
  workspace:
    containerResourceCaps:
      limits:
        cpu: '102'
        memory: 200Gi
      requests:
        cpu: '102'
        memory: 200Mi
kind: DevWorkspaceOperatorConfig
metadata:
  name: devworkspace-operator-config
  namespace: openshift-operators

And create a devworkspace with no cpu limits or requests:

Devworkspace

apiVersion: workspace.devfile.io/v1alpha2
kind: DevWorkspace
metadata:
  name: devfile
  namespace: admin-devspaces
spec:
  started: true
  template:
    components:
      - container:
          image: 'quay.io/devfile/universal-developer-image:latest'
          memoryLimit: 5Gi
          memoryRequest: 3Gi
          sourceMapping: /projects
        name: tools-2

Then it looks like the cpu: 102 is being applied to the devworkspace:

'Error creating DevWorkspace deployment: Detected unrecoverable event FailedScheduling: 0/1 nodes are available: 1 Insufficient cpu. preemption: 0/1 nodes are available: 1 No preemption victims found for incoming pod.'

If I update the DWOC to set the cpu limit and request cap to 10, the devworkspace starts, and the container has a limit and request set to 10.

I think the cap should not be applied to the cpu in both cases?

[tolusha]

@dkwon17
From my understanding limits must be applied since there were not defined ones in the devworkspace.

This feature request proposes an enhancement to Red Hat Dev Spaces that allows Dev Spaces 
administrators to configure the CheCluster to ignore resource requests and limits specified 
in devfiles. Instead, the cluster should enforce standardized resource values for 
all workspaces at the cluster level. 

[rohanKanojia]

@tolusha : I have a minor question about the precedence order of these fields. I see there is an older field defaultContainerResources for specifying default resource requirements, which seems intended to define default CPU/memory values when a DevWorkspace doesn’t specify any (and mentions that no CPU is added by default).:

devworkspace-operator/apis/controller/v1alpha1/devworkspaceoperatorconfig_types.go

Lines 222 to 226 in b19af06

	// DefaultContainerResources defines the resource requirements (memory/cpu limit/request) used for
	// container components that do not define limits or requests. In order to not set a field by default,
	// the value "0" should be used. By default, the memory limit is 128Mi and the memory request is 64Mi.
	// No CPU limit or request is added by default.
	DefaultContainerResources *corev1.ResourceRequirements `json:"defaultContainerResources,omitempty"`

I was under the impression that containerResourceCaps were meant to act as upper bounds when resource values are defined in the DevWorkspace, rather than being applied when no CPU requests/limits are set at all. When I saw the CPU value being applied even with no CPU specified in the DevWorkspace, it made me wonder if I’m interpreting this incorrectly.

Could you please review if these expectations are correct?

DevWorkspace CPU specified	DWOC setting	Expected behavior (assumed)
No	containerResourceCaps.cpu = 102	CPU request/limit added to container as containerResourceCaps
Yes (lower than cap)	containerResourceCaps.cpu = 102	DevWorkspace CPU preserved
Yes (higher than cap)	containerResourceCaps.cpu = 10	CPU capped at 10
No	defaultContainerResources.cpu = 2	Default CPU request/limit set to 2
Yes	defaultContainerResources.cpu = 2	DevWorkspace CPU preserved
No	defaultContainerResources.cpu = 2, containerResourceCaps.cpu = 10	Default CPU applied (2), within cap
No	defaultContainerResources.cpu = 20, containerResourceCaps.cpu = 10	Default CPU capped to 10
Yes (cpu = 5)	defaultContainerResources.cpu = 2, containerResourceCaps.cpu = 10	DevWorkspace CPU preserved (within cap)
Yes (cpu = 20)	defaultContainerResources.cpu = 2, containerResourceCaps.cpu = 10	DevWorkspace CPU capped to 10

[tolusha]

Hello, @rohanKanojia
The table is correct.

[ibuziuk]

@tolusha, if a container does not specify limits / requests the maximums from DWOC will be used, right?
This is an important nuance that should be documented at the specification level, I believe.
@cgruver could you confirm that this is a requested behaviour?

[dkwon17]

Semantically when I hear ContainerResourceCaps, to me it sounds like only an upper limit, and not a default if limits / requests are not defined in the devfile.

If we want defaults applied when limits/requests are not defined in the devfile, IMHO the DefaultContainerResources should be the field for that

[dkwon17]

if a container does not specify limits / requests the maximums from DWOC will be used,

I would rather not have this functionality, since right now, DefaultContainerResources will be used instead which IMHO makes more sense

[tolusha]

@dkwon17
Updated

[ibuziuk]

@dkwon17 @tolusha folks, we need to have this PR merged for the upcoming release. If more time is required for review we should consider postpoing the upstream release.

[tolusha]

@tolusha, if a container does not specify limits / requests the maximums from DWOC will be used, right?
This is an important nuance that should be documented at the specification level, I believe.

@ibuziuk correct

[openshift-ci]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: dkwon17, rohanKanojia, tolusha

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details

Needs approval from an approver in each of these files:

• OWNERS [dkwon17]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[dkwon17]

/retest

[dkwon17]

/retest