Odo dev on podman on RHEL 8 fails with"error during container init: error setting cgroup config for procHooks process: cannot set memory limit: container could not join or create cgroup: OCI runtime error Error: failed to start 1 containers "

[sbouchet]

/kind bug

What versions of software are you using?

Operating System:
RHEL 8.7
podman 4.2.0 from https://podman.io/docs/installation#rhel8

Output of odo version:
odo v3.10.0 (039ef07)

How did you run odo exactly?

trying to run odo dev --platform podman --forward-localhost

Actual behavior

complete stack :

__
/  \__     Developing using the "quarkus" Devfile
\__/  \    Platform: podman
/  \__/    odo version: v3.9.0
\__/

↪ Running on podman in Dev mode
✗  Deploying pod [7s]
Cleaning up resources
✗  exit status 125: 
Complete Podman output:
Trying to pull registry.access.redhat.com/ubi8/openjdk-17:latest...
Getting image source signatures
Checking if image destination supports signatures
Copying blob sha256:3ba3333d1709318bb6e11b393259e7fb8977b0afb1489f4f030ea640ecf428e3
Copying blob sha256:06f86e50a0b74ff9eb161a7d781228877c90e8ff57e9689e8cb8b0f092a2a9f9
Copying config sha256:4d3db921ed0efa6fba3a14865cd9a17457693ad7d20c7701394c831358766acb
Writing manifest to image destination
Storing signatures
[error starting container 1a9615dc26bbb0707b904d042bfea48ca1b3ba65bd4e7ce6d753f47cdc59302d: runc: runc create failed: unable to start container process: error during container init: error setting cgroup config for procHooks process: cannot set memory limit: container could not join or create cgroup: OCI runtime error]
Pod:
9306644f27893a4596e40b48b3195077eca96ecb929dbd136e7d8ac63dfb9e19
Containers:
1a9615dc26bbb0707b904d042bfea48ca1b3ba65bd4e7ce6d753f47cdc59302d
7e586108cb49a6c9dfde44c99e9bb93f5d511b4a963ff5a835a7877f6d6f04b3

error starting container 1a9615dc26bbb0707b904d042bfea48ca1b3ba65bd4e7ce6d753f47cdc59302d: runc: runc create failed: unable to start container process: error during container init: error setting cgroup config for procHooks process: cannot set memory limit: container could not join or create cgroup: OCI runtime error

Error: failed to start 1 containers 

seems like cgroups is v1 and can't change that to cgroupsv2

Expected behavior

odo dev on podman shouldrun fine ( tested OK with same component on cluster )

Any logs, error output, etc?

output of podman info

podman info 
host:
  arch: amd64
  buildahVersion: 1.27.3
  cgroupControllers: []
  cgroupManager: cgroupfs
  cgroupVersion: v1
  conmon:
    package: conmon-2.1.4-1.module+el8.7.0+17824+66a0202b.x86_64
    path: /usr/bin/conmon
    version: 'conmon version 2.1.4, commit: 3af09f8ff9780bd806d2e2db2a8518250622d362'
  cpuUtilization:
    idlePercent: 90.92
    systemPercent: 1.9
    userPercent: 7.19
  cpus: 12
  distribution:
    distribution: '"rhel"'
    version: "8.7"
  eventLogger: file
  hostname: sbouchet.remote.csb
  idMappings:
    gidmap:
    - container_id: 0
      host_id: 116013
      size: 1
    - container_id: 1
      host_id: 165536
      size: 65536
    uidmap:
    - container_id: 0
      host_id: 116013
      size: 1
    - container_id: 1
      host_id: 165536
      size: 65536
  kernel: 4.18.0-425.19.2.el8_7.x86_64
  linkmode: dynamic
  logDriver: k8s-file
  memFree: 2182713344
  memTotal: 67073994752
  networkBackend: cni
  ociRuntime:
    name: runc
    package: runc-1.1.4-1.module+el8.7.0+17824+66a0202b.x86_64
    path: /usr/bin/runc
    version: |-
      runc version 1.1.4
      spec: 1.0.2-dev
      go: go1.18.4
      libseccomp: 2.5.2
  os: linux
  remoteSocket:
    exists: true
    path: /run/user/116013/podman/podman.sock
  security:
    apparmorEnabled: false
    capabilities: CAP_NET_RAW,CAP_CHOWN,CAP_DAC_OVERRIDE,CAP_FOWNER,CAP_FSETID,CAP_KILL,CAP_NET_BIND_SERVICE,CAP_SETFCAP,CAP_SETGID,CAP_SETPCAP,CAP_SETUID,CAP_SYS_CHROOT
    rootless: true
    seccompEnabled: true
    seccompProfilePath: /usr/share/containers/seccomp.json
    selinuxEnabled: true
  serviceIsRemote: false
  slirp4netns:
    executable: /usr/bin/slirp4netns
    package: slirp4netns-1.2.0-2.module+el8.7.0+17824+66a0202b.x86_64
    version: |-
      slirp4netns version 1.2.0
      commit: 656041d45cfca7a4176f6b7eed9e4fe6c11e8383
      libslirp: 4.4.0
      SLIRP_CONFIG_VERSION_MAX: 3
      libseccomp: 2.5.2
  swapFree: 33680519168
  swapTotal: 33705422848
  uptime: 47h 7m 13.00s (Approximately 1.96 days)
plugins:
  authorization: null
  log:
  - k8s-file
  - none
  - passthrough
  - journald
  network:
  - bridge
  - macvlan
  - ipvlan
  volume:
  - local
registries:
  search:
  - registry.access.redhat.com
  - registry.redhat.io
  - docker.io
store:
  configFile: /home/sbouchet/.config/containers/storage.conf
  containerStore:
    number: 1
    paused: 0
    running: 1
    stopped: 0
  graphDriverName: overlay
  graphOptions: {}
  graphRoot: /home/sbouchet/.local/share/containers/storage
  graphRootAllocated: 405821980672
  graphRootUsed: 235231731712
  graphStatus:
    Backing Filesystem: xfs
    Native Overlay Diff: "true"
    Supports d_type: "true"
    Using metacopy: "false"
  imageCopyTmpDir: /var/tmp
  imageStore:
    number: 35
  runRoot: /run/user/116013/containers
  volumePath: /home/sbouchet/.local/share/containers/storage/volumes
version:
  APIVersion: 4.2.0
  Built: 1673617532
  BuiltTime: Fri Jan 13 14:45:32 2023
  GitCommit: ""
  GoVersion: go1.18.4
  Os: linux
  OsArch: linux/amd64
  Version: 4.2.0

[feloy]

Podman installed with https://podman.io/docs/installation#rhel8

(https://redhat-internal.slack.com/archives/CHS495NE9/p1684309844011959?thread_ts=1684251738.302469&cid=CHS495NE9)

[feloy]

@sbouchet A first workaround is to remove the fields memoryLimit from the containers defined into the Devfile

We will work on not setting these values when cgroupv1 is used

[sbouchet]

Confirmed that removing that field on devfile and using odo 3.11 works great on my RHEL 8.6.