memory.numa_stat not found

[dashpole]

I was trying build image using dockerfile in deploy/Dockerfile .. i was facing error failed to get container "/" with error: unable to find data in memory cache

Failed to update stats for container "/": open /sys/fs/cgroup/memory/memory.numa_stat: no such file or directory

@iwankgb

[dashpole]

Probably due to opencontainers/runc#2278 after the recent runc update

[vynu]

container run time log:

W0612 01:32:29.038910 1 nvidia.go:61] NVIDIA GPU metrics will not be available: no NVIDIA devices found
W0612 01:32:29.046539 1 sysinfo.go:203] Nodes topology is not available, providing CPU topology
W0612 01:32:29.127079 1 container.go:545] Failed to update stats for container "/": open /sys/fs/cgroup/memory/memory.numa_stat: no such file or directory, continuing to push stats
W0612 01:32:29.138688 1 container.go:545] Failed to update stats for container "/kubepods": open /sys/fs/cgroup/memory/kubepods/memory.numa_stat: no such file or directory, continuing to push stats
W0612 01:32:29.141693 1 container.go:545] Failed to update stats for container "/docker": open /sys/fs/cgroup/memory/docker/memory.numa_stat: no such file or directory, continuing to push stats
W0612 01:32:29.146003 1 container.go:545] Failed to update stats for container "/004-format": open /sys/fs/cgroup/memory/004-format/memory.numa_stat: no such file or directory, continuing to push stats
W0612 01:32:29.150515 1 container.go:545] Failed to update stats for container "/systemreserved": open /sys/fs/cgroup/memory/systemreserved/memory.numa_stat: no such file or directory, continuing to push stats
W0612 01:32:29.162792 1 container.go:545] Failed to update stats for container "/systemreserved/acpid": open /sys/fs/cgroup/memory/systemreserved/acpid/memory.numa_stat: no such file or directory, continuing to push stats
W0612 01:32:29.164904 1 container.go:545] Failed to update stats for container "/010-bridge": open /sys/fs/cgroup/memory/010-bridge/memory.numa_stat: no such file or directory, continuing to push stats
W0612 01:32:29.174271 1 container.go:545] Failed to update stats for container "/011-ip": open /sys/fs/cgroup/memory/011-ip/memory.numa_stat: no such file or directory, continuing to push stats
W0612 01:32:29.182247 1 container.go:545] Failed to update stats for container "/docker/152573d89df5b08b62746139d5575b1fe1986b30aff90174a35739f9c58b2e14": open /sys/fs/cgroup/memory/docker/152573d89df5b08b62746139d5575b1fe1986b30aff90174a35739f9c58b2e14/memory.numa_stat: no such file or directory, continuing to push stats
W0612 01:32:29.187066 1 container.go:545] Failed to update stats for container "/systemreserved/diagnose": open /sys/fs/cgroup/memory/systemreserved/diagnose/memory.numa_stat: no such file or directory, continuing to push stats
W0612 01:32:29.190670 1 container.go:545] Failed to update stats for container "/009-mount-kube-images": open /sys/fs/cgroup/memory/009-mount-kube-images/memory.numa_stat: no such file or directory, continuing to push stats
W0612 01:32:29.193351 1 container.go:545] Failed to update stats for container "/systemreserved/ntpd": open /sys/fs/cgroup/memory/systemreserved/ntpd/memory.numa_stat: no such file or directory, continuing to push stats
W0612 01:32:29.196571 1 container.go:545] Failed to update stats for container "/systemreserved/socks": open /sys/fs/cgroup/memory/systemreserved/socks/memory.numa_stat: no such file or directory, continuing to push stats
W0612 01:32:29.198613 1 container.go:545] Failed to update stats for container "/systemreserved/trim-after-delete": open /sys/fs/cgroup/memory/systemreserved/trim-after-delete/memory.numa_stat: no such file or directory, continuing to push stats
W0612 01:32:29.200099 1 container.go:545] Failed to update stats for container "/kmsg": open /sys/fs/cgroup/memory/kmsg/memory.numa_stat: no such file or directory, continuing to push stats
W0612 01:32:29.204176 1 container.go:545] Failed to update stats for container "/systemreserved/vpnkit-forwarder": open /sys/fs/cgroup/memory/systemreserved/vpnkit-forwarder/memory.numa_stat: no such file or directory, continuing to push stats
W0612 01:32:29.213973 1 container.go:545] Failed to update stats for container "/008-mount-docker": open /sys/fs/cgroup/memory/008-mount-docker/memory.numa_stat: no such file or directory, continuing to push stats
W0612 01:32:29.215844 1 container.go:545] Failed to update stats for container "/podruntime": open /sys/fs/cgroup/memory/podruntime/memory.numa_stat: no such file or directory, continuing to push stats
W0612 01:32:29.217798 1 container.go:545] Failed to update stats for container "/001-sysfs": open /sys/fs/cgroup/memory/001-sysfs/memory.numa_stat: no such file or directory, continuing to push stats
W0612 01:32:29.219255 1 container.go:545] Failed to update stats for container "/podruntime/docker": open /sys/fs/cgroup/memory/podruntime/docker/memory.numa_stat: no such file or directory, continuing to push stats
W0612 01:32:29.220127 1 container.go:545] Failed to update stats for container "/003-sysctl": open /sys/fs/cgroup/memory/003-sysctl/memory.numa_stat: no such file or directory, continuing to push stats
W0612 01:32:29.222662 1 container.go:545] Failed to update stats for container "/006-mount": open /sys/fs/cgroup/memory/006-mount/memory.numa_stat: no such file or directory, continuing to push stats
W0612 01:32:29.226720 1 container.go:545] Failed to update stats for container "/systemreserved/host-timesync-daemon": open /sys/fs/cgroup/memory/systemreserved/host-timesync-daemon/memory.numa_stat: no such file or directory, continuing to push stats
W0612 01:32:29.229358 1 container.go:545] Failed to update stats for container "/007-swap": open /sys/fs/cgroup/memory/007-swap/memory.numa_stat: no such file or directory, continuing to push stats
W0612 01:32:29.231023 1 container.go:545] Failed to update stats for container "/000-metadata": open /sys/fs/cgroup/memory/000-metadata/memory.numa_stat: no such file or directory, continuing to push stats
W0612 01:32:29.232634 1 container.go:545] Failed to update stats for container "/002-binfmt": open /sys/fs/cgroup/memory/002-binfmt/memory.numa_stat: no such file or directory, continuing to push stats
W0612 01:32:29.234710 1 container.go:545] Failed to update stats for container "/005-extend": open /sys/fs/cgroup/memory/005-extend/memory.numa_stat: no such file or directory, continuing to push stats
W0612 01:33:29.273972 1 container.go:545] Failed to update stats for container "/docker": open /sys/fs/cgroup/memory/docker/memory.numa_stat: no such file or directory, continuing to push stats

[vynu]

localhost:8080/containers/ ERROR

failed to get container "/" with error: unable to find data in memory cache

[vynu]

contents of /sys/fs/cgroup/memory

/ # cd /sys/fs/cgroup/memory
/sys/fs/cgroup/memory # ls -ls
total 0
0 drwxr-xr-x 2 root root 0 Jun 12 01:30 000-metadata
0 drwxr-xr-x 2 root root 0 Jun 12 01:30 001-sysfs
0 drwxr-xr-x 2 root root 0 Jun 12 01:30 002-binfmt
0 drwxr-xr-x 2 root root 0 Jun 12 01:30 003-sysctl
0 drwxr-xr-x 2 root root 0 Jun 12 01:30 004-format
0 drwxr-xr-x 2 root root 0 Jun 12 01:30 005-extend
0 drwxr-xr-x 2 root root 0 Jun 12 01:30 006-mount
0 drwxr-xr-x 2 root root 0 Jun 12 01:30 007-swap
0 drwxr-xr-x 2 root root 0 Jun 12 01:30 008-mount-docker
0 drwxr-xr-x 2 root root 0 Jun 12 01:30 009-mount-kube-images
0 drwxr-xr-x 2 root root 0 Jun 12 01:30 010-bridge
0 drwxr-xr-x 2 root root 0 Jun 12 01:30 011-ip
0 -rw-r--r-- 1 root root 0 Jun 12 01:32 cgroup.clone_children
0 --w--w--w- 1 root root 0 Jun 12 01:32 cgroup.event_control
0 -rw-r--r-- 1 root root 0 Jun 12 01:32 cgroup.procs
0 -r--r--r-- 1 root root 0 Jun 12 01:32 cgroup.sane_behavior
0 drwxr-xr-x 3 root root 0 Jun 12 01:32 docker
0 drwxr-xr-x 2 root root 0 Jun 12 01:30 kmsg
0 drwxr-xr-x 2 root root 0 Jun 12 01:32 kubepods
0 -rw-r--r-- 1 root root 0 Jun 12 01:32 memory.failcnt
0 --w------- 1 root root 0 Jun 12 01:32 memory.force_empty
0 -rw-r--r-- 1 root root 0 Jun 12 01:32 memory.kmem.failcnt
0 -rw-r--r-- 1 root root 0 Jun 12 01:30 memory.kmem.limit_in_bytes
0 -rw-r--r-- 1 root root 0 Jun 12 01:32 memory.kmem.max_usage_in_bytes
0 -r--r--r-- 1 root root 0 Jun 12 01:32 memory.kmem.slabinfo
0 -rw-r--r-- 1 root root 0 Jun 12 01:32 memory.kmem.tcp.failcnt
0 -rw-r--r-- 1 root root 0 Jun 12 01:30 memory.kmem.tcp.limit_in_bytes
0 -rw-r--r-- 1 root root 0 Jun 12 01:32 memory.kmem.tcp.max_usage_in_bytes
0 -r--r--r-- 1 root root 0 Jun 12 01:32 memory.kmem.tcp.usage_in_bytes
0 -r--r--r-- 1 root root 0 Jun 12 01:32 memory.kmem.usage_in_bytes
0 -rw-r--r-- 1 root root 0 Jun 12 01:32 memory.limit_in_bytes
0 -rw-r--r-- 1 root root 0 Jun 12 01:32 memory.max_usage_in_bytes
0 -rw-r--r-- 1 root root 0 Jun 12 01:32 memory.memsw.failcnt
0 -rw-r--r-- 1 root root 0 Jun 12 01:30 memory.memsw.limit_in_bytes
0 -rw-r--r-- 1 root root 0 Jun 12 01:32 memory.memsw.max_usage_in_bytes
0 -r--r--r-- 1 root root 0 Jun 12 01:32 memory.memsw.usage_in_bytes
0 -rw-r--r-- 1 root root 0 Jun 12 01:32 memory.move_charge_at_immigrate
0 -rw-r--r-- 1 root root 0 Jun 12 01:30 memory.oom_control
0 ---------- 1 root root 0 Jun 12 01:32 memory.pressure_level
0 -rw-r--r-- 1 root root 0 Jun 12 01:30 memory.soft_limit_in_bytes
0 -r--r--r-- 1 root root 0 Jun 12 01:32 memory.stat
0 -rw-r--r-- 1 root root 0 Jun 12 01:30 memory.swappiness
0 -r--r--r-- 1 root root 0 Jun 12 01:32 memory.usage_in_bytes
0 -rw-r--r-- 1 root root 0 Jun 12 01:30 memory.use_hierarchy
0 -rw-r--r-- 1 root root 0 Jun 12 01:32 notify_on_release
0 drwxr-xr-x 3 root root 0 Jun 12 01:30 podruntime
0 -rw-r--r-- 1 root root 0 Jun 12 01:32 release_agent
0 drwxr-xr-x 9 root root 0 Jun 12 01:30 systemreserved
0 -rw-r--r-- 1 root root 0 Jun 12 01:32 tasks
/sys/fs/cgroup/memory #

[vynu]

/sys/fs/cgroup/memory # cat /etc/os-release
NAME="Alpine Linux"
ID=alpine
VERSION_ID=3.12.0
PRETTY_NAME="Alpine Linux v3.12"
HOME_URL="https://alpinelinux.org/"
BUG_REPORT_URL="https://bugs.alpinelinux.org/"
/sys/fs/cgroup/memory # lsb_release -sirc
sh: lsb_release: not found
/sys/fs/cgroup/memory # uname -sr
Linux 4.19.76-linuxkit
/sys/fs/cgroup/memory #

[vynu]

/proc/cpuinfo

/sys/fs/cgroup/memory # cat /proc/cpuinfo
processor : 0
vendor_id : GenuineIntel
cpu family : 6
model : 78
model name : Intel(R) Core(TM) i5-6267U CPU @ 2.90GHz
stepping : 3
cpu MHz : 2900.000
cache size : 4096 KB
physical id : 0
siblings : 1
core id : 0
cpu cores : 1
apicid : 0
initial apicid : 0
fpu : yes
fpu_exception : yes
cpuid level : 22
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ss ht pbe syscall nx pdpe1gb lm constant_tsc rep_good nopl xtopology nonstop_tsc cpuid tsc_known_freq pni pclmulqdq dtes64 ds_cpl ssse3 sdbg fma cx16 xtpr pcid sse4_1 sse4_2 movbe popcnt aes xsave avx f16c rdrand hypervisor lahf_lm abm 3dnowprefetch pti fsgsbase bmi1 avx2 bmi2 erms xsaveopt arat
bugs : cpu_meltdown spectre_v1 spectre_v2 spec_store_bypass l1tf mds swapgs
bogomips : 5808.00
clflush size : 64
cache_alignment : 64
address sizes : 39 bits physical, 48 bits virtual
power management:

processor : 1
vendor_id : GenuineIntel
cpu family : 6
model : 78
model name : Intel(R) Core(TM) i5-6267U CPU @ 2.90GHz
stepping : 3
cpu MHz : 2900.000
cache size : 4096 KB
physical id : 1
siblings : 1
core id : 0
cpu cores : 1
apicid : 1
initial apicid : 1
fpu : yes
fpu_exception : yes
cpuid level : 22
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ss ht pbe syscall nx pdpe1gb lm constant_tsc rep_good nopl xtopology nonstop_tsc cpuid tsc_known_freq pni pclmulqdq dtes64 ds_cpl ssse3 sdbg fma cx16 xtpr pcid sse4_1 sse4_2 movbe popcnt aes xsave avx f16c rdrand hypervisor lahf_lm abm 3dnowprefetch pti fsgsbase bmi1 avx2 bmi2 erms xsaveopt arat
bugs : cpu_meltdown spectre_v1 spectre_v2 spec_store_bypass l1tf mds swapgs
bogomips : 5801.14
clflush size : 64
cache_alignment : 64
address sizes : 39 bits physical, 48 bits virtual

[katarzyna-z]

Fix to runc: opencontainers/runc#2470