Description
openedon Aug 28, 2023
Issue Description
The issue we are facing is that when we load podman images that are compressed .tar.xz. These .tar.xz files have multiple podman images in them. When we load this file, it takes several minutes to complete the load. The first 4 minutes are it just sitting there, the last 30 seconds is work actually being done as we see the images being loaded.
The reason for this bug is that docker (via docker images, not podman) loads its version of the compressed .tar.xz file in like 30 seconds.
4m30s vs 30s.
Why is Podman taking so long? And yes, both files are roughly the same size.
I have tried this on my workstation (I9 9900K) and home VM (Ryzen 7 7700X). Both are using Fedora33 VM with 4 CPUs. So yeah I could increase VM specs, but that wouldn't be realistic to what I have at work. Also wouldn't solve the issue as Docker tar.xz loads just fine.
Tried on Podman 2.1.1 and 3.3.1.
Steps to reproduce the issue
Steps to reproduce the issue
- Load Fedora 33 VM
- Install 3.3.1 Podman
- sudo dnf install podman-docker-3:3.3.1-1.fc33.noarch
- Load public images to simulate number and size of work images
- sudo podman pull cockroachdb/cockroach:latest
- sudo podman pull minio/minio:RELEASE.2023-08-23T10-07-06Z.fips
- sudo podman pull minio/mc:latest
- sudo podman pull alpine:latest
- sudo podman pull busybox:latest
- sudo podman pull docker:latest
- sudo podman pull rabbitmq:latest
- Save multiple podman images to singular compressed .tar.xz. This will take 5-10 minutes.
sudo podman save -m \
docker.io/library/rabbitmq:latest \
docker.io/minio/mc:latest \
docker.io/minio/minio:RELEASE.2023-08-23T10-07-06Z.fips \
docker.io/library/alpine:latest \
docker.io/cockroachdb/cockroach:latest \
docker.io/library/docker:latest \
docker.io/library/busybox:latest | xz > ImagesPodmanCompressed.tar.xz - Clear out podman images so you can then load this file again.
- sudo podman rmi $(sudo podman images -q)
- Load this ImagesPodmanCompressed.tar.xz file. Notice how it just sits there for several minutes (BAD) before it begins work.
- time sudo podman load -i ImagesPodmanCompressed.tar.xz
Describe the results you received
Notice how it just sits there for several minutes (BAD) before it begins work. It takes 4m30s for Podman to load this type of file but it only takes Docker 30s.
Describe the results you expected
Images should start to load immediately. This is the behavior of Docker for this same type of singular compressed .tar.zx.
podman info output
host:
arch: amd64
buildahVersion: 1.22.3
cgroupControllers: []
cgroupManager: systemd
cgroupVersion: v2
conmon:
package: conmon-2.0.27-2.fc33.x86_64
path: /usr/bin/conmon
version: 'conmon version 2.0.27, commit: '
cpus: 4
distribution:
distribution: fedora
version: "33"
eventLogger: journald
hostname: localhost.localdomain
idMappings:
gidmap:
- container_id: 0
host_id: 1000
size: 1
- container_id: 1
host_id: 100000
size: 65536
uidmap:
- container_id: 0
host_id: 1000
size: 1
- container_id: 1
host_id: 100000
size: 65536
kernel: 5.8.15-301.fc33.x86_64
linkmode: dynamic
memFree: 3793604608
memTotal: 12334272512
ociRuntime:
name: crun
package: crun-0.15-5.fc33.x86_64
path: /usr/bin/crun
version: |-
crun version 0.15
commit: 56ca95e61639510c7dbd39ff512f80f626404969
spec: 1.0.0
+SYSTEMD +SELINUX +APPARMOR +CAP +SECCOMP +EBPF +YAJL
os: linux
remoteSocket:
path: /run/user/1000/podman/podman.sock
security:
apparmorEnabled: false
capabilities: AP_CHOWN,CAP_DAC_OVERRIDE,CAP_FOWNER,CAP_FSETID,CAP_KILL,CAP_NET_BIND_SERVICE,CAP_SETFCAP,CAP_SETGID,CAPSETPCAP,CAP_SETUID,CAP_SYS_CHROOT
rootless: true
seccompEnabled: true
seccompProfilePath: /usr/share/containers/seccomp.json
selinuxEnabled: true
serviceIsRemote: false
slirp4netns:
executable: /usr/bin/slirp4netns
package: slirp4netns-1.1.4-4.dev.giteecccdb.fc33.x86_64
version: |-
slirp4netns version 1.1.4+dev
commit: eecccdb96f587b11d7764556ffacfeaffe4b6e11
libslirp: 4.3.1
SLIRP_CONFIG_VERSION_MAX: 3
libseccomp: 2.5.0
swapFree: 4294963200
swapTotal: 4294963200
uptime: 2h 45m 30.77s (Approximately 0.08 days)
registries:
search:
- registry.fedoraproject.org
- registry.access.redhat.com
- docker.io
store:
configFile: /home/user/.config/containers/storage.conf
containerStore:
number: 0
paused: 0
running: 0
stopped: 0
graphDriverName: overlay
graphOptions:
overlay.mount_program:
Executable: /usr/bin/fuse-overlayfs
Package: fuse-overlayfs-1.1.2-1.fc33.x86_64
Version: |-
fusermount3 version: 3.9.3
fuse-overlayfs: version 1.1.0
FUSE library version 3.9.3
using FUSE kernel interface version 7.31
graphRoot: /home/user/.local/share/containers/storage
graphStatus:
Backing Filesystem: btrfs
Native Overlay Diff: "false"
Supports d_type: "true"
Using metacopy: "false"
imageStore:
number: 0
runRoot: /run/user/1000/containers
volumePath: /home/user/.local/share/containers/storage/volumes
version:
APIVersion: 3.3.1
Built: 1630356406
BuiltTime: Mon Aug 30 16:46:46 2021
GitCommit: ""
GoVersion: go1.15.14
OsArch: linux/amd64
Version: 3.3.1Podman in a container
No
Privileged Or Rootless
None
Upstream Latest Release
No
Additional environment details
Workstation:
I9 9900K
16 GB ram
Windows 10
Oracle VM VirtualBox, Fedora 33, 4cpu/8gb ram
Podman version 2.1.1
File takes about 4m30s to load
Home PC:
Ryzen 7 7700X
32GB ram
Windows 10
Oracle VM VirtualBox, Fedora 33, 4cpu/16gb ram
Podman version 3.3.1
File takes about 3min to load
Additional information
None