oci-image-tool: error extracting layer: symlink /lib/systemd/systemd tmp2/bin/systemd: file exists

[dohnto]

Hello, I am trying to examine a docker images without a docker tool. I have created a simple smallest docker image, where I encounter following error. The image can be found in docker repository: dohnto/debian-skopeo. The dockerfile is following

FROM debian:8.3
RUN apt-get update -y && \
    apt-get upgrade -y 

Note, that during building such an image, a systemd will be upgraded:

...
The following packages will be upgraded:
  apt base-files dmsetup dpkg e2fslibs e2fsprogs gnupg gpgv libapt-pkg4.12
  libc-bin libc6 libcomerr2 libdevmapper1.02.1 libgcrypt20 libpam-modules
  libpam-modules-bin libpam0g libpcre3 libss2 libsystemd0 libudev1
  multiarch-support perl-base systemd systemd-sysv tzdata udev
27 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
...

My workflow in further examination is following:

$ skopeo --version
skopeo version 0.1.17-dev commit: 507d09876d0620d46de7c8c20eb9b58d2dffa482
$ skopeo copy docker://dohnto/debian-skopeo oci:tmp
$ oci-image-tool validate tmp
reference "latest": OK
tmp: OK
$ oci-image-tool unpack --ref latest tmp tmp2
unpacking failed: error extracting layer: symlink /lib/systemd/systemd tmp2/bin/systemd: file exists
$ ls -l tmp2/bin/systemd
lrwxrwxrwx 1 tom tom 20 15. říj 16.38 tmp2/bin/systemd -> /lib/systemd/systemd

I expected this to work and the oci-image-tool to actually unpack the oci image into a folder I could chroot. Thank you for any help or workaround how to solve this issue.

Version information:

• OS: fedora23 workstation, kernel 4.7.3-100.fc23.x86_64
• running under systemd
• skopeo: 0.1.17-dev commit: 507d09876d0620d46de7c8c20eb9b58d2dffa482
• oci-image-tool: build from source: v0.5.0-63-g32a3cb9

[wking]

On Sat, Oct 15, 2016 at 07:53:22AM -0700, Tomas Dohnalek wrote:

$ oci-image-tool unpack --ref latest tmp tmp2
unpacking failed: error extracting layer: symlink /lib/systemd/systemd tmp2/bin/systemd: file exists

Maybe you have multiple layers in the OCI image and two of them have
entries for /bin/systemd? I expect unpackLayer should be stating the
target path and removing (recursively?) existing entries before
attemping to create the new entry. There's more explicit wording
around this idea in-flight in 1.

If you need a hack to work around this before the spec and tools are
updated, you can probably drop an:

os.RemoveAll(path)

in around 2.

Cheers,
Trevor

This email may be signed or encrypted with GnuPG (http://www.gnupg.org).
For more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy

[dohnto]

@wking ,thank you for hint where to search for the problem. I have managed to go further by putting

if _, err := os.Stat(path); !os.IsNotExist(err) {
    os.RemoveAll(path)
}

in here [1]. Now it fails on ../modules tmp2/etc/modules-load.d/modules.conf which does not seem to pass the condition.

unpacking failed: error extracting layer: symlink ../modules tmp2/etc/modules-load.d/modules.conf: file exists

Maybe the problem is in relative link:

$ docker run -it dohnto/debian-skopeo  ls -l /etc/modules-load.d/modules.conf
lrwxrwxrwx 1 root root 10 Sep  2 22:02 /etc/modules-load.d/modules.conf -> ../modules

[1]

image-tools/image/manifest.go

Line 235 in f24d27b

[wking]

On Mon, Oct 17, 2016 at 11:02:45AM -0700, Tomas Dohnalek wrote:

@wking ,thank you for hint where to search for the problem. I have managed to go further by putting

if _, err := os.Stat(path); !os.IsNotExist(err) {
    os.RemoveAll(path)
}

You don't want to be following links. Use Lstat, or just skip the
stat, call os.RemoveAll, and ignore does-not-exist errors it may
return. The leading stat is racy against parallel removals, so we
should be ignoring does-not-exist anyway. And once that's in place, I
don't see a point to the stat guard.

[dohnto]

Ok, I removed the checks and put the removal in here [1] and here [2], because file

$ docker run -it dohnto/debian-skopeo  ls -l /usr/bin/perl5.20.2
-rwxr-xr-x 2 root root 10416 Jul 22 15:59 /usr/bin/perl5.20.2

file /usr/bin/perl5.20.2 is a link. Now I came across this error:

$ ./oci-unpack --ref latest tmp tmp2 
unpacking failed: error extracting layer: unable to open file: open tmp2/usr/share/zoneinfo/Asia/Kolkata: too many levels of symbolic links

$ docker run -it dohnto/debian-skopeo  ls -l /usr/share/zoneinfo/Asia/Kolkata
-rw-r--r-- 1 root root 265 Jul  5 18:15 /usr/share/zoneinfo/Asia/Kolkata

Does not seem to be link at all so I am lost again.

But

[1]

image-tools/image/manifest.go

Line 235 in f24d27b

[2]

image-tools/image/manifest.go

Line 224 in f24d27b

[wking]

On Tue, Oct 18, 2016 at 01:04:35AM -0700, Tomas Dohnalek wrote:

Ok, I removed the checks and put the removal in here 1 and here 2, …

I'm not sure what's going on with your symlink issue, but I don't
think you should be putting removal in the link and symlink blocks.
The idea is that a non-directory tar entry should clobber any previous
content at that path. It doesn't have anything to do with links.
I've filed #42 applying the logic that's in-flight with
opencontainers/image-spec#317 if you want to give that a shot.

[dohnto]

Your pull request #42 seem to solve my problem completely. Thank you.