Description
openedon Apr 6, 2021
Hi there! My project has code to unpack an OCI image into a flat local rootfs directory, and I just noticed mutate.Extract exists for doing the same thing. I hoped to switch to it, but one of my tests started failing.
The current implementation of mutate.Extract unpacks the layers in reverse-order to avoid creating files which are later removed. I took this same approach initially and ran into the bug I'm raising here: if a target for a hard-link is created in one layer and then removed in a later layer, it fails to create any hard-links to it present in the earlier layers. (Something like that; sorry, it's been a while.)
My fix (concourse/registry-image-resource@ecf520b) was to un-optimize it and just unpack in the original order, removing files whenever it sees a whiteout. It was kind of a bummer.
This is low priority for me, just raising the issue in case someone else runs into the same error. This is a bit of an edge case so I'd understand if the performance trade-offs make it not worth fixing, but in my case I had to be compatible with everything; I'm happy to just keep maintaining my code for this. Feel free to just close this if you don't want to fix it.
Here's a Dockerfile that can demonstrate the issue:
FROM alpine
RUN apk --no-cache add git
RUN rm /usr/bin/gitA patch and test to run (in my project) to reproduce the error
Apply to registry-image resource:
diff --git a/commands/in.go b/commands/in.go
index b085c42..7fcc553 100644
--- a/commands/in.go
+++ b/commands/in.go
@@ -8,10 +8,12 @@ import (
"os"
"path/filepath"
+ "github.com/concourse/go-archive/tarfs"
resource "github.com/concourse/registry-image-resource"
"github.com/fatih/color"
"github.com/google/go-containerregistry/pkg/name"
v1 "github.com/google/go-containerregistry/pkg/v1"
+ "github.com/google/go-containerregistry/pkg/v1/mutate"
"github.com/google/go-containerregistry/pkg/v1/remote"
"github.com/google/go-containerregistry/pkg/v1/tarball"
"github.com/sirupsen/logrus"
@@ -194,7 +196,7 @@ func ociFormat(dest string, tag name.Tag, image v1.Image) error {
}
func rootfsFormat(dest string, image v1.Image, debug bool, stderr io.Writer) error {
- err := unpackImage(filepath.Join(dest, "rootfs"), image, debug, stderr)
+ err := tarfs.Extract(mutate.Extract(image), filepath.Join(dest, "rootfs"))
if err != nil {
return fmt.Errorf("extract image: %w", err)
}Test failure:
fetching concourse/test-image-removed-hardlinks@sha256:ee6f9ea0fe639ba0f30b3d6085f6c7813660a4de3e84bbec87c32e521d200dc9
ERRO[0000] download failed: save image: write rootfs: extract image: link /tmp/docker-image-in-dir081144662/rootfs/usr/bin/git /tmp/docker-image-in-dir081144662/rootfs/usr/bin/git-receive-pack: no such file or directory
• Failure in Spec Setup (JustBeforeEach) [0.827 seconds]
In
/home/vito/src/registry-image-resource/in_test.go:25
a hardlink that is later removed [JustBeforeEach]
/home/vito/src/registry-image-resource/in_test.go:213
works
/home/vito/src/registry-image-resource/in_test.go:222
Unexpected error:
<*exec.ExitError | 0xc000031060>: {
ProcessState: {
pid: 22284,
status: 256,
rusage: {
Utime: {Sec: 0, Usec: 104605},
Stime: {Sec: 0, Usec: 58114},
Maxrss: 28472,
Ixrss: 0,
Idrss: 0,
Isrss: 0,
Minflt: 1264,
Majflt: 0,
Nswap: 0,
Inblock: 0,
Oublock: 640,
Msgsnd: 0,
Msgrcv: 0,
Nsignals: 0,
Nvcsw: 1344,
Nivcsw: 40,
},
},
Stderr: nil,
}
exit status 1
occurred
/home/vito/src/registry-image-resource/in_test.go:74