Description
Actual behavior
During some of our multistage building with kaniko >= 1.10 we are getting:
panic: runtime error: index out of range [18] with length 18 goroutine 1 [running]: github.com/google/go-containerregistry/pkg/v1/mutate.Time({0x1bb62a0, 0xc000201e80}, {0xc000201e00?, 0xc032c9e800?, 0x0?}) /src/vendor/github.com/google/go-containerregistry/pkg/v1/mutate/mutate.go:405 +0x9db github.com/google/go-containerregistry/pkg/v1/mutate.Canonical({0x1bb62a0?, 0xc000201e80?}) /src/vendor/github.com/google/go-containerregistry/pkg/v1/mutate/mutate.go:509 +0x2d github.com/GoogleContainerTools/kaniko/pkg/executor.DoBuild(0x2728d60) /src/pkg/executor/build.go:700 +0xee5 github.com/GoogleContainerTools/kaniko/cmd/executor/cmd.glob..func2(0x2713ce0?, {0x1952b21?, 0x33?, 0x37?}) /src/cmd/executor/cmd/root.go:163 +0x1c7 github.com/spf13/cobra.(*Command).execute(0x2713ce0, {0xc000050390, 0x33, 0x37}) /src/vendor/github.com/spf13/cobra/command.go:944 +0x847 github.com/spf13/cobra.(*Command).ExecuteC(0x2713ce0) /src/vendor/github.com/spf13/cobra/command.go:1068 +0x3bd github.com/spf13/cobra.(*Command).Execute(0x195f383?) /src/vendor/github.com/spf13/cobra/command.go:992 +0x19 main.main() /src/cmd/executor/main.go:31 +0x6c
This only happens when --reproducible flag is used, so when timestamps are striped out from layers.
Full cmd is the following:
/kaniko/executor --registry-mirror $KANIKO_EXECUTOR_MIRROR --context $CI_PROJECT_DIR --dockerfile $KANIKO_EXECUTOR_DOCKERFILE $FORMATTEDTAGLIST $IMAGE_LABELS --reproducible --digest-file=$CI_PROJECT_DIR/digest $KANIKO_EXECUTOR_EXTRA_OPTS
The issue is most likely related to a bug in https://github.com/google/go-containerregistry which is fixed by google/go-containerregistry#1715. This is available in go-containerregistry v0.16 but kanko project still uses 0.15.2 for some time. Would be nice to bump it to 0.16
Expected behavior
The build should succeed as it does for kaniko < 1.9
To Reproduce
The problem only occurs with certain specific structures of images that a build is based on, like those that have empty layers (according to google/go-containerregistry#1715).
For sure a build will fail if you have in your Dockerfile simply:
FROM beyssoftware/hadoop-server-base:1.0.1r3
and you run the build with --reproducible
Note: beyssoftware/hadoop-server-base:1.0.1r3 is our image that seems to have a structure that triggers the bug for kaniko>= 1.10, but it is completely fine if we use it as a base while running with kaniko < 1.10
The output from the build should be:
panic: runtime error: index out of range [8] with length 8 goroutine 1 [running]: github.com/google/go-containerregistry/pkg/v1/mutate.Time({0x1bb62a0, 0xc000188c80}, {0xc000188c00?, 0xc0001bb400?, 0x0?}) /src/vendor/github.com/google/go-containerregistry/pkg/v1/mutate/mutate.go:405 +0x9db github.com/google/go-containerregistry/pkg/v1/mutate.Canonical({0x1bb62a0?, 0xc000188c80?}) /src/vendor/github.com/google/go-containerregistry/pkg/v1/mutate/mutate.go:509 +0x2d github.com/GoogleContainerTools/kaniko/pkg/executor.DoBuild(0x2728d60) /src/pkg/executor/build.go:700 +0xee5 github.com/GoogleContainerTools/kaniko/cmd/executor/cmd.glob..func2(0x2713ce0?, {0x19[52](https://sources.factory.be-ys.net/BigeYs/infrastructure/big-data/hadoop/hadoop-docker/-/jobs/3497957#L52)b21?, 0x2d?, 0x2f?}) /src/cmd/executor/cmd/root.go:163 +0x1c7 github.com/spf13/cobra.(*Command).execute(0x2713ce0, {0xc000004[61](https://sources.factory.be-ys.net/BigeYs/infrastructure/big-data/hadoop/hadoop-docker/-/jobs/3497957#L61)0, 0x2d, 0x2f}) /src/vendor/github.com/spf13/cobra/command.go:944 +0x847 github.com/spf13/cobra.(*Command).ExecuteC(0x2713ce0) /src/vendor/github.com/spf13/cobra/command.go:1068 +0x3bd github.com/spf13/cobra.(*Command).Execute(0x195f383?) /src/vendor/github.com/spf13/cobra/command.go:992 +0x19 main.main() /src/cmd/executor/main.go:31 +0x6c
| Description | Yes/No |
|---|---|
| Please check if this a new feature you are proposing | |
| Please check if the build works in docker but not in kaniko | |
Please check if this error is seen when you use --cache flag |
|
| Please check if your dockerfile is a multistage dockerfile |