Kaniko finds a Dockerfile in CWD ahead of one in the context

[TBBle]

Actual behavior

Given a filesystem layout, e.g. a Git checkout:

• /workspace/Dockerfile
• /workspace/subproject/Dockerfile

When the CWD is /workspace/, the following command, e.g., when using Kaniko under GitLab CI,

/kaniko/exectuor --context /workspace/subproject

executes using /workspace/Dockerfile in context /workspace/subproject/.

Expected behavior

The build should use /workspace/subproject/Dockerfile in context /workspace/subproject/

To Reproduce

1. Execute gcr.io/kaniko-project/executor:debug-v0.24.0 as described in the README.

docker run -it --entrypoint=/busybox/sh -w /workspace gcr.io/kaniko-project/executor:debug-v0.24.0

2. Create a workspace as described above:

echo "BANANA" > Dockerfile
mkdir subproject
echo "FROM alpine:latest" > subproject/Dockerfile
echo "COPY somefile somefile" >> subproject/Dockerfile
touch subproject/somefile

3. Run Kaniko using /workspace/subproject as the context

/kaniko/executor --context /workspace/subproject --no-push

4. Actual result:

error building image: parsing dockerfile: Dockerfile parse error line 1: unknown instruction: BANANA

5. Workaround

cd subproject/
/kaniko/executor --context /workspace/subproject --no-push

6. Alternative workaround

cd /
/kaniko/executor --context /workspace/subproject --no-push

7. Not a workaround: This will actually rerun the last attempt, because it finds the Dockerfile copied out of the context by a previous build.

cd /kaniko
/kaniko/executor --context /workspace/subproject --no-push

Additional Information

• Dockerfile: Included in the reproduction steps above.
• Build Context: Included in the reproduction steps above.
• Kaniko Image (fully qualified with digest): gcr.io/kaniko-project/executor:debug-v0.24.0 (sha256:be318701bf818b309cb5b6bcbc709e88161db4229d105423abe5b66948a2f676)

The problem appears to be that the code that resolves a DockerFile accidentally succeeds by checking for CWD-relative Dockerfile before checking context-relative Dockerfile. It probably needs to be a little smarter than that, checking a context-relative path for Dockerfile names that don't start with . or contain .., and then fall back to CWD-relative.

The README doesn't mention the behaviour of checking CWD-relative for Dockerfile, as it describes the context as the source of the Dockerfile.

I suspect this issue has fallen through the cracks because all the examples in the README.md of build-from-filesystem use absolute --dockerfile parameters, and clearly the primary Kaniko workflow is to build with the context pointing at something other than the filesystem.

Triage Notes for the Maintainers

Description	Yes/No
Please check if this a new feature you are proposing	- [No]
Please check if the build works in docker but not in kaniko	- [Yes]
Please check if this error is seen when you use --cache flag	- [See below]
Please check if your dockerfile is a multistage dockerfile	- [No]

Is the --cache question here supposed to be "only seen" or "also seen"? My live repro uses --cache, but the minimal repro here uses --no-push so I can't use --cache.

[TBBle]

I'm not sure if it's related, but if I change
--context /workspace/subproject to --context /workspace --context-sub-path subproject it seems to run the build in the /workspace, not /workspace/subproject. I haven't tried this with non-filesystem paths though, so I guess it's possible that it only works with non-filesystem --context.

[tejal29]

@TBBle Thank you for bringing this to our attention. Our docs can definitely get some more love.
context is supposed to be the source/root of the build context like a github repo, while --context-subpath is the path with in. The reason, we have context and context-subpath is because at times, a Dockerfile may contain relative filepaths e.g. "../rootFile" and in order for kaniko to determine if the file within the source build context, we need a pointer to where the source is mounted.

Agreed, it is confusing. do you ideas to make it better?

[TBBle]

My assumption is that a Dockerfile can't ../ to outside the context. So what's not clear to me, is --context-sub-path supposed to change the context path (which it doesn't), and otherwise, what is it supposed to do?

That said, it's beside-the-point for this bug report, except that when I tried it as a fix, it actually made things worse.

[TBBle]

Hmm, why is this "kind/question"? It's intended to be a bug-report. Or is this behaviour Works-as-intended, and it's just the documentation doesn't mention the CWD at all?

[ejose19]

I've noticed this as well, and I agree that both --context and --context-sub-path needs fixing, as they're not working as expected.

Some examples of the issue with the default /workspace as context:

• Setting -context to /workspace/subfolder, still picks the dockerfile at /workspace/Dockerfile, and one needs to specify the full path on --dockerfile=/workspace/subfolder/Dockerfile in order to get it to work
• Setting -context-sub-path to /workspace/subfolder basically does nothing, the Dockerfile picked is still /workspace/Dockerfile and any relative path inside Dockerfile still references /workspace as base.

What would I expect when using those flags:

• context-sub-path must be inside context and it sets the new "root" to be used for building
• if context-sub-path is defined, any other flag that interacts with filesystem must use it as the root (like --dockerfile), instead of context

so if context-sub-path is defined and its Dockerfile uses COPY ../x ., even if ../x is within context, it should be forbidden, as now the subpath is the new context during build.

[selfagency]

two years later, --context-sub-path is still broken in google cloud's officially recommended caching solution, and i can't build my monorepo's containers in gcb without slow and convoluted self-authored caching, but gcp's happy to raise prices nonetheless.

[tparvu]

I just got bit by this. It is almost 4 years since the original post.

[toransahu]

+1

[hnizdil]

+1