Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

when a directory with symlinks is copied, the symlinks are badly created in the target image #437

Open
mtsyganov opened this issue Nov 7, 2018 · 11 comments · Fixed by #456
Labels
area/multi-stage builds issues related to kaniko multi-stage builds area/symlinks categorized differs-from-docker kind/bug Something isn't working needs-reproduction norepro core team can't replicate this issue, require more info ok-to-close? possible-dupe priority/p1 Basic need feature compatibility with docker build. we should be working on this next. works-with-docker

Comments

@mtsyganov
Copy link

mtsyganov commented Nov 7, 2018

Actual behavior
In the multistage build process when a directory with symlinks is copied, the symlinks are badly created in the target image. Too many levels of symbolic links are generated.

Expected behavior
Symlinks are copied from source image into the target image as they are created in the source image

To Reproduce
Steps to reproduce the behavior:

  1. create an image with symbolic link. For example /usr/local/psql/bin/postmaster -> postgres (you can check registry.gitlab.com/gitlab-org/build/cng/postgresql:9.6.8)
  2. build a multistage image where directory with symlinks is copied to target image COPY --from=pg_image /usr/local/psql/bin/ /usr/bin/
  3. run created image and check the copied symlinks. In the source image they are defined as /usr/local/psql/bin/postmaster -> postgres, but in the target image as /usr/bin/postmaster -> /usr/bin/postmaster/postgres. Correct symlink should be in the target image /usr/bin/postmaster -> postgres

Additional Information

  • Dockerfile
FROM registry.gitlab.com/gitlab-org/build/cng/postgresql:9.6.8 as pg_image
FROM ubuntu
COPY --from=pg_image /usr/local/psql/bin/ /usr/bin/
mtsyganov pushed a commit to mtsyganov/kaniko that referenced this issue Nov 7, 2018
@priyawadhwa priyawadhwa added the kind/bug Something isn't working label Nov 12, 2018
@priyawadhwa priyawadhwa self-assigned this Nov 16, 2018
priyawadhwa pushed a commit to priyawadhwa/kaniko that referenced this issue Nov 17, 2018
When we execute multistage builds, we store the fs of each intermediate
stage at /kaniko/<stage number> if it's used later in the build. This
created a bug when extracting hardlinks, because we weren't appending
the new directory to the link path.

So, if `/tmp/file1` and `/tmp/file2` were hardlinked, kaniko was trying
to link `/kaniko/0/tmp/file1` to `/tmp/file2` instead of
`/kaniko/0/tmp/file2`. This change will append the correct directory to
the link, and fixes GoogleContainerTools#437 GoogleContainerTools#362 GoogleContainerTools#352 GoogleContainerTools#342.
priyawadhwa pushed a commit to priyawadhwa/kaniko that referenced this issue Nov 17, 2018
When we execute multistage builds, we store the fs of each intermediate
stage at /kaniko/<stage number> if it's used later in the build. This
created a bug when extracting hardlinks, because we weren't appending
the new directory to the link path.

So, if `/tmp/file1` and `/tmp/file2` were hardlinked, kaniko was trying
to link `/kaniko/0/tmp/file1` to `/tmp/file2` instead of
`/kaniko/0/tmp/file2`. This change will append the correct directory to
the link, and fixes GoogleContainerTools#437 GoogleContainerTools#362 GoogleContainerTools#352 GoogleContainerTools#342.
priyawadhwa pushed a commit to priyawadhwa/kaniko that referenced this issue Nov 17, 2018
When we execute multistage builds, we store the fs of each intermediate
stage at /kaniko/<stage number> if it's used later in the build. This
created a bug when extracting hardlinks, because we weren't appending
the new directory to the link path.

So, if `/tmp/file1` and `/tmp/file2` were hardlinked, kaniko was trying
to link `/kaniko/0/tmp/file1` to `/tmp/file2` instead of
`/kaniko/0/tmp/file2`. This change will append the correct directory to
the link, and fixes GoogleContainerTools#437 GoogleContainerTools#362 GoogleContainerTools#352 GoogleContainerTools#342.
@mtsyganov
Copy link
Author

@priyawadhwa retested with latest debug image. problem is still reproducable

@priyawadhwa priyawadhwa reopened this Nov 20, 2018
@priyawadhwa
Copy link
Collaborator

@mtsyganov sorry about that, I reopened the issue

@andrewrynhard
Copy link
Contributor

andrewrynhard commented Nov 30, 2018

@priyawadhwa Where is this on the list of priorities? This is a major blocker for my particular use case. If low on your list I'd consider picking this up, but would be really happy if a fix is coming soon.

EDIT:

Looks like #459 is in the works! Any chance someone take a look at this?

@priyawadhwa
Copy link
Collaborator

@mtsyganov and @andrewrynhard, I just merged #459, does that fix your respective bugs?

@miguelitoq76
Copy link

miguelitoq76 commented Dec 4, 2018

Dear @priyawadhwa we had the same error during the use of kaniko in a custom image,
this was a for us a problem, until we started to create a Dummy build via executor --no_push , to force a filesystem cleanup. See my post in #434

I assume, it would be a good idea, to start the build with a "delete FS", what do you think ?

@priyawadhwa
Copy link
Collaborator

@miguelitoq76 sorry, I'm a bit confused -- do you want kaniko to delete the filesystem before starting the image build? Wouldn't that essentially be the same as directly using the kaniko image (which is built from scratch)?

@priyawadhwa priyawadhwa added the area/multi-stage builds issues related to kaniko multi-stage builds label Jul 25, 2019
@tequilarista
Copy link

Triage - collapsing 'waiting for response' category into 'blocked'

@tejal29 tejal29 added the priority/p1 Basic need feature compatibility with docker build. we should be working on this next. label Sep 27, 2019
@cvgw
Copy link
Contributor

cvgw commented Nov 15, 2019

I am unable to repro this issue; leading me to think that it has since been fixed. If this issue can still be repro'd please share the appropriate steps to do so.

@cvgw cvgw added needs-reproduction norepro core team can't replicate this issue, require more info labels Nov 15, 2019
@Kami-no
Copy link

Kami-no commented Jan 9, 2020

Still got this with fresh gcr.io/kaniko-project/executor:debug

GitLab CI + Kaniko

@cvgw
Copy link
Contributor

cvgw commented Jan 10, 2020

@Kami-no would you mind sharing an example Dockerfile and the options you are passing to kaniko?

@raijinsetsu
Copy link

Is this related to #1036 ?

@aaron-prindle aaron-prindle changed the title multi-stage copy of symlinks when a directory with symlinks is copied, the symlinks are badly created in the target image Jul 5, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/multi-stage builds issues related to kaniko multi-stage builds area/symlinks categorized differs-from-docker kind/bug Something isn't working needs-reproduction norepro core team can't replicate this issue, require more info ok-to-close? possible-dupe priority/p1 Basic need feature compatibility with docker build. we should be working on this next. works-with-docker
Projects
None yet
Development

Successfully merging a pull request may close this issue.

10 participants