Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove leading slash in layer tarball paths (Closes: #726) #729

Conversation

xanonid
Copy link
Contributor

@xanonid xanonid commented Jul 29, 2019

This PR removes leading slashes in the path of files in the layer.tar tarballs. This is necessary to be more compatible with docker and some security scanners.

Closes: #726

@tejal29
Copy link
Member

tejal29 commented Aug 9, 2019

@xanonid thank you for the PR and sorry for delay.
Currently some of the integration tests are failing.

--- FAIL: TestRun/test_Dockerfile_test_mv_add (3.44s)
        integration_test.go:224: diff = [
              {
                "Image1": "gcr.io/kaniko-test/docker-dockerfile_test_mv_add",
                "Image2": "gcr.io/kaniko-test/kaniko-dockerfile_test_mv_add",
                "DiffType": "File",
                "Diff": {
                  "Adds": [
                    {
                      "Name": "/tmp/tars",
                      "Size": 10813
                    },
                    {
                      "Name": "/tmp/tars/file",
                      "Size": 12
                    },
                    {
                      "Name": "/tmp/tars/file.bz2",
                      "Size": 196
                    },
                    {
                      "Name": "/tmp/tars/file.tar",
                      "Size": 10240
                    },
                    {
                      "Name": "/tmp/tars/file.tar.gz",
                      "Size": 199
                    },
                    {
                      "Name": "/tmp/tars/sys.tar.gz",
                      "Size": 166
                    }
                  ],
                  "Dels": null,
                  "Mods": null
                }
              },
              {
                "Image1": "gcr.io/kaniko-test/docker-dockerfile_test_mv_add",
                "Image2": "gcr.io/kaniko-test/kaniko-dockerfile_test_mv_add",
                "DiffType": "Metadata",
                "Diff": {
                  "Adds": [],
                  "Dels": []
                }
              }
            ]
        integration_test.go:227: []integration.diffOutput differ (-got, +want): {[]integration.diffOutput}[0].Diff.Adds:
            	-: []integration.fileDiff{{Name: "/tmp/tars", Size: 10813}, {Name: "/tmp/tars/file", Size: 12}, {Name: "/tmp/tars/file.bz2", Size: 196}, {Name: "/tmp/tars/file.tar", Size: 10240}, {Name: "/tmp/tars/file.tar.gz", Size: 199}, {Name: "/tmp/tars/sys.tar.gz", Size: 166}}
            	+: []integration.fileDiff(nil)

--- FAIL: TestRun/test_Dockerfile_test_run (10.21s)
        integration_test.go:224: diff = [
              {
                "Image1": "gcr.io/kaniko-test/docker-dockerfile_test_run",
                "Image2": "gcr.io/kaniko-test/kaniko-dockerfile_test_run",
                "DiffType": "File",
                "Diff": {
                  "Adds": [
                    {
                      "Name": "/etc/baz",
                      "Size": 4
                    }
                  ],
                  "Dels": null,
                  "Mods": null
                }
              },
              {
                "Image1": "gcr.io/kaniko-test/docker-dockerfile_test_run",
                "Image2": "gcr.io/kaniko-test/kaniko-dockerfile_test_run",
                "DiffType": "Metadata",
                "Diff": {
                  "Adds": [],
                  "Dels": []
                }
              }
            ]
        integration_test.go:227: []integration.diffOutput differ (-got, +want): {[]integration.diffOutput}[0].Diff.Adds:
            	-: []integration.fileDiff{{Name: "/etc/baz", Size: 4}}
            	+: []integration.fileDiff(nil)

Let me know if you need more help in figuring these failures.

@xanonid xanonid force-pushed the Remove_leading_slash_in_layer_tarball_paths branch from 319e534 to 30db2c0 Compare August 9, 2019 15:40
@xanonid
Copy link
Contributor Author

xanonid commented Aug 12, 2019

@tejal29 After rebasing and with my latest commit, which remove also the leadings slash from whiteout file paths, the integration test seems to run now fine. Could you, please, trigger kokoro again?

PS: To get it really running (local docker version 18.09.7), I needed to apply following change (the corresponding error message is mentioned in #368 ):

diff --git a/pkg/util/fs_util.go b/pkg/util/fs_util.go
index 4abfcd4..be9ef7b 100644
--- a/pkg/util/fs_util.go
+++ b/pkg/util/fs_util.go
@@ -99,6 +99,11 @@ func GetFSFromImage(root string, img v1.Image) ([]string, error) {
                        path := filepath.Join(root, filepath.Clean(hdr.Name))
                        base := filepath.Base(path)
                        dir := filepath.Dir(path)
+                       if base == ".wh..wh..opq" {
+                               // FIXME: skip to prevent error in next step
+                               continue
+                       }
+
                        if strings.HasPrefix(base, ".wh.") {
                                logrus.Debugf("Whiting out %s", path)
                                name := strings.TrimPrefix(base, ".wh."

@tejal29 tejal29 merged commit 873b577 into GoogleContainerTools:master Sep 13, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

File paths should not start with / in layer.tar - clair does not work correctly with kaniko images
4 participants