With buildpack & auto sync a full re-build happens unpredictably

[etanshaul]

Tested using skaffold 1.17.2

1. Clone the Cloud Code Go hello-world k8s sample: https://github.com/GoogleCloudPlatform/cloud-code-samples/tree/master/golang/go-hello-world
2. Configure your build to use buildpacks:

build:
  tagPolicy:
    sha256: {}
  artifacts:
  - image: go-hello-world
    context: .
    buildpack:
      builder: gcr.io/buildpacks/builder:v1

3. Run skaffold dev
4. Make a change in main.go

Expected: Skaffolds auto-sync for buildpacks would automtically sync the change without performing a full container rebuild.

Actual: The full build happens.

Note: This only seems to happen unpredictably; in my tests, some changes sync, others trigger a full rebuild (the changes are identical each time - e.g. a simple fmt.Println string modification in this case).

Example output (the first couple changes did a full rebuild, the last change synced).

[abbycar]

+1 here's a video of the same behavior

[tejal29]

@briandealwis i am going to look into this as a starter bug for buildpacks :)

[tejal29]

Some notes from my debugging on command line

1. First error i see, is .swp file is being detected as modified which triggers the re-build.

Deployments stabilized in 2.533034424s
Watching for changes...
DEBU[0148] Change detected notify.Create: "/Users/tejaldesai/workspace/cloud-code-samples/golang/go-hello-world/cmd/hello-world/4913" 
DEBU[0148] Change detected notify.Rename: "/Users/tejaldesai/workspace/cloud-code-samples/golang/go-hello-world/cmd/hello-world/main.go~" 
DEBU[0148] Change detected notify.Create: "/Users/tejaldesai/workspace/cloud-code-samples/golang/go-hello-world/cmd/hello-world/main.go" 
DEBU[0148] Change detected notify.Remove: "/Users/tejaldesai/workspace/cloud-code-samples/golang/go-hello-world/cmd/hello-world/main.go~" 
DEBU[0148] Change detected notify.Rename: "/Users/tejaldesai/workspace/cloud-code-samples/golang/go-hello-world/cmd/hello-world/main.go" 
DEBU[0148] Change detected notify.Write: "/Users/tejaldesai/workspace/cloud-code-samples/golang/go-hello-world/cmd/hello-world/main.go" 
DEBU[0148] Change detected notify.Remove: "/Users/tejaldesai/workspace/cloud-code-samples/golang/go-hello-world/cmd/hello-world/4913" 
INFO[0149] files modified: [cmd/hello-world/.main.go.swp cmd/hello-world/main.go] 
DEBU[0149] Using default workdir '/' for go-hello-world:ac7fb047c293514cd108ed3851b1606d270efb5b0bbb809258e7964fa85346ae 
INFO[0149] Changed file cmd/hello-world/.main.go.swp does not match any sync pattern. Skipping sync 

[tejal29]

2. For IDEs, i see .idea folder files are detected as changed

time="2020-12-15T10:16:00-08:00" level=debug msg="Change detected notify.Write: \"/Users/tejaldesai/workspace/cloud-code-samples/golang/go-hello-world/cmd/hello-world/main.go\""
time="2020-12-15T10:16:01-08:00" level=debug msg="Change detected notify.Write: \"/Users/tejaldesai/workspace/cloud-code-samples/golang/go-hello-world/.idea/workspace.xml\""
time="2020-12-15T10:16:02-08:00" level=info msg="files modified: [.idea/workspace.xml cmd/hello-world/main.go]"
time="2020-12-15T10:16:02-08:00" level=debug msg="Using default workdir '/' for go-hello-world:6a7356664f6004e3636ee0ac5837750353d08ea79adafa9311f9b2afc5412a8a"
time="2020-12-15T10:16:02-08:00" level=info msg="Changed file .idea/workspace.xml does not match any sync pattern. Skipping sync"

[gsquared94]

@tejal29 are you still working on this; or should we lower the priority (exceeded the threshold for P1s)?

[tejal29]

This is a P1 but i havent been actively working on it.

[tejal29]

After discussing the issue with @briandealwis, We need to figure a way in skaffold to ignore files .skaffoldignore and add few expected files like .idea/, .vscode etc

Changing this to feature request