Skip to content

Commit

Permalink
snapshotter: use syncfs system call (#2816)
Browse files Browse the repository at this point in the history
`sync` system call triggers a full page cache sync which may not always
work, especially in kubernetes environment where it is easy to be
interfered by others. I have seen several cases where a broken nfs mount
is blocking kaniko from doing its job.

With `syncfs`, it only writes cache back to disk for the current
filesystem that is used by kaniko which is supposed to be more reliable.
  • Loading branch information
zhouhaibing089 authored Oct 30, 2023
1 parent 546191e commit e65bce1
Showing 1 changed file with 16 additions and 1 deletion.
17 changes: 16 additions & 1 deletion pkg/snapshot/snapshot.go
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,7 @@ import (
"io/ioutil"
"os"
"path/filepath"
"runtime"
"sort"
"syscall"

Expand All @@ -31,6 +32,7 @@ import (
"github.com/GoogleContainerTools/kaniko/pkg/util"

"github.com/sirupsen/logrus"
"golang.org/x/sys/unix"
)

// For testing
Expand Down Expand Up @@ -155,7 +157,20 @@ func (s *Snapshotter) scanFullFilesystem() ([]string, []string, error) {
// for example the hashing function that determines if files are equal uses the mtime of the files,
// which can lag if sync is not called. Unfortunately there can still be lag if too much data needs
// to be flushed or the disk does its own caching/buffering.
syscall.Sync()
if runtime.GOOS == "linux" {
dir, err := os.Open(s.directory)
if err != nil {
return nil, nil, err
}
defer dir.Close()
_, _, errno := syscall.Syscall(unix.SYS_SYNCFS, dir.Fd(), 0, 0)
if errno != 0 {
return nil, nil, errno
}
} else {
// fallback to full page cache sync
syscall.Sync()
}

s.l.Snapshot()

Expand Down

0 comments on commit e65bce1

Please sign in to comment.