[DRAFT] Ubuntu 20.04 allows executable rewrite unexpectedly

[zhouhaibing089]

1. Launch a ubuntu 20.04 environment - https://releases.ubuntu.com/focal/ (Tested with Virtualbox)

2. Install Docker - apt install docker.io

3. Prepare a Dockerfile like below:

FROM busybox
COPY --from=gcr.io/kaniko-project/executor:v1.23.2 /kaniko /kaniko

4. Build it via kaniko

$ docker run -it --entrypoint /bin/sh --rm -v $(pwd):/workspace gcr.io/kaniko-project/executor:v1.23.2-debug
/workspace # executor --no-push

(which works)

/workspace # executor --no-push

...
error building image: error building stage: failed to execute command: copying dir: creating file: open /kaniko/executor: text file busy

executor is a binary from /kaniko directory, and when copying /kaniko directory from another image, it actually copies to the current container /kaniko, so an error like text file busy is expected, the problem is why the first run is permitted without text file busy error.

Notes

• ubuntu 20.04 uses kernel 5.4
• overlayfs

[zhouhaibing089]

strace

First run:

$ strace -p <pid> -f -e trace="/(open|exec)" -o syscall.txt
$ cat syscall.txt
2705  openat(AT_FDCWD, "/root/.ash_history", O_WRONLY|O_CREAT|O_APPEND, 0600) = 3
2895  execve("/app", ["/app"], 0x559425259660 /* 6 vars */) = 0
2895  openat(AT_FDCWD, "/sys/kernel/mm/transparent_hugepage/hpage_pmd_size", O_RDONLY) = 3
2895  openat(AT_FDCWD, "/etc/localtime", O_RDONLY) = 3
2895  openat(AT_FDCWD, "/app", O_RDONLY|O_CLOEXEC) = 3
2895  openat(AT_FDCWD, "/app", O_RDWR|O_CREAT|O_TRUNC|O_CLOEXEC, 0666) = 3

Second run:

2900  execve("/app", ["/app"], 0x559425259660 /* 6 vars */) = 0
2900  openat(AT_FDCWD, "/sys/kernel/mm/transparent_hugepage/hpage_pmd_size", O_RDONLY) = 3
2900  openat(AT_FDCWD, "/etc/localtime", O_RDONLY) = 3
2900  openat(AT_FDCWD, "/app", O_RDONLY|O_CLOEXEC) = 3
2902  openat(AT_FDCWD, "/app", O_RDWR|O_CREAT|O_TRUNC|O_CLOEXEC, 0666) = -1 ETXTBSY (Text file busy)

This is to give us an idea on how many openat system calls were being made.

ftrace

Enable function-fork:

$ cat /sys/kernel/tracing/trace_options | grep function-fork
function-fork

If not, run echo function-fork >> /sys/kernel/tracing/trace_options

Now clear the existing trace if there are any:

$ echo nop > /sys/kernel/tracing/current_tracer

Now enable function_graph trace:

$ echo <pid> > /sys/kernel/tracing/set_ftrace_pid
$ echo __x64_sys_openat > /sys/kernel/tracing/set_graph_function
$ echo function_graph > /sys/kernel/tracing/current_tracer
$ cat /sys/kernel/tracing/trace > output

Disable the tracing after we are done:

$ echo nop > /sys/kernel/tracing/current_tracer

Check that we do get 9 __sys_openat:

$ cat output | grep __x64_sys_openat
 31)               |  __x64_sys_openat() {
 34)               |  __x64_sys_openat() {
 34)               |  __x64_sys_openat() {
 34)               |  __x64_sys_openat() {
 34)               |  __x64_sys_openat() {
 32)               |  __x64_sys_openat() {
 32)               |  __x64_sys_openat() {
 32)               |  __x64_sys_openat() {
 34)               |  __x64_sys_openat() {

We care about the call at 4th, 5th (the first run) and 8th, 9th (the second run):

[zhouhaibing089]

Testing file:

package main

import (
	"bytes"
	"fmt"
	"io"
	"log"
	"os"
	"syscall"
)

func main() {
	if len(os.Args) > 1 && os.Args[1] == "help" {
		fmt.Println("This program is used to verify whether a process can rewrite itself.")
		os.Exit(0)
	}

	// print pid
	log.Printf("pid: %d", os.Getpid())

	path, err := os.Executable()
	if err != nil {
		log.Fatal("failed to find executable")
	}

	showStat(path)

	file, err := os.Open(path)
	if err != nil {
		log.Fatalf("failed to open %s: %s", path, err)
	}
	data, err := io.ReadAll(file)
	if err != nil {
		log.Fatalf("failed to read file: %s", err)
	}
	file.Close()

	// trying to rewrite it
	newfile, err := os.Create(path)
	if err != nil {
		log.Fatalf("failed to create %s: %s", path, err)
	}
	defer newfile.Close()
	_, err = io.Copy(newfile, bytes.NewReader(data))
	if err != nil {
		log.Fatalf("failed to copy file: %s", err)
	}

	showStat(path)
}

func showStat(path string) {
	fi, err := os.Stat(path)
	if err != nil {
		log.Fatalf("failed to stat %s: %s", path, err)
	}

	stat, ok := fi.Sys().(*syscall.Stat_t)
	if !ok {
		log.Fatalf("failed to get underlying stat for %s", path)
	}

	log.Printf("path: %s, dev: %d, inode %d", path, stat.Dev, stat.Ino)
}

Dockerfile

FROM golang:1.22
WORKDIR /go/src/app
COPY . .
RUN CGO_ENABLED=0 go build -o /go/bin/app

FROM busybox
COPY --from=0 /go/bin/app /app

[zhouhaibing089]

On 5.15:

strace

2474  openat(AT_FDCWD, "/root/.ash_history", O_WRONLY|O_CREAT|O_APPEND, 0600) = 3
2613  execve("/app", ["/app"], 0x559240ec4660 /* 6 vars */) = 0
2613  openat(AT_FDCWD, "/sys/kernel/mm/transparent_hugepage/hpage_pmd_size", O_RDONLY) = 3
2613  openat(AT_FDCWD, "/etc/localtime", O_RDONLY) = 3
2613  openat(AT_FDCWD, "/app", O_RDONLY|O_CLOEXEC) = 3
2613  openat(AT_FDCWD, "/app", O_RDWR|O_CREAT|O_TRUNC|O_CLOEXEC, 0666) = -1 ETXTBSY (Text file busy)