memo: benchmark

[AkihiroSuda]

image	command	regular runc (root) (config)	runrootless	runrootless+seccomp
docker gentoo/stage3-amd64	emerge --sync	52s	1m43s	2m54s
ditto	emerge zsh (after emerge --sync)	2m1s	9m3s	(crashed quickly)
alpine	apk add gcc	1.4s	2.2s	2.0s
ditto	apk add openjdk8	3.1s	4.4s	3.14s
ditto	git clone https://github.com/torvalds/linux.git	6m38s	10m43s	(crashed quickly)

• PRoot overhead seems significant for emerge, especially during compiling packages
• For apk add, overhead is negligible
• Suggestion:
  -- Enable PRoot only during apk/apt/yum operation
  -- Disable PRoot for compilation

[AkihiroSuda]

• env: Ubuntu 18.04 on VMware Fusion

• runc: 63bb0fe9d001cdad7b43621aa2072b294e7f1cd3 (opencontainers/runc@63bb0fe)

• runsc: a8b90a7158d4197428639c912d97f3bdbaf63f5a

• workload: for f in $(seq 1 100); do /usr/bin/time -f '%e' dd if=/dev/urandom of=/dev/zero bs=4096 count=10000 2>&1 | grep -v records; done | awk '{a+=$1} END{print a/NR}'

---

• runc (as root): 0.2063
• rootless runc: 0.2065
• rootless runc + ptrace: 2.771
• rootless runc + ptrace + seccomp: 0.2041
• runsc (ptrace, as root): 0.423

note: KVM version of runsc didn't work