Skip to content

runsc: add --cpu-num-fixed to pin the sandbox CPU count - #14142

Open
mayur-tolexo wants to merge 1 commit into
google:masterfrom
mayur-tolexo:cpu-num-fixed
Open

runsc: add --cpu-num-fixed to pin the sandbox CPU count#14142
mayur-tolexo wants to merge 1 commit into
google:masterfrom
mayur-tolexo:cpu-num-fixed

Conversation

@mayur-tolexo

@mayur-tolexo mayur-tolexo commented Aug 16, 2026

Copy link
Copy Markdown

The number of CPUs a sandbox sees is derived at boot from the cgroup CPU quota
and is not updated afterwards. When a container's CPU limit is raised in place
(for example with Kubernetes in-place pod resize) the host cgroup quota changes
but the guest keeps the CPU count it booted with, so workloads that size their
parallelism to the reported count (Go's GOMAXPROCS, make -j$(nproc), the JVM,
nginx worker_processes auto) stay capped at the old value and can't use the CPU
that was just added without recreating the sandbox. This is the guest-visibility
side of #14141.

This adds a --cpu-num-fixed flag. When set, it fixes the number of CPUs exposed
to the sandbox to that value, independent of the cgroup quota and the host CPU
count, and takes precedence over --cpu-num-from-quota. A sandbox can then boot
with headroom CPUs and have a later in-place quota increase take effect without a
restart; the cgroup quota still bounds real CPU usage, so the extra CPUs are only
usable up to the quota. The value may exceed the host CPU count. The flag is
honored whether or not runsc reads a host cgroup, so it works with both
--systemd-cgroup and --ignore-cgroups.

Verified on kind (Kubernetes 1.35, containerd 2.2.0, systrap, arm64) with the
runsc runtime installed as --cpu-num-fixed=8.

With --cpu-num-fixed=8 --systemd-cgroup=true:

$ kubectl exec gv-sys -- nproc
8
# from the runsc debug log:
$ grep -o -- '--cpu-num-fixed=[0-9]*\|--systemd-cgroup=true\|--cpu-num [0-9]*' boot.log | sort -u
--cpu-num 8
--cpu-num-fixed=8
--systemd-cgroup=true
$ grep CPUs: boot.log
I0816 17:05:05.112745       1 loader.go:617] CPUs: 8

With --cpu-num-fixed=8 --ignore-cgroups=true (runsc reads no host cgroup):

$ kubectl exec gv-sentry -- nproc
8
$ grep -o -- '--cpu-num-fixed=[0-9]*\|--ignore-cgroups=true\|--cpu-num [0-9]*' boot.log | sort -u
--cpu-num 8
--cpu-num-fixed=8
--ignore-cgroups=true
$ grep CPUs: boot.log
I0816 17:00:48.311592       1 loader.go:617] CPUs: 8

And the motivating case, an in-place CPU resize actually becoming usable: a pod
booted with a 1-CPU limit and --cpu-num-fixed=8 reports 8 CPUs; with the quota
at 1 a CPU-bound, nproc-sized workload consumes ~1.0 cores, and after an in-place
resize of the CPU limit to 8 (no restart) the same workload consumes ~8.0 cores.
Without the flag the guest stays at the boot-time count of 2 and the workload
stays capped at ~2.0 cores.

The CPU-count derivation is factored into a small helper (cpuNumForSandbox) with
a unit test, and there is an end-to-end TestNumCPUFixed that installs a
-cpunumfixed runtime and checks the guest CPU count is pinned regardless of the
container's cpuset.

The number of CPUs exposed to the sandbox is derived at boot from the
cgroup CPU quota and is not updated afterwards, so raising a container's
CPU quota in place (for example via Kubernetes in-place pod resize) does
not give the guest any more usable CPUs. Workloads that size their
parallelism to the reported CPU count stay capped at the boot value until
the sandbox is recreated. See google#14141.

Add a --cpu-num-fixed flag that pins the CPU count to a fixed value,
independent of both the cgroup quota and the host CPU count. A sandbox can
then boot with headroom CPUs and have a later in-place quota increase take
effect without a restart. The value may exceed the host CPU count; the
cgroup quota still bounds real CPU usage. It applies whether the cgroup is
managed externally or by the sentry, and takes precedence over
--cpu-num-from-quota.

Signed-off-by: Mayur Das <mayur.das@neevcloud.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant