Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cgroup v1 initialization causes "invalid format for CGroupSubsys" when cgroup path contains colon #6389

Closed
PoliM opened this issue Oct 24, 2022 · 1 comment
Labels
bug Something isn't working

Comments

@PoliM
Copy link

PoliM commented Oct 24, 2022

Describe the bug
The memory_ballast extensions fails to initialize when reading the cgroup configuration.
The cgroup configuration contains three fields separated by colons. Now it seems that when the cgroup driver is systemd, the third field (the cgroup path) can contain colons in its value too. Since the initialization code just splits the line by colons it detects a wrong number of fields which result in an error showing "invalid format for CGroupSubsys". More details below under "Additional context".

Steps to reproduce

What did you expect to see?
An empty log.

What did you see instead?
2022/10/21 14:21:12 collector server run finished with error: failed to start extensions: invalid format for CGroupSubsys: "13:rdma:/kubepods-burstable-pod4f4de620_8801_4afa_a9c5_e7bbae3a38bb.slice:cri-containerd:50aefe89fd66b08c88f54c9850c44f84e240455e777551429c31e799274efce2"

What version did you use?
Docker image: otel/opentelemetry-collector-contrib:0.60.0

What config did you use?
otel-collector-config.txt

Environment
Kubernetes: 1.23
OS: SUSE Linux Enterprise Server 15 SP4 Kernel: 5.14.21-150400.24.21-default

Additional context
The error is created in the operation NewCGroupSubsysFromLine where it splits the string by the separator and checks that there are three fields. See:

func NewCGroupSubsysFromLine(line string) (*CGroupSubsys, error) {

Here is a sample of "old-style" cgroup configs:
sh-4.4$ cat /proc/self/cgroup
13:freezer:/kubepods/burstable/podd726b5c0-b28a-4a01-98a0-100a5bdc5c9a/c9976349bfb59dabbe1fd9ae8520c0517dd3e56702f4fa57b8929641fdf87eeb
12:cpuset:/kubepods/burstable/podd726b5c0-b28a-4a01-98a0-100a5bdc5c9a/c9976349bfb59dabbe1fd9ae8520c0517dd3e56702f4fa57b8929641fdf87eeb

And here some with the new format which contains colons in the path:
sh-4.4$ cat /proc/self/cgroup
13:pids:/system.slice/rke2-agent.service/kubepods-burstable-pod4d692290_f5ee_45b9_8093_50de46b24c12.slice:cri-containerd:811794ed3b7ecfb2f5f43eb3a3f3f66ab73944ed6258cc6377ffc8b513ce8a65
12:rdma:/kubepods-burstable-pod4d692290_f5ee_45b9_8093_50de46b24c12.slice:cri-containerd:811794ed3b7ecfb2f5f43eb3a3f3f66ab73944ed6258cc6377ffc8b513ce8a65

@PoliM PoliM added the bug Something isn't working label Oct 24, 2022
@rockdrilla
Copy link
Contributor

Hi there! I've done minor fix for this issue. I've been hit the same issue too while writing own "/proc/{pid}/cgroup" parser.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants