runtime/cgo: pthread_create failed: Operation not permitted

[michaeljs1990]

All of a sudden today in CI on the docker 1.20 image I am getting this error. In general the OS hasn't mattered to me since I'm not installing anything in the container however over the past week all of these bookworm upgrades have broken multiple things. This has been an absolute pain. I pinned back to bullseye to fix the issue.

runtime/cgo: pthread_create failed: Operation not permitted
SIGABRT: abort
PC=0x7f[29](https://gitlab.com/michaeljs1990/-/jobs/4480933541#L29)9d101ccc m=0 sigcode=18446744073709551610
goroutine 0 [idle]:
runtime: g 0: unknown pc 0x7f299d101ccc
stack: frame={sp:0x7ffc0e570a[30](https://gitlab.com/michaeljs1990/-/jobs/4480933541#L30), fp:0x0} stack=[0x7ffc0dd71eb0,0x7ffc0e570ec0)
0x00007ffc0e570930:  0x0000000000000000  0x0000000000000000 
0x00007ffc0e570940:  0x0000000000000000  0x0000000000000000 
0x00007ffc0e570950:  0x0000000000000000  0x0000000000000000 
0x00007ffc0e570960:  0x0000000000000000  0x0000000000000000 
0x00007ffc0e570970:  0x0000000000000000  0x0000000000000000 
0x00007ffc0e570980:  0x0000000000000000  0x0000000000000000 
0x00007ffc0e570990:  0x0000000000000000  0x0000000000000000 
0x00007ffc0e5709a0:  0x0000000000000000  0x0000000000000000 
0x00007ffc0e5709b0:  0x0000000000000000  0x0000000000000000 
0x00007ffc0e5709c0:  0x0000000000000000  0x0000000000000000 
0x00007ffc0e5709d0:  0x0000000000000000  0x0000000000000000 
0x00007ffc0e5709e0:  0x0000000000000000  0x0000000000000000 
0x00007ffc0e5709f0:  0x0000000000000000  0x0000000000000000 
0x00007ffc0e570a00:  0x0000000000000000  0x0000000000000000 
0x00007ffc0e570a10:  0x0000000000000000  0x0000000000000000 
0x00007ffc0e570a20:  0x0000000000000000  0x00007f299d101cbe 
0x00007ffc0e570a30: <0x0000000000000000  0x723d2808a3ffd900 
0x00007ffc0e570a40:  0x0000000000000006  0x00007f299d075b80 
0x00007ffc0e570a50:  0x0000000000000001  0x00007ffc0e570d00 
0x00007ffc0e570a60:  0x0000000000e69fa0  0x00007f299d0b2ef2 
0x00007ffc0e570a70:  0x00007f299d24ae70  0x00007f299d09d472 
0x00007ffc0e570a80:  0x0000000000000020  0x0000000000000000 
0x00007ffc0e570a90:  0x0000000000000000  0x0000000000000000 
0x00007ffc0e570aa0:  0x0000000000000000  0x0000000000000000 
0x00007ffc0e570ab0:  0x0000000000000000  0x0000000000000000 
0x00007ffc0e570ac0:  0x0000000000000000  0x0000000000000000 
0x00007ffc0e570ad0:  0x0000000000000000  0x0000000000000000 
0x00007ffc0e570ae0:  0x0000000000000000  0x0000000000000000 
0x00007ffc0e570af0:  0x0000000000000000  0x0000000000000000 
0x00007ffc0e570b00:  0x00007f299d075b80  0x00007f299d0f480a 
0x00007ffc0e570b10:  0x00007f299d24a840  0x723d2808a3ffd900 
0x00007ffc0e570b20:  0x00007f299d24a840  0x00007f299d24a840 
runtime: g 0: unknown pc 0x7f299d101ccc
stack: frame={sp:0x7ffc0e570a30, fp:0x0} stack=[0x7ffc0dd71eb0,0x7ffc0e570ec0)
0x00007ffc0e570930:  0x0000000000000000  0x0000000000000000 
0x00007ffc0e570940:  0x0000000000000000  0x0000000000000000 
0x00007ffc0e570950:  0x0000000000000000  0x0000000000000000 
0x00007ffc0e570960:  0x0000000000000000  0x0000000000000000 
0x00007ffc0e570970:  0x0000000000000000  0x0000000000000000 
0x00007ffc0e570980:  0x0000000000000000  0x0000000000000000 
0x00007ffc0e570990:  0x0000000000000000  0x0000000000000000 
0x00007ffc0e5709a0:  0x0000000000000000  0x0000000000000000 
0x00007ffc0e5709b0:  0x0000000000000000  0x0000000000000000 
0x00007ffc0e5709c0:  0x0000000000000000  0x0000000000000000 
0x00007ffc0e5709d0:  0x0000000000000000  0x0000000000000000 
0x00007ffc0e5709e0:  0x0000000000000000  0x0000000000000000 
0x00007ffc0e5709f0:  0x0000000000000000  0x0000000000000000 
0x00007ffc0e570a00:  0x0000000000000000  0x0000000000000000 
0x00007ffc0e570a10:  0x0000000000000000  0x0000000000000000 
0x00007ffc0e570a20:  0x0000000000000000  0x00007f299d101cbe 
0x00007ffc0e570a30: <0x0000000000000000  0x723d2808a3ffd900 
0x00007ffc0e570a40:  0x0000000000000006  0x00007f299d075b80 
0x00007ffc0e570a50:  0x0000000000000001  0x00007ffc0e570d00 
0x00007ffc0e570a60:  0x0000000000e69fa0  0x00007f299d0b2ef2 
0x00007ffc0e570a70:  0x00007f299d24ae70  0x00007f299d09d472 
0x00007ffc0e570a80:  0x0000000000000020  0x0000000000000000 
0x00007ffc0e570a90:  0x0000000000000000  0x0000000000000000 
0x00007ffc0e570aa0:  0x0000000000000000  0x0000000000000000 
0x00007ffc0e570ab0:  0x0000000000000000  0x0000000000000000 
0x00007ffc0e570ac0:  0x0000000000000000  0x0000000000000000 
0x00007ffc0e570ad0:  0x0000000000000000  0x0000000000000000 
0x00007ffc0e570ae0:  0x0000000000000000  0x0000000000000000 
0x00007ffc0e570af0:  0x0000000000000000  0x0000000000000000 
0x00007ffc0e570b00:  0x00007f299d075b80  0x00007f299d0f480a 
0x00007ffc0e570b10:  0x00007f299d24a840  0x723d2808a3ffd900 
0x00007ffc0e570b20:  0x00007f299d24a840  0x00007f299d24a840 
goroutine 1 [running]:
runtime.systemstack_switch()
	/usr/local/go/src/runtime/asm_amd64.s:463 fp=0xc000042780 sp=0xc000042778 pc=0x4672c0
runtime.main()
	/usr/local/go/src/runtime/proc.go:170 +0x6d fp=0xc0000427e0 sp=0xc000042780 pc=0x43942d
runtime.goexit()
	/usr/local/go/src/runtime/asm_amd64.s:1598 +0x1 fp=0xc0000427e8 sp=0xc0000427e0 pc=0x4694e1
rax    0x0
rbx    0x1
rcx    0x7f299d101ccc
rdx    0x6
rdi    0x1
rsi    0x1
rbp    0x7f299d075b80
rsp    0x7ffc0e570a30
r8     0x0
r9     0x73
r10    0x8
r11    0x246
r12    0x6
r13    0x7ffc0e570d00
r14    0xe69fa0
r15    0x1
rip    0x7f299d101ccc
rflags 0x246
cs     0x[33](https://gitlab.com/michaeljs1990/-/jobs/4480933541#L33)
fs     0x0
gs     0x0

[yosifkit]

I'd suggest updating docker and libseccomp on the host. Newer base OS's use newer system calls and an older libseccomp can block them since they are unknown to it. You can verify that it is libseccomp by running the bookworm image with --security-opt seccomp=unconfined.

[rizkyekoputra]

got the same issue using golang:1.20 since yesterday, some of operation not permitted

#16 0.231 make: mkdir: Operation not permitted
#16 0.231 make: /bin/sh: Operation not permitted
#16 0.232 make: pwd: Operation not permitted
#16 0.232 make: cat: Operation not permitted
#16 0.232 make: date: Operation not permitted
#16 0.232 make: git: Operation not permitted
#16 0.232 make: /bin/sh: Operation not permitted
#16 0.232 make: pwd: Operation not permitted
#16 0.233 make: cat: Operation not permitted
#16 0.233 make: date: Operation not permitted
#16 0.233 make: git: Operation not permitted
#16 0.233 make: /bin/sh: Operation not permitted

[jame-developer]

same issue here with golang:1.20.5 but golang:1.20.5-bullseye is working

[ffjlabo]

Same issue occured with golang:1.19 👀

Maybe it's because of this fix?
cd60713

[tianon]

Yep, definitely seccomp failures:

I'd suggest updating docker and libseccomp on the host. Newer base OS's use newer system calls and an older libseccomp can block them since they are unknown to it. You can verify that it is libseccomp by running the bookworm image with --security-opt seccomp=unconfined.

[yosifkit]

Root cause: it is Docker with libseccomp; a newer syscall used in Debian Bookworm packages/libs is being blocked.

libseccomp lets you configure allowed syscalls for a process. Docker sets a default seccomp profile for all containers such that only certain syscalls are allowed and everything else is blocked (so, newer syscalls that are not yet known to libseccomp or docker are blocked).

• verify that it is libseccomp by running the Bookworm-based image with --security-opt seccomp=unconfined
  • use only as a test; it is less secure to keep running them "unconfined"
  • learn more: https://docs.docker.com/engine/security/seccomp/
• one fix:
  • update libseccomp and docker on the host running the containers
• one workaround:
  • switch to the *bullseye images (in the golang images, these will continue to be maintained/updated until the earliest of the respective Golang end of life or the next Debian release, Debian Trixie)

[michaeljs1990]

I'd suggest updating docker and libseccomp on the host. Newer base OS's use newer system calls and an older libseccomp can block them since they are unknown to it. You can verify that it is libseccomp by running the bookworm image with --security-opt seccomp=unconfined.

Sadly we were running a fairly recent OS everywhere except our CI machines. Guess it's time for an upgrade :) thanks for the explanation!

Since this is related to the host machine and not the docker image I'm not sure how relevant this ticket is. I'll leave it open but feel free to close it out since it doesn't seem like their is anything to be done on the docker image side.

[wu0407]

sloved by upgrate to docker 20.10.24

[ejain]

Stuck with an older version of Docker, but switching to a "-bullseye" image (as suggested by @yosifkit above) solved the issue for us (for now).

[eoos]

same issue here with golang:1.20.5 but golang:1.20.5-bullseye is working

Really useful!
Thx!