Unhandled exception causes docker container to hang on ARM64

[blaskoa]

Description

Dotnet applications running in docker containers on arm64 stop responding when unhandled exception occurs and keep a 100% utilization on a single CPU core (as measured by docker stats).

I tested the issue on the following environments:

• Jetson Xavier NX Developer Kit
• M1 MacBook
• Windows/WSL with qemu emulation of arm64 (default emulation setup by docker desktop)

The only difference in the environments is that with qemu an additional line is present in the output and shell stops responding to Ctrl+C signals:

qemu: uncaught target signal 6 (Aborted) - core dumped

Issue is only present when the dotnet process is set as docker entrypoint e.g. ENTRYPOINT [ "dotnet", "arm-dotnet-repro.dll" ]
Issue is NOT present when running the dotnet binaries natively outside of docker containers, directly on the host.
Issue is NOT present when wrapping the entrypoint with shell script (see reproduction repo).
Issue is NOT present when the dotnet binary is exectued via interactive shell in docker container e.g. docker run --rm -it --entrypoint bash ex-repro:arm64-1 and then inside the container shell call dotnet arm-dotnet-repro.dll

The behavior is present on dotnet 5 and dotnet 6 runtimes (I just tested these ones).

Reproduction Steps

see https://github.com/blaskoa/arm-dotnet-repro for minimal reproducer with some available workarounds I found

Expected behavior

On unhandled exception the container should exit with non-0 exit code.

Basically the same as it behaves on x64

Actual behavior

On unhandled exception the container keeps "running" and utilizes 1 CPU core to 100%

Regression?

No response

Known Workarounds

• Using a shell script as an entrypoint in docker container
• Using a global exception handler

see https://github.com/blaskoa/arm-dotnet-repro
some of the workarounds have some side effects which might render them unsuitable

Configuration

No response

Other information

No response

I couldn't figure out the best area label to add to this issue. If you have write-permissions please help me learn by adding exactly one area label.

[janvorli]

I can confirm the bad behavior. I am investigating it.

[janvorli]

This is quite strange. The hang occurs in the abort implementation in the glibc here: https://code.woboq.org/userspace/glibc/stdlib/abort.c.html#107
It keeps looping on the ABORT_INSTRUCTION, invoking the SIGTRAP handler and then retrying the instruction again and again forever. It also means that the SIGABRT didn't abort the process as expected, as that's tried before the ABORT_INSTRUCTION is used.

[tmds]

It keeps looping on the ABORT_INSTRUCTION, invoking the SIGTRAP handler and then retrying the instruction again and again forever. It also means that the SIGABRT didn't abort the process as expected, as that's tried before the ABORT_INSTRUCTION is used.

@janvorli how does SIGTRAP come into play here? Or did you mean SIGABRT?

Does it happen also with a simple C program that calls abort?

[janvorli]

The abort instruction is an invalid instruction that the abort in the standard c library seems to use as a way to make the process exit as a last resort if SIGABRT didn't have any effect. So it is handled by SIGTRAP. At least that's what I've figured from the abort source code.
I don't think it would happen with a simple C test calling abort. I think it is somehow related to how we setup / cleanup SIGABRT (and maybe also SIGTRAP) handler

[bgever]

This seems to be related to #69923. Which causes also other runtime issues on my M1/Aarch64 MacBook Pro.

When adding the below environment variable into the Dockerfile, the hang no longer happens and the container exits properly upon an unhandled exception.

ENV COMPlus_ZapDisable=1

UPDATE: Apologies, I had a cached layer that had an explicit Environment.Exit(1); statement before, the above doesn't work.

Thank you for your example repo @blaskoa, the AppDomain.CurrentDomain.UnhandledException with explicit exit, works great for me.

[mangod9]

Hi @AntonLapounov , since Jan is out could you please check if this repros on .NET 6? If so it might not block 7.

[AntonLapounov]

From the original post:

The behavior is present on dotnet 5 and dotnet 6 runtimes (I just tested these ones).