Top level exception does not stop docker container

[philipszalla-cpit]

Describe the bug

We switched our asp.net application from .NET 8.0 to .NET 9.0. We discovered an issue with .NET 9.0:

When an exception occurs in the Program.cs, the dotnet process does not terminate and the container continues running and uses 100% CPU.

Expected behaviour:

1. Top level exception occurs
2. Docker container stops

Current behaviour:

1. Top level exception occurs
2. Docker container is still running
3. Docker container CPU usage stays at 100%

Which .NET image(s) are you using?

mcr.microsoft.com/dotnet/runtime:9.0@sha256:c31b1ae5818421e20c0792e9e169004b1ca637486ef85b1c662921d82b053a03

Steps to reproduce

I can reproduce this using a new ASP.NET 9.0 project or just a .NET 9.0 command-line project. If I run the same project outside of a container, the process stops correctly.

Environment:

• MacBook Pro M3 Pro (Arm)
• AWS EC2 t4g (Arm)

Test.csproj:

<Project Sdk="Microsoft.NET.Sdk">

    <PropertyGroup>
        <OutputType>Exe</OutputType>
        <TargetFramework>net9.0</TargetFramework>
        <ImplicitUsings>enable</ImplicitUsings>
        <Nullable>enable</Nullable>
    </PropertyGroup>

    <ItemGroup>
      <Content Include="..\..\.dockerignore">
        <Link>.dockerignore</Link>
      </Content>
    </ItemGroup>

</Project>

Program.cs:

Console.WriteLine("Hello, World!");

throw new Exception("test");

Dockerfile:

FROM mcr.microsoft.com/dotnet/runtime:9.0 AS base
USER $APP_UID
WORKDIR /app

FROM mcr.microsoft.com/dotnet/sdk:9.0 AS build
ARG BUILD_CONFIGURATION=Release
WORKDIR /src
COPY ["Test.csproj", "."]
RUN dotnet restore "Test.csproj"
COPY . .
WORKDIR "/src"
RUN dotnet build "Test.csproj" -c $BUILD_CONFIGURATION -o /app/build

FROM build AS publish
ARG BUILD_CONFIGURATION=Release
RUN dotnet publish "Test.csproj" -c $BUILD_CONFIGURATION -o /app/publish /p:UseAppHost=false

FROM base AS final
WORKDIR /app
COPY --from=publish /app/publish .
ENTRYPOINT ["dotnet", "Test.dll"]

.dockerignore:

**/.dockerignore
**/.env
**/.git
**/.gitignore
**/.project
**/.settings
**/.toolstarget
**/.vs
**/.vscode
**/.idea
**/*.*proj.user
**/*.dbmdl
**/*.jfm
**/azds.yaml
**/bin
**/charts
**/docker-compose*
**/Dockerfile*
**/node_modules
**/npm-debug.log
**/obj
**/secrets.dev.yaml
**/values.dev.yaml
LICENSE
README.md

Run this:

docker build -t test .
docker run --rm test

Other information

The issue is reproducible using a new .NET project generated by JetBrains Rider.

Host OSs are:

• macOS 15.1.1 (24B91)
• Amazon Linux 2023 (used with AWS EKS)

Output of docker version

Client:
 Version:           27.4.0
 API version:       1.47
 Go version:        go1.22.10
 Git commit:        bde2b89
 Built:             Sat Dec  7 10:35:43 2024
 OS/Arch:           darwin/arm64
 Context:           desktop-linux

Server: Docker Desktop 4.37.2 (179585)
 Engine:
  Version:          27.4.0
  API version:      1.47 (minimum version 1.24)
  Go version:       go1.22.10
  Git commit:       92a8393
  Built:            Sat Dec  7 10:38:33 2024
  OS/Arch:          linux/arm64
  Experimental:     false
 containerd:
  Version:          1.7.21
  GitCommit:        472731909fa34bd7bc9c087e4c27943f9835f111
 runc:
  Version:          1.1.13
  GitCommit:        v1.1.13-0-g58aa920
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0

Output of docker info

Client:
 Version:    27.4.0
 Context:    desktop-linux
 Debug Mode: false
 Plugins:
  ai: Ask Gordon - Docker Agent (Docker Inc.)
    Version:  v0.5.1
    Path:     /Users/ps/.docker/cli-plugins/docker-ai
  buildx: Docker Buildx (Docker Inc.)
    Version:  v0.19.2-desktop.1
    Path:     /Users/ps/.docker/cli-plugins/docker-buildx
  compose: Docker Compose (Docker Inc.)
    Version:  v2.31.0-desktop.2
    Path:     /Users/ps/.docker/cli-plugins/docker-compose
  debug: Get a shell into any image or container (Docker Inc.)
    Version:  0.0.37
    Path:     /Users/ps/.docker/cli-plugins/docker-debug
  desktop: Docker Desktop commands (Beta) (Docker Inc.)
    Version:  v0.1.0
    Path:     /Users/ps/.docker/cli-plugins/docker-desktop
  dev: Docker Dev Environments (Docker Inc.)
    Version:  v0.1.2
    Path:     /Users/ps/.docker/cli-plugins/docker-dev
  extension: Manages Docker extensions (Docker Inc.)
    Version:  v0.2.27
    Path:     /Users/ps/.docker/cli-plugins/docker-extension
  feedback: Provide feedback, right in your terminal! (Docker Inc.)
    Version:  v1.0.5
    Path:     /Users/ps/.docker/cli-plugins/docker-feedback
  init: Creates Docker-related starter files for your project (Docker Inc.)
    Version:  v1.4.0
    Path:     /Users/ps/.docker/cli-plugins/docker-init
  sbom: View the packaged-based Software Bill Of Materials (SBOM) for an image (Anchore Inc.)
    Version:  0.6.0
    Path:     /Users/ps/.docker/cli-plugins/docker-sbom
  scout: Docker Scout (Docker Inc.)
    Version:  v1.15.1
    Path:     /Users/ps/.docker/cli-plugins/docker-scout

Server:
 Containers: 26
  Running: 7
  Paused: 0
  Stopped: 19
 Images: 70
 Server Version: 27.4.0
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Using metacopy: false
  Native Overlay Diff: true
  userxattr: false
 Logging Driver: json-file
 Cgroup Driver: cgroupfs
 Cgroup Version: 2
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local splunk syslog
 CDI spec directories:
  /etc/cdi
  /var/run/cdi
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: 472731909fa34bd7bc9c087e4c27943f9835f111
 runc version: v1.1.13-0-g58aa920
 init version: de40ad0
 Security Options:
  seccomp
   Profile: unconfined
  cgroupns
 Kernel Version: 6.10.14-linuxkit
 Operating System: Docker Desktop
 OSType: linux
 Architecture: aarch64
 CPUs: 11
 Total Memory: 17.54GiB
 Name: docker-desktop
 ID: 1808eb13-9c72-496f-9afe-c70273fb9be6
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 HTTP Proxy: http.docker.internal:3128
 HTTPS Proxy: http.docker.internal:3128
 No Proxy: hubproxy.docker.internal
 Labels:
  com.docker.desktop.address=unix:///Users/ps/Library/Containers/com.docker.docker/Data/docker-cli.sock
 Experimental: false
 Insecure Registries:
  hubproxy.docker.internal:5555
  127.0.0.0/8
 Live Restore Enabled: false

WARNING: daemon is not using the default seccomp profile

[lbussell]

I can't reproduce this using your precise instructions/files on my Windows AMD64 machine - @richlander can you give this a try on your ARM Mac?

PS> docker run --rm test
Hello, World!
Unhandled exception. System.Exception: test
   at Program.<Main>$(String[] args) in /src/Program.cs:line 3
PS>

One thing I noticed is that mcr.microsoft.com/dotnet/runtime:9.0@sha256:c31b1ae5818421e20c0792e9e169004b1ca637486ef85b1c662921d82b053a03 is a manifest list tag. Can you validate which architecture you're pulling and running, and ensure you aren't running under any kind of emulation/translation? The latest ARM64 specific tag would be mcr.microsoft.com/dotnet/runtime:9.0.2-bookworm-slim-arm64v8.

[richlander]

I don't have an Arm Mac anymore. I gave it back.

[akoken]

I can confirm that it doesn't work for me either. Also tried with the image mcr.microsoft.com/dotnet/runtime:9.0.2-bookworm-slim-arm64v8.

OS: Mac Studio M2 Max
Docker Version: (Rosetta enabled)

docker version

Client:
 Version:           27.5.1
 API version:       1.47
 Go version:        go1.22.11
 Git commit:        9f9e405
 Built:             Wed Jan 22 13:37:19 2025
 OS/Arch:           darwin/arm64
 Context:           desktop-linux

Server: Docker Desktop 4.38.0 (181591)
 Engine:
  Version:          27.5.1
  API version:      1.47 (minimum version 1.24)
  Go version:       go1.22.11
  Git commit:       4c9b3b0
  Built:            Wed Jan 22 13:41:25 2025
  OS/Arch:          linux/arm64
  Experimental:     false
 containerd:
  Version:          1.7.25
  GitCommit:        bcc810d6b9066471b0b6fa75f557a15a1cbf31bb
 runc:
  Version:          1.1.12
  GitCommit:        v1.1.12-0-g51d5e946
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0

[lbussell]

@jkotas could this be a runtime issue? Or do you have any troubleshooting tips to rule out the runtime?

[jkotas]

This may be related to #109779 . cc @janvorli @mikem8361

@lbussell I do not have access to arm64 box at the moment. Could you please check:

• Does the problem repro on live .NET 10?

• Does the problem go away if you pass --init argument to docker run (workaround described in App hangs after taking crashdump in official .NET 9 x64 glibc docker images #109779 (comment))?

[lbussell]

@mthalman is working on a repro, otherwise I can try later this evening with my Mac.

If anyone else wants to check with .NET 10, the equivalent image(s) would be mcr.microsoft.com/dotnet/nightly/[runtime|sdk]:10.0-preview-trixie-slim-arm64v8.

[philipszalla-cpit]

@lbussell

I can't reproduce this using your precise instructions/files on my Windows AMD64 machine - @richlander can you give this a try on your ARM Mac?

PS> docker run --rm test
Hello, World!
Unhandled exception. System.Exception: test
   at Program.<Main>$(String[] args) in /src/Program.cs:line 3
PS>

One thing I noticed is that mcr.microsoft.com/dotnet/runtime:9.0@sha256:c31b1ae5818421e20c0792e9e169004b1ca637486ef85b1c662921d82b053a03 is a manifest list tag. Can you validate which architecture you're pulling and running, and ensure you aren't running under any kind of emulation/translation? The latest ARM64 specific tag would be mcr.microsoft.com/dotnet/runtime:9.0.2-bookworm-slim-arm64v8.

Using the tag mcr.microsoft.com/dotnet/runtime:9.0.2-bookworm-slim-arm64v8 causes the same problem.

@jkotas

This may be related to dotnet/runtime#109779 . cc @janvorli @mikem8361

@lbussell I do not have access to arm64 box at the moment. Could you please check:

• Does the problem repro on live .NET 10?
• Does the problem go away if you pass --init argument to docker run (workaround described in App hangs after taking crashdump in official .NET 9 x64 glibc docker images runtime#109779 (comment))?

• Yes, with .NET 10 (mcr.microsoft.com/dotnet/nightly/runtime:10.0-preview-trixie-slim-arm64v8) I have the same issue.
• And yes, the problem goes away if I pass --init to docker run. But this is no workaround for us, as we are using kuberentes and after a short research I couldn't find any way to get this behaviour (--init) with the CRI.

[akoken]

This solution may not be the most elegant, but it should work.

FROM mcr.microsoft.com/dotnet/runtime:9.0 AS base
RUN apt-get update && apt-get install -y tini
USER $APP_UID
WORKDIR /app

FROM mcr.microsoft.com/dotnet/sdk:9.0 AS build
ARG BUILD_CONFIGURATION=Release
WORKDIR /src
COPY ["Test.csproj", "."]
RUN dotnet restore "Test.csproj"
COPY . .
WORKDIR "/src"
RUN dotnet build "Test.csproj" -c $BUILD_CONFIGURATION -o /app/build

FROM build AS publish
ARG BUILD_CONFIGURATION=Release
RUN dotnet publish "Test.csproj" -c $BUILD_CONFIGURATION -o /app/publish /p:UseAppHost=false

FROM base AS final
WORKDIR /app
COPY --from=publish /app/publish .
ENTRYPOINT ["/usr/bin/tini", "--"]
CMD ["dotnet", "Test.dll"]

[mthalman]

I can repro this on my MacBook. The suggested workaround of installing tini doesn't resolve the issue for me. Using the --init option does resolve it.

I backtracked to see when this behavior first started happening. It looks like it was introduced with the release of 9.0 RC1. Works fine in 9.0 Preview 7 (the previous release).

[lbussell]

I was also able to repro on .NET 9 and 10. So it seems like #110379 may not have worked?

@mthalman, the tini workaround does work for me on .NET 9 and 10 as well.

[jkotas]

I backtracked to see when this behavior first started happening. It looks like it was introduced with the release of 9.0 RC1. Works fine in 9.0 Preview 7 (the previous release).

The problem could have been introduced by change in either .NET runtime or in the Linux distro. It would be useful to eliminate the later as a possibility. Could you please check whether it repros in the latest .NET 9.0 container with dotnet runtime overwritten with .NET 9 Preview 7 bits?

[mthalman]

Could you please check whether it repros in the latest .NET 9.0 container with dotnet runtime overwritten with .NET 9 Preview 7 bits?

It still does not repro using the latest distro w/ P7 bits. Here's the Dockerfile I used to check that:

FROM mcr.microsoft.com/dotnet/runtime:9.0.0-preview.7 AS base

FROM mcr.microsoft.com/dotnet/runtime-deps:9.0 AS distro

COPY --from=base /usr/share/dotnet /usr/share/dotnet
RUN ln -s /usr/share/dotnet/dotnet /usr/bin/dotnet

USER $APP_UID
WORKDIR /app

FROM mcr.microsoft.com/dotnet/sdk:9.0.100-preview.7 AS build
ARG BUILD_CONFIGURATION=Release
WORKDIR /src
COPY ["Test.csproj", "."]
RUN dotnet restore "Test.csproj"
COPY . .
WORKDIR "/src"
RUN dotnet build "Test.csproj" -c $BUILD_CONFIGURATION -o /app/build

FROM build AS publish
ARG BUILD_CONFIGURATION=Release
RUN dotnet publish "Test.csproj" -c $BUILD_CONFIGURATION -o /app/publish /p:UseAppHost=false

FROM distro AS final
WORKDIR /app
COPY --from=publish /app/publish .
ENTRYPOINT ["dotnet", "Test.dll"]