Skip to content

[Bug]: Deadlock between DockerClientFactory and RyukResourceReaper with JUnit 5 parallel tests #9120

Open

Description

Module

Core

Testcontainers version

1.20.1

Using the latest Testcontainers version?

Yes

Host OS

MacOS

Host Arch

arm64

Docker version

Client:
 Version:           27.1.1
 API version:       1.46
 Go version:        go1.21.12
 Git commit:        6312585
 Built:             Tue Jul 23 19:54:12 2024
 OS/Arch:           darwin/arm64
 Context:           desktop-linux

Server: Docker Desktop 4.33.0 (160616)
 Engine:
  Version:          27.1.1
  API version:      1.46 (minimum version 1.24)
  Go version:       go1.21.12
  Git commit:       cc13f95
  Built:            Tue Jul 23 19:57:14 2024
  OS/Arch:          linux/arm64
  Experimental:     false
 containerd:
  Version:          1.7.19
  GitCommit:        2bf793ef6dc9a18e00cb12efb64355c2c9d5eb41
 runc:
  Version:          1.7.19
  GitCommit:        v1.1.13-0-g58aa920
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0

What happened?

I'm attempting to run tests in parallel with JUnit 5. One test spins up a static ComposeContainer with .withLocalCompose(true) and another spins up a static KafkaContainer. This leads to a deadlock on startup, where one thread acquires the lock on RyukResourceReaper and then fails to acquire the lock in DockerClientFactory, while the other thread does the opposite.

Relevant log output

Kafka container thread:

"testcontainers-lifecycle-0" #35 [41731] daemon prio=5 os_prio=31 cpu=207.18ms elapsed=23.45s tid=0x000000012225ba00 nid=41731 waiting for monitor entry  [0x00000001735fa000]
   java.lang.Thread.State: BLOCKED (on object monitor)
	at org.testcontainers.utility.RyukResourceReaper.maybeStart(RyukResourceReaper.java:74)
	- waiting to lock <0x000000060201f118> (a org.testcontainers.utility.RyukResourceReaper)
	at org.testcontainers.utility.RyukResourceReaper.init(RyukResourceReaper.java:42)
	at org.testcontainers.DockerClientFactory.client(DockerClientFactory.java:232)
	- locked <0x000000060201ee60> (a [Ljava.lang.Object;)
	at org.testcontainers.DockerClientFactory$1.getDockerClient(DockerClientFactory.java:106)
	at com.github.dockerjava.api.DockerClientDelegate.authConfig(DockerClientDelegate.java:109)
	at org.testcontainers.containers.GenericContainer.start(GenericContainer.java:329)

Compose container thread:

"testcontainers-lifecycle-1" #37 [37891] daemon prio=5 os_prio=31 cpu=3.14ms elapsed=23.41s tid=0x0000000122254600 nid=37891 waiting for monitor entry  [0x0000000173a12000]
   java.lang.Thread.State: BLOCKED (on object monitor)
	at org.testcontainers.DockerClientFactory.client(DockerClientFactory.java:185)
	- waiting to lock <0x000000060201ee60> (a [Ljava.lang.Object;)
	at org.testcontainers.DockerClientFactory$1.getDockerClient(DockerClientFactory.java:106)
	at com.github.dockerjava.api.DockerClientDelegate.authConfig(DockerClientDelegate.java:109)
	at org.testcontainers.containers.GenericContainer.start(GenericContainer.java:329)
	at org.testcontainers.utility.RyukResourceReaper.maybeStart(RyukResourceReaper.java:78)
	- locked <0x000000060201f118> (a org.testcontainers.utility.RyukResourceReaper)
	at org.testcontainers.utility.RyukResourceReaper.registerLabelsFilterForCleanup(RyukResourceReaper.java:51)
	at org.testcontainers.containers.ComposeDelegate.registerContainersForShutdown(ComposeDelegate.java:247)
	at org.testcontainers.containers.ComposeContainer.start(ComposeContainer.java:125)
	- locked <0x000000060201f2a8> (a java.lang.Object)

Additional Information

No response

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions