TestKit: synchronous `TestActor` start #7772

Aaronontheweb · 2025-08-19T22:01:01Z

Changes

fixes #7770 - but this is just a prototype
supersedes, closes #7771

Checklist

For significant changes, please ensure that the following have been completed (delete if not relevant):

This change follows the Akka.NET API Compatibility Guidelines.
This change follows the Akka.NET Wire Compatibility Guidelines.
I have reviewed my own pull request.
Design discussion issue TestKit deadlock during parallel test execution with TestActor initialization #7770
Changes in public API reviewed, if any.

This reverts commit bdd77f9.

This commit resolves a deadlock that occurs when running tests in parallel, where the initial TestActor creation gets stuck during async initialization with CallingThreadDispatcher. The root cause was that SystemActorOf hardcodes async=true initialization, creating a RepointableActorRef that requires processing a Supervise system message. With CallingThreadDispatcher, this creates a circular dependency: - TestKit constructor blocks waiting for TestActor initialization - CallingThreadDispatcher only runs on the calling thread - The calling thread is blocked, so Supervise message never gets processed The solution bypasses SystemActorOf and directly calls AttachChild with async=false, enabling true synchronous initialization while preserving full system integration including supervision tree and mailbox configuration. This maintains compatibility with CallingThreadDispatcher for deterministic testing while eliminating startup deadlocks in parallel test scenarios. Resolves issue where TestProbe child actor creation and implicit sender functionality would fail due to incomplete TestActor initialization.

- Use AttachChild with isSystemService=true to exempt TestActor from serialization verification - Resolves 700+ test failures caused by UnboundedChannelWriter serialization errors

Resolves deadlock that occurs when TestKit instances are created in parallel and actors try to interact with TestActor during initialization. The issue was caused by CallingThreadDispatcher creating RepointableActorRef which requires async initialization, leading to deadlocks. Changes: - Add AttachChildWithAsync internal method to ActorCell to control sync/async actor creation - Modify TestKitBase to create TestActor synchronously (LocalActorRef) instead of async (RepointableActorRef) - Update Xunit/Xunit2 TestKits to create logger actors synchronously - Replace Ask with Tell for logger initialization to avoid synchronous wait deadlocks - Add InternalsVisibleTo for Xunit TestKits to access internal Akka methods - Maintain LoggerInitialized response for protocol compatibility (has IDeadLetterSuppression) Fixes akkadotnet#7770

Aaronontheweb

Detailed my changes

Aaronontheweb · 2025-08-20T21:38:35Z

src/contrib/testkits/Akka.TestKit.Xunit/TestKit.cs

+
+        // Create logger actor synchronously to avoid deadlock during parallel test execution
+        // Use AttachChildWithAsync with isAsync:false to create LocalActorRef instead of RepointableActorRef
+        var logger = systemImpl.Provider.SystemGuardian.Cell.AttachChildWithAsync(


Using the new AttachChildWithAsync method, which allows a top-level actor to start synchronously. Should really only be used during testing. But this means we don't have to wait for the logger to reply back - it should "just work"

This eliminates 1 source of deadlock / contention pressure when running lots of tests in parallel

Aaronontheweb · 2025-08-20T21:40:13Z

src/contrib/testkits/Akka.TestKit.Xunit2/TestKit.cs

-                loggerTask.ConfigureAwait(false).GetAwaiter().GetResult();
+                // Create logger actor synchronously to avoid deadlock during parallel test execution
+                // Use AttachChildWithAsync with isAsync:false to create LocalActorRef instead of RepointableActorRef
+                var logger = systemImpl.Provider.SystemGuardian.Cell.AttachChildWithAsync(


Same category of fix from before

Aaronontheweb · 2025-08-20T21:40:52Z

src/core/Akka.API.Tests/verify/CoreAPISpec.ApproveCore.Net.verified.txt

 [assembly: System.Runtime.CompilerServices.InternalsVisibleToAttribute("Akka.Streams.Tests")]
 [assembly: System.Runtime.CompilerServices.InternalsVisibleToAttribute("Akka.TestKit")]
 [assembly: System.Runtime.CompilerServices.InternalsVisibleToAttribute("Akka.TestKit.Tests")]
+[assembly: System.Runtime.CompilerServices.InternalsVisibleToAttribute("Akka.TestKit.Xunit")]


Needed to make the xUnit testkits friend assemblies in order to access the AttachChildWithAsync

Aaronontheweb · 2025-08-20T21:41:45Z

src/core/Akka.TestKit.Tests/TestActorRefTests/ParallelTestActorDeadlockSpec.cs

+
+namespace Akka.TestKit.Tests.TestActorRefTests
+{
+    public class ParallelTestActorDeadlockSpec


Repurposed @Arkatufus 's reproduction from akkadotnet/Akka.Hosting#643 - got it to pass with these changes. Both the logger and the test initialization changes were required.

Aaronontheweb · 2025-08-20T21:42:24Z

src/core/Akka.TestKit/TestKitBase.cs

+
+            var systemImpl = system.AsInstanceOf<ActorSystemImpl>();
+            // Use the new AttachChildWithAsync method to create TestActor synchronously
+            var testActor = systemImpl.Provider.SystemGuardian.Cell.AttachChildWithAsync(


Key fix: start the actor with this method so it's automatically initialized with an ActorCell - no need to spin and check afterwards

Aaronontheweb · 2025-08-20T21:45:45Z

src/core/Akka/Actor/ActorCell.Children.cs

+        /// <returns>A reference to the initialized child actor.</returns>
+        internal IActorRef AttachChildWithAsync(Props props, bool isSystemService, bool isAsync, string? name = null)
+        {
+            return MakeChild(props, name == null ? GetRandomActorName() : CheckName(name), isAsync, isSystemService);


The key fix - the ability to attach a system service actor and have it start synchronously.

This, very likely, screws with actor supervision if there are any crashes at startup, hence why this API is internal and should not be used outside of this one narrow case.

The test had a race condition where the PingerActor sends 'ping' to TestActor during PreStart, but the test was expecting 'test-message' first. This could cause ExpectMsgAsync to receive the wrong message and fail. Fixed by properly expecting the 'ping' message first before sending and expecting the 'test-message'.

Arkatufus

LGTM

* Force synchronous start for `TestActor` fix akkadotnet#7770 * separate creation of implicit, default `TestActor` from additional ones * force `TestActor` to start via CTD tweak instead * don't wait for `TestActor` to start * Revert "don't wait for `TestActor` to start" This reverts commit bdd77f9. * run default `TestActor` without `CallingThreadDispatcher` * fix TestKit deadlock during parallel test execution This commit resolves a deadlock that occurs when running tests in parallel, where the initial TestActor creation gets stuck during async initialization with CallingThreadDispatcher. The root cause was that SystemActorOf hardcodes async=true initialization, creating a RepointableActorRef that requires processing a Supervise system message. With CallingThreadDispatcher, this creates a circular dependency: - TestKit constructor blocks waiting for TestActor initialization - CallingThreadDispatcher only runs on the calling thread - The calling thread is blocked, so Supervise message never gets processed The solution bypasses SystemActorOf and directly calls AttachChild with async=false, enabling true synchronous initialization while preserving full system integration including supervision tree and mailbox configuration. This maintains compatibility with CallingThreadDispatcher for deterministic testing while eliminating startup deadlocks in parallel test scenarios. Resolves issue where TestProbe child actor creation and implicit sender functionality would fail due to incomplete TestActor initialization. * Fix TestKit serialization issue - Use AttachChild with isSystemService=true to exempt TestActor from serialization verification - Resolves 700+ test failures caused by UnboundedChannelWriter serialization errors * still working on synchronous `TestActor` startup * Fix TestKit deadlock during parallel test execution Resolves deadlock that occurs when TestKit instances are created in parallel and actors try to interact with TestActor during initialization. The issue was caused by CallingThreadDispatcher creating RepointableActorRef which requires async initialization, leading to deadlocks. Changes: - Add AttachChildWithAsync internal method to ActorCell to control sync/async actor creation - Modify TestKitBase to create TestActor synchronously (LocalActorRef) instead of async (RepointableActorRef) - Update Xunit/Xunit2 TestKits to create logger actors synchronously - Replace Ask with Tell for logger initialization to avoid synchronous wait deadlocks - Add InternalsVisibleTo for Xunit TestKits to access internal Akka methods - Maintain LoggerInitialized response for protocol compatibility (has IDeadLetterSuppression) Fixes akkadotnet#7770 * added API approvals * remove `EnsureTestActorReady` method * API approvals * ensure calls can't get contaminated with references * fix API approvals * Fix race condition in ParallelTestActorDeadlockSpec The test had a race condition where the PingerActor sends 'ping' to TestActor during PreStart, but the test was expecting 'test-message' first. This could cause ExpectMsgAsync to receive the wrong message and fail. Fixed by properly expecting the 'ping' message first before sending and expecting the 'test-message'.

* Force synchronous start for `TestActor` fix #7770 * separate creation of implicit, default `TestActor` from additional ones * force `TestActor` to start via CTD tweak instead * don't wait for `TestActor` to start * Revert "don't wait for `TestActor` to start" This reverts commit bdd77f9. * run default `TestActor` without `CallingThreadDispatcher` * fix TestKit deadlock during parallel test execution This commit resolves a deadlock that occurs when running tests in parallel, where the initial TestActor creation gets stuck during async initialization with CallingThreadDispatcher. The root cause was that SystemActorOf hardcodes async=true initialization, creating a RepointableActorRef that requires processing a Supervise system message. With CallingThreadDispatcher, this creates a circular dependency: - TestKit constructor blocks waiting for TestActor initialization - CallingThreadDispatcher only runs on the calling thread - The calling thread is blocked, so Supervise message never gets processed The solution bypasses SystemActorOf and directly calls AttachChild with async=false, enabling true synchronous initialization while preserving full system integration including supervision tree and mailbox configuration. This maintains compatibility with CallingThreadDispatcher for deterministic testing while eliminating startup deadlocks in parallel test scenarios. Resolves issue where TestProbe child actor creation and implicit sender functionality would fail due to incomplete TestActor initialization. * Fix TestKit serialization issue - Use AttachChild with isSystemService=true to exempt TestActor from serialization verification - Resolves 700+ test failures caused by UnboundedChannelWriter serialization errors * still working on synchronous `TestActor` startup * Fix TestKit deadlock during parallel test execution Resolves deadlock that occurs when TestKit instances are created in parallel and actors try to interact with TestActor during initialization. The issue was caused by CallingThreadDispatcher creating RepointableActorRef which requires async initialization, leading to deadlocks. Changes: - Add AttachChildWithAsync internal method to ActorCell to control sync/async actor creation - Modify TestKitBase to create TestActor synchronously (LocalActorRef) instead of async (RepointableActorRef) - Update Xunit/Xunit2 TestKits to create logger actors synchronously - Replace Ask with Tell for logger initialization to avoid synchronous wait deadlocks - Add InternalsVisibleTo for Xunit TestKits to access internal Akka methods - Maintain LoggerInitialized response for protocol compatibility (has IDeadLetterSuppression) Fixes #7770 * added API approvals * remove `EnsureTestActorReady` method * API approvals * ensure calls can't get contaminated with references * fix API approvals * Fix race condition in ParallelTestActorDeadlockSpec The test had a race condition where the PingerActor sends 'ping' to TestActor during PreStart, but the test was expecting 'test-message' first. This could cause ExpectMsgAsync to receive the wrong message and fail. Fixed by properly expecting the 'ping' message first before sending and expecting the 'test-message'.

Aaronontheweb added 7 commits August 19, 2025 13:50

Force synchronous start for TestActor

02406bc

fix akkadotnet#7770

separate creation of implicit, default TestActor from additional ones

e97b143

force TestActor to start via CTD tweak instead

539867d

don't wait for TestActor to start

bdd77f9

Revert "don't wait for TestActor to start"

62e7cf1

This reverts commit bdd77f9.

run default TestActor without CallingThreadDispatcher

f4d9968

Aaronontheweb force-pushed the spike-synchronous-testactor-start branch from fedcd76 to c152a8f Compare August 20, 2025 00:56

Fix TestKit serialization issue

50bf0db

- Use AttachChild with isSystemService=true to exempt TestActor from serialization verification - Resolves 700+ test failures caused by UnboundedChannelWriter serialization errors

Aaronontheweb force-pushed the spike-synchronous-testactor-start branch from c152a8f to 50bf0db Compare August 20, 2025 01:07

Aaronontheweb added 4 commits August 20, 2025 15:41

still working on synchronous TestActor startup

11a5694

added API approvals

53a2588

Merge branch 'dev' into spike-synchronous-testactor-start

b957f75

Aaronontheweb added akka-testkit Akka.NET Testkit issues perf labels Aug 20, 2025

Aaronontheweb added 2 commits August 20, 2025 16:20

remove EnsureTestActorReady method

7474f52

API approvals

6a71d66

Aaronontheweb commented Aug 20, 2025

View reviewed changes

Aaronontheweb marked this pull request as ready for review August 20, 2025 21:46

Aaronontheweb changed the title ~~Spike: synchronous TestActor start~~ TestKit: synchronous TestActor start Aug 20, 2025

Aaronontheweb added 5 commits August 20, 2025 19:38

ensure calls can't get contaminated with references

6e928a1

Merge branch 'dev' into spike-synchronous-testactor-start

c6bc56a

fix API approvals

303088e

Merge branch 'dev' into spike-synchronous-testactor-start

b23b8ab

Arkatufus approved these changes Aug 21, 2025

View reviewed changes

Aaronontheweb merged commit bf44386 into akkadotnet:dev Aug 21, 2025
8 of 11 checks passed

Aaronontheweb deleted the spike-synchronous-testactor-start branch August 21, 2025 19:12

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

TestKit: synchronous `TestActor` start #7772

TestKit: synchronous `TestActor` start #7772

Uh oh!

Aaronontheweb commented Aug 19, 2025 •

edited

Loading

Uh oh!

Aaronontheweb left a comment

Uh oh!

Aaronontheweb Aug 20, 2025

Uh oh!

Aaronontheweb Aug 20, 2025

Uh oh!

Aaronontheweb Aug 20, 2025

Uh oh!

Aaronontheweb Aug 20, 2025

Uh oh!

Aaronontheweb Aug 20, 2025

Uh oh!

Aaronontheweb Aug 20, 2025

Uh oh!

Aaronontheweb Aug 20, 2025

Uh oh!

Arkatufus left a comment

Uh oh!

Uh oh!

Uh oh!

TestKit: synchronous TestActor start #7772

TestKit: synchronous TestActor start #7772

Uh oh!

Conversation

Aaronontheweb commented Aug 19, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Changes

Checklist

Uh oh!

Aaronontheweb left a comment

Choose a reason for hiding this comment

Uh oh!

Aaronontheweb Aug 20, 2025

Choose a reason for hiding this comment

Uh oh!

Aaronontheweb Aug 20, 2025

Choose a reason for hiding this comment

Uh oh!

Aaronontheweb Aug 20, 2025

Choose a reason for hiding this comment

Uh oh!

Aaronontheweb Aug 20, 2025

Choose a reason for hiding this comment

Uh oh!

Aaronontheweb Aug 20, 2025

Choose a reason for hiding this comment

Uh oh!

Aaronontheweb Aug 20, 2025

Choose a reason for hiding this comment

Uh oh!

Aaronontheweb Aug 20, 2025

Choose a reason for hiding this comment

Uh oh!

Arkatufus left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

TestKit: synchronous `TestActor` start #7772

TestKit: synchronous `TestActor` start #7772

Aaronontheweb commented Aug 19, 2025 •

edited

Loading