Blocker found while bumping deps (#3348)
Marten 9.13.0-stable and 9.14.0 hang Wolverine's daemon catch-up under
UseWolverineManagedEventSubscriptionDistribution. These MartenTests.TestHelpers tests time out:
catch_up_when_using_wolverine_distribution (main + ancillary store)
catch_up_then_restart
second_stage_waiting
catch_up_and_then_do_nothing
Only the currently-pinned 9.13.0-alpha.4 passes, so #3348 keeps Marten at alpha.4.
Root cause
Marten #4904 (shipped in 9.14.0) intentionally changed
ForceAllMartenDaemonActivityToCatchUpAsync so that when the store is
DaemonMode.ExternallyManaged (what Wolverine-managed distribution uses) it no longer drives
Stop→CatchUp→Start. Instead it degrades to a read-only WaitForNonStaleDataAsync wait, to avoid a
ProgressionProgressOutOfOrderException race where both Marten's forced catch-up and Wolverine's
externally-owned agent advance the same mt_event_progression row. Marten's own comment:
The proper active-quiesce — the external coordinator suspending agent assignment during a forced
catch-up — lives in that external coordinator, not here; tracked on #4904.
Wolverine's PauseThenCatchUpOnMartenDaemonActivity (src/Persistence/Wolverine.Marten/TestingExtensions.cs)
pauses the daemons and then relies on ForceAll to drive them forward. Under 9.14.0 the call just
waits; the paused agents never advance; it blocks until the tracked-session timeout.
What Wolverine needs
Implement the "external coordinator" active-quiesce catch-up that Marten #4904 punted to us — most
likely on WolverineProjectionCoordinator:
- Suspend Wolverine agent assignment/reassignment for the store's subscription agents.
- Force those agents (which Wolverine owns) to catch up to the current high-water mark.
- Resume normal assignment.
Then update PauseThenCatchUpOnMartenDaemonActivity (and any production catch-up path) to use that
coordinator-driven catch-up under managed distribution instead of Marten's now-passive ForceAll.
Once done, bump Marten to 9.14.0+ and re-run the catch-up suite.
Repro
Check out #3348 with Marten pinned to 9.14.0 (or 9.13.0 stable) and run:
dotnet test src/Persistence/MartenTests/MartenTests.csproj --framework net9.0 --filter "FullyQualifiedName~TestHelpers.catch_up_when_using_wolverine_distribution"
— fails (TimeoutException) on 9.14.0/9.13.0-stable, passes on 9.13.0-alpha.4.
Blocker found while bumping deps (#3348)
Marten 9.13.0-stable and 9.14.0 hang Wolverine's daemon catch-up under
UseWolverineManagedEventSubscriptionDistribution. TheseMartenTests.TestHelperstests time out:catch_up_when_using_wolverine_distribution(main + ancillary store)catch_up_then_restartsecond_stage_waitingcatch_up_and_then_do_nothingOnly the currently-pinned 9.13.0-alpha.4 passes, so #3348 keeps Marten at alpha.4.
Root cause
Marten #4904 (shipped in 9.14.0) intentionally changed
ForceAllMartenDaemonActivityToCatchUpAsyncso that when the store isDaemonMode.ExternallyManaged(what Wolverine-managed distribution uses) it no longer drivesStop→CatchUp→Start. Instead it degrades to a read-only
WaitForNonStaleDataAsyncwait, to avoid aProgressionProgressOutOfOrderExceptionrace where both Marten's forced catch-up and Wolverine'sexternally-owned agent advance the same
mt_event_progressionrow. Marten's own comment:Wolverine's
PauseThenCatchUpOnMartenDaemonActivity(src/Persistence/Wolverine.Marten/TestingExtensions.cs)pauses the daemons and then relies on
ForceAllto drive them forward. Under 9.14.0 the call justwaits; the paused agents never advance; it blocks until the tracked-session timeout.
What Wolverine needs
Implement the "external coordinator" active-quiesce catch-up that Marten #4904 punted to us — most
likely on
WolverineProjectionCoordinator:Then update
PauseThenCatchUpOnMartenDaemonActivity(and any production catch-up path) to use thatcoordinator-driven catch-up under managed distribution instead of Marten's now-passive
ForceAll.Once done, bump Marten to 9.14.0+ and re-run the catch-up suite.
Repro
Check out #3348 with Marten pinned to 9.14.0 (or 9.13.0 stable) and run:
dotnet test src/Persistence/MartenTests/MartenTests.csproj --framework net9.0 --filter "FullyQualifiedName~TestHelpers.catch_up_when_using_wolverine_distribution"— fails (TimeoutException) on 9.14.0/9.13.0-stable, passes on 9.13.0-alpha.4.