JAVA-5950 Update Transactions Convenient API with exponential backoff on retries #1852

nhachicha · 2025-12-09T13:47:53Z

Relevant specification changes:

DRIVERS-1934: withTransaction API retries too frequently (#1851)
DRIVERS-1934: clarify drivers back off before all transaction retries (#1868) - the change has not yet been addressed in this PR, because it was introduced after creation of the PR.
DRIVERS-3364: Fix Retry Backoff is Enforced prose test. - the change has not yet been addressed in this PR, because it was introduced after creation of the PR.

Copilot

Pull request overview

This PR implements exponential backoff with jitter for transaction retries in MongoDB's withTransaction convenience API. The implementation adds a configurable backoff mechanism that applies delays between retry attempts when transient transaction errors occur, following the MongoDB specification with a growth factor of 1.5 for transactions.

Key Changes

Introduces ExponentialBackoff utility class with factory methods for transaction retries (5ms base, 500ms max, 1.5x growth) and command retries (100ms base, 10s max, 2.0x growth)
Integrates backoff logic into ClientSessionImpl.withTransaction() to delay between retry attempts
Adjusts test configuration to verify backoff behavior with multiple retry attempts

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 2 comments.

File	Description
driver-core/src/main/com/mongodb/internal/ExponentialBackoff.java	New utility class implementing exponential backoff with jitter using ThreadLocalRandom
driver-sync/src/main/com/mongodb/client/internal/ClientSessionImpl.java	Adds backoff delay before transaction retries and uses CSOT timeout when available
driver-core/src/test/unit/com/mongodb/internal/ExponentialBackoffTest.java	Comprehensive unit tests validating backoff calculations, growth factors, and maximum caps
driver-sync/src/test/functional/com/mongodb/client/WithTransactionProseTest.java	New functional test verifying exponential backoff behavior and adjusted existing test configuration

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

driver-sync/src/main/com/mongodb/client/internal/ClientSessionImpl.java

Copilot · 2025-12-09T14:25:58Z

driver-sync/src/test/functional/com/mongodb/client/WithTransactionProseTest.java

+            AtomicInteger retryCount = new AtomicInteger(0);
+
+            session.withTransaction(() -> {
+                retryCount.incrementAndGet();  // Count the attempt before the operation that might fail


The test verifies the retry count but does not validate that exponential backoff delays are actually applied. Consider measuring elapsed time and asserting minimum expected delays to ensure backoff is functioning correctly. For example, with 3 retries at delays of ~5ms, ~7.5ms, and ~11.25ms, the total elapsed time should be at least the sum of minimum expected delays.

ExponentialBackoffTest covers these unit tests already.

dariakp · 2025-12-17T19:43:50Z

@nhachicha Please take note of mongodb/specifications#1868

stIncMale

I haven't reviewed ExponentialBackoffTest, because it depends on ExponentialBackoff, where I left many suggestions.
I haven't reviewed ClientSessionImpl, because it has to implement the new specification change DRIVERS-1934: clarify drivers back off before all transaction retries (#1868).
The last reviewed commit is 90ec4d5.

driver-core/src/main/com/mongodb/internal/ExponentialBackoff.java

driver-sync/src/test/functional/com/mongodb/client/WithTransactionProseTest.java

driver-sync/src/main/com/mongodb/client/internal/ClientSessionImpl.java

dariakp · 2026-01-08T19:46:46Z

@stIncMale @nhachicha Flagging one more relevant spec test adjustment here: mongodb/specifications#1876

stIncMale · 2026-01-08T20:04:04Z

@dariakp Thank you for the heads up, I updated the PR description.

…ff on retries

…Impl.java Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

…s exceeded (ex operationContext.getTimeoutContext().getReadTimeoutMS())

…odb/specifications#1868

…tionProseTest.java Co-authored-by: Valentin Kovalenko <valentin.male.kovalenko@gmail.com>

Co-authored-by: Valentin Kovalenko <valentin.male.kovalenko@gmail.com>

stIncMale

This is a partial review, where I reviewed only ExponentialBackoff.java.

The last reviewed commit is 36ecbf9.

driver-core/src/main/com/mongodb/internal/time/ExponentialBackoff.java

driver-core/src/main/com/mongodb/internal/ExponentialBackoff.java

driver-core/src/main/com/mongodb/internal/time/ExponentialBackoff.java

stIncMale

This is a continuation of #1852 (review).

This is a partial review, where I reviewed only ClientSessionImpl.java in addition to ExponentialBackoff.java.

The last reviewed commit is 36ecbf9.

driver-sync/src/main/com/mongodb/client/internal/ClientSessionImpl.java

driver-sync/src/test/functional/com/mongodb/client/WithTransactionProseTest.java

…mmediate retry without backoff.

stIncMale

The last reviewed commit is c97073e.

The test changes have not been re-reviewed. I'll re-review them soon:

ExponentialBackoffTest
AbstractClientSideOperationsTimeoutProseTest
WithTransactionProseTest.testRetryBackoffIsEnforced/testExponentialBackoffOnTransientError

stIncMale

The last reviewed commit is f0973a1.

I reviewed everything except for AbstractClientSideOperationsTimeoutProseTest.java.

stIncMale · 2026-02-02T21:37:33Z

driver-core/src/test/unit/com/mongodb/internal/time/ExponentialBackoffTest.java

+import static org.junit.jupiter.api.Assertions.assertEquals;
+import static org.junit.jupiter.api.Assertions.assertTrue;
+
+public class ExponentialBackoffTest {


Suggested change

public class ExponentialBackoffTest {

class ExponentialBackoffTest {

stIncMale · 2026-02-02T21:50:38Z

driver-core/src/test/unit/com/mongodb/internal/time/ExponentialBackoffTest.java

+            assertTrue(backoff >= 0 && backoff <= Math.round(expectedMaxValues[attemptNumber - 1]),
+                String.format("Attempt %d: backoff should be 0-%d ms, got: %d", attemptNumber,
+                        Math.round(expectedMaxValues[attemptNumber - 1]), backoff));


Let's introduce the expectedBackoff variable to avoid repeating Math.round(expectedMaxValues[attemptNumber - 1]), this also makes the assertion more readable.

I am proposing to replace 0-%d ms with between 0 ms and %d ms. Seeing something like "should be 0-5 ms" is a bit weird. Either plain english (what I suggested), or the proper math notation seems better.

Suggested change

assertTrue(backoff >= 0 && backoff <= Math.round(expectedMaxValues[attemptNumber - 1]),

String.format("Attempt %d: backoff should be 0-%d ms, got: %d", attemptNumber,

Math.round(expectedMaxValues[attemptNumber - 1]), backoff));

long expectedBackoff = Math.round(expectedMaxValues[attemptNumber - 1]);

assertTrue(backoff >= 0 && backoff <= expectedBackoff,

String.format("Attempt %d: backoff should be between 0 ms and %d ms, got: %d", attemptNumber, expectedBackoff, backoff));

stIncMale · 2026-02-02T21:54:05Z

driver-core/src/test/unit/com/mongodb/internal/time/ExponentialBackoffTest.java

+    @Test
+    void testTransactionRetryBackoffRespectsMaximum() {
+        // Even at high attempt numbers, backoff should never exceed 500ms
+        for (int attemptNumber = 1; attemptNumber < 26; attemptNumber++) {
+            long backoff = ExponentialBackoff.calculateTransactionBackoffMs(attemptNumber);
+            assertTrue(backoff >= 0 && backoff <= 500,
+                String.format("Attempt %d: backoff should be capped at 500 ms, got: %d ms", attemptNumber, backoff));
+        }
+    }


Let's make ExponentialBackoff.TRANSACTION_MAX_MS package-access and annotate with @VisibleForTesting(otherwise = PRIVATE). Then use TRANSACTION_MAX_MS here in both the comment (as ExponentialBackoff.TRANSACTION_MAX_MS) and the code & the format argument. This way a reader does not have to look where the magic number "500" comes from, and if it ever changes, the test won't need to be updated.

stIncMale · 2026-02-02T21:57:31Z

driver-core/src/test/unit/com/mongodb/internal/time/ExponentialBackoffTest.java

+
+    @Test
+    void testTransactionRetryBackoff() {
+        // Test that the backoff sequence follows the expected pattern with growth factor 1.5


We duplicate the value of TRANSACTION_GROWTH here, but not the other parameters TRANSACTION_BASE_MS and TRANSACTION_MAX_MS. We should either refer to all of them (without duplicating their values), or to none. I propose not to mention the parameters at all, because we simply test the calculateTransactionBackoffMs method, and the parameters are implied by the method.

The same suggestion applies to the Expected backoffs with jitter=1.0 and growth factor 1.5 comment in testCustomJitter.

stIncMale · 2026-02-02T21:58:42Z

driver-core/src/test/unit/com/mongodb/internal/time/ExponentialBackoffTest.java

+public class ExponentialBackoffTest {
+
+    @Test
+    void testTransactionRetryBackoff() {


The method we test here is called calculateTransactionBackoffMs.

It does not have "retry" in the name.

Shouldn't the test name be testCalculateTransactionBackoffMs?

The same applies to the testTransactionRetryBackoffRespectsMaximum test name.

stIncMale · 2026-02-03T15:26:27Z

driver-sync/src/test/functional/com/mongodb/client/WithTransactionProseTest.java

-    private boolean canRunTests() {
+    /**
+     * See
+     * <a href="https://github.com/mongodb/specifications/blob/master/source/transactions-convenient-api/tests/README.md#retry-backoff-is-enforceds">Convenient API Prose Tests</a>.


Suggested change

* <a href="https://github.com/mongodb/specifications/blob/master/source/transactions-convenient-api/tests/README.md#retry-backoff-is-enforceds">Convenient API Prose Tests</a>.

* <a href="https://github.com/mongodb/specifications/blob/master/source/transactions-convenient-api/tests/README.md#retry-backoff-is-enforced">Retry Backoff is Enforced</a>.

stIncMale · 2026-02-03T15:37:24Z

driver-sync/src/test/functional/com/mongodb/client/WithTransactionProseTest.java

+        failPointAdminDb.runCommand(Document.parse("{'configureFailPoint': 'failCommand', 'mode': {'times': 13}, " + "'data': {'failCommands': ['commitTransaction'], 'errorCode': 251}}"));
+
+        long withBackoffTime;
+        try (ClientSession session = client.startSession()) {
+            long startNanos = System.nanoTime();
+            session.withTransaction(() -> {
+                collection.insertOne(session, Document.parse("{ _id : 'backoff-test-full-jitter' }"));
+                return null;
+            });
+            long endNanos = System.nanoTime();
+            withBackoffTime = TimeUnit.NANOSECONDS.toMillis(endNanos - startNanos);
+        } finally {
+            ExponentialBackoff.clearTestJitterSupplier();
+            failPointAdminDb.runCommand(Document.parse("{'configureFailPoint': 'failCommand', 'mode': 'off'}"));
+        }


This comment is still relevant. The code pieces that duplicate each other are https://github.com/nhachicha/mongo-java-driver/blob/f0973a1d537c91911d2be7c44b9960c1b2ddf9dd/driver-sync/src/test/functional/com/mongodb/client/WithTransactionProseTest.java#L222-L237 and https://github.com/nhachicha/mongo-java-driver/blob/f0973a1d537c91911d2be7c44b9960c1b2ddf9dd/driver-sync/src/test/functional/com/mongodb/client/WithTransactionProseTest.java#L239-L253.

stIncMale · 2026-02-03T15:38:37Z

driver-sync/src/test/functional/com/mongodb/client/WithTransactionProseTest.java

+        // Test 2: Run with jitter = 1 (full backoff)
+        ExponentialBackoff.setTestJitterSupplier(() -> 1.0);
+
+        failPointAdminDb.runCommand(Document.parse("{'configureFailPoint': 'failCommand', 'mode': {'times': 13}, " + "'data': {'failCommands': ['commitTransaction'], 'errorCode': 251}}"));


This will go away on its own when #1852 (comment) is done.

stIncMale · 2026-02-03T15:42:17Z

driver-sync/src/test/functional/com/mongodb/client/WithTransactionProseTest.java

+        long expectedWithBackoffTime = noBackoffTime + 2200;  // 2.2 seconds as per spec
+        long actualDifference = Math.abs(withBackoffTime - expectedWithBackoffTime);
+
+        assertTrue(actualDifference < 1000, String.format("Expected withBackoffTime to be ~%dms (noBackoffTime %dms + 2200ms), " + "but got %dms. Difference: %dms (tolerance: 1000ms per spec)", expectedWithBackoffTime, noBackoffTime, withBackoffTime, actualDifference));


This comment is still relevant. The code it applies to now is https://github.com/nhachicha/mongo-java-driver/blob/f0973a1d537c91911d2be7c44b9960c1b2ddf9dd/driver-sync/src/test/functional/com/mongodb/client/WithTransactionProseTest.java#L255-L260.

stIncMale · 2026-02-03T15:44:29Z

driver-sync/src/test/functional/com/mongodb/client/WithTransactionProseTest.java

+        long actualDifference = Math.abs(withBackoffTime - expectedWithBackoffTime);
+
+        assertTrue(actualDifference < 1000, String.format("Expected withBackoffTime to be ~% dms (noBackoffTime %d ms + 1800 ms), but"
+                + " got %d ms. Difference: %d ms (tolerance: 1000 ms per spec)", expectedWithBackoffTime, noBackoffTime, withBackoffTime,


We should use 500 ms instead of 1000 ms: https://github.com/mongodb/specifications/blob/9e2acc6c9f77977cdbc0705cd8e236ed4d1b6551/source/transactions-convenient-api/tests/README.md?plain=1#L102.

stIncMale

The last reviewed commit is f0973a1.

stIncMale · 2026-02-04T06:06:29Z

...ync/src/test/functional/com/mongodb/client/AbstractClientSideOperationsTimeoutProseTest.java

        commandListener = new TestCommandListener();
+
+        if (!isAsync()) {
+            // setting jitter to 0 to make test using withTransaction deterministic (i.e retries immediately) otherwise we might get


test -> tests?

stIncMale · 2026-02-04T06:30:26Z

...ync/src/test/functional/com/mongodb/client/AbstractClientSideOperationsTimeoutProseTest.java

+
+        if (!isAsync()) {
+            ExponentialBackoff.clearTestJitterSupplier();
+        }


This method looks sketchy. Let's write it such that every step is guaranteed regardless of the success of the previous steps (try-with-resources comes really handy here): stIncMale@a27a453.

nhachicha self-assigned this Dec 9, 2025

nhachicha mentioned this pull request Dec 9, 2025

JAVA-5950 Update Transactions Convenient API with exponential backoff on retries #1848

Closed

nhachicha requested a review from Copilot December 9, 2025 14:25

Copilot AI reviewed Dec 9, 2025

View reviewed changes

nhachicha marked this pull request as ready for review December 9, 2025 15:24

nhachicha requested a review from a team as a code owner December 9, 2025 15:24

nhachicha requested review from stIncMale and strogiyotec and removed request for strogiyotec December 9, 2025 15:24

vbabanin self-requested a review December 9, 2025 16:11

stIncMale requested changes Jan 3, 2026

View reviewed changes

nhachicha requested a review from stIncMale January 16, 2026 00:56

nhachicha and others added 15 commits January 19, 2026 17:53

JAVA-5950 - Update Transactions Convenient API with exponential backo…

1ed116f

…ff on retries

Simplifying test, clean up.

eb8b4ad

Fixing test

b8b0e1a

Update driver-sync/src/main/com/mongodb/client/internal/ClientSession…

c05ce05

…Impl.java Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

retrigger checks

aa96659

retrigger checks

98fc57b

retrigger checks

f98262e

retrigger checks

bfc89fc

test cleanup

d9405ef

retrigger checks

5de452b

Test cleanup

1867ff5

retrigger checks

f89d62d

Update the implementation according to the spec

3d646ae

Added prose test

9b4bf15

Flaky test

da83704

nhachicha and others added 14 commits January 19, 2026 17:53

Remove extra Test annotation

cb95167

Throwing correct exception when CSOT is used

ef734a0

Simplifying implementation by relying on CSOT to throw when timeout i…

96b5ed7

…s exceeded (ex operationContext.getTimeoutContext().getReadTimeoutMS())

Fixing implementation according to spec changes in JAVA-6046 and mong…

43eda52

…odb/specifications#1868

Update driver-sync/src/test/functional/com/mongodb/client/WithTransac…

a4193dd

…tionProseTest.java Co-authored-by: Valentin Kovalenko <valentin.male.kovalenko@gmail.com>

Update driver-sync/src/test/functional/com/mongodb/client/WithTransac…

f2d8263

…tionProseTest.java Co-authored-by: Valentin Kovalenko <valentin.male.kovalenko@gmail.com>

Update driver-sync/src/test/functional/com/mongodb/client/WithTransac…

f3daab0

…tionProseTest.java Co-authored-by: Valentin Kovalenko <valentin.male.kovalenko@gmail.com>

Update driver-sync/src/test/functional/com/mongodb/client/WithTransac…

7afd9d4

…tionProseTest.java Co-authored-by: Valentin Kovalenko <valentin.male.kovalenko@gmail.com>

Update driver-sync/src/test/functional/com/mongodb/client/WithTransac…

ccb8d03

…tionProseTest.java Co-authored-by: Valentin Kovalenko <valentin.male.kovalenko@gmail.com>

Update driver-sync/src/test/functional/com/mongodb/client/WithTransac…

bbb9a68

…tionProseTest.java Co-authored-by: Valentin Kovalenko <valentin.male.kovalenko@gmail.com>

Update driver-sync/src/test/functional/com/mongodb/client/WithTransac…

c44872d

…tionProseTest.java Co-authored-by: Valentin Kovalenko <valentin.male.kovalenko@gmail.com>

Update driver-core/src/main/com/mongodb/internal/ExponentialBackoff.java

f0dd916

Co-authored-by: Valentin Kovalenko <valentin.male.kovalenko@gmail.com>

PR feedback

0e00c90

remove annotation

36ecbf9

nhachicha force-pushed the nh/withTransaction_delay branch from 5c2145c to 36ecbf9 Compare January 19, 2026 18:01

stIncMale requested changes Jan 22, 2026

View reviewed changes

stIncMale reviewed Jan 26, 2026

View reviewed changes

stIncMale and others added 5 commits January 28, 2026 16:54

Fix/improve ClientSessionImpl, remove ClientSessionClock

4cea337

PR feedback

8bc1de2

Reduce flakiness in CSOT tests using withTransaction, by forcing an i…

cae949e

…mmediate retry without backoff.

Using correct jitter value in CSOT tests using withTransaction

6a17834

Guarding setting a custom jitter for only sync tests

98b8d28

nhachicha requested a review from stIncMale January 29, 2026 16:12

stIncMale requested changes Feb 2, 2026

View reviewed changes

nhachicha added 3 commits February 2, 2026 13:36

PR feedback

ddfe918

PR feedback

c97073e

Fixing unit test

f0973a1

stIncMale requested changes Feb 3, 2026

View reviewed changes

stIncMale requested changes Feb 4, 2026

View reviewed changes

	public class ExponentialBackoffTest {
	class ExponentialBackoffTest {

	* <a href="https://github.com/mongodb/specifications/blob/master/source/transactions-convenient-api/tests/README.md#retry-backoff-is-enforceds">Convenient API Prose Tests</a>.
	* <a href="https://github.com/mongodb/specifications/blob/master/source/transactions-convenient-api/tests/README.md#retry-backoff-is-enforced">Retry Backoff is Enforced</a>.

JAVA-5950 Update Transactions Convenient API with exponential backoff on retries #1852

Are you sure you want to change the base?

JAVA-5950 Update Transactions Convenient API with exponential backoff on retries #1852

Uh oh!

Conversation

nhachicha commented Dec 9, 2025 • edited by stIncMale Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Key Changes

Reviewed changes

Uh oh!

Uh oh!

Copilot AI Dec 9, 2025

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

dariakp commented Dec 17, 2025

Uh oh!

stIncMale left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

dariakp commented Jan 8, 2026

Uh oh!

stIncMale commented Jan 8, 2026

Uh oh!

stIncMale left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

stIncMale left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

stIncMale left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

stIncMale left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

nhachicha commented Dec 9, 2025 •

edited by stIncMale

Loading

stIncMale left a comment •

edited

Loading