Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: update retry strategy for mutation calls to handle aborted transactions #1270

Closed
wants to merge 2 commits into from

Conversation

aakashanandg
Copy link
Contributor

Updating retry strategy for mutation calls to handle aborted transactions

  • This PR updates the retry strategy for mutation calls to handle aborted transactions more effectively. Previously, the retry mechanism didn't handle certain edge cases for aborted transactions, leading to failures. The updated strategy ensures retries in these scenarios to improve the robustness of the mutation operations.

Test Results:

  • All unit tests related to the retry logic and mutation operations pass successfully.

Fixes #<1133> 🦕

@aakashanandg aakashanandg requested review from a team as code owners December 16, 2024 13:40
@product-auto-label product-auto-label bot added the size: m Pull request size is medium. label Dec 16, 2024
@product-auto-label product-auto-label bot added the api: spanner Issues related to the googleapis/python-spanner API. label Dec 16, 2024
method,
allowed_exceptions={InternalServerError: _check_rst_stream_error},
)
response = self._session.run_in_transaction(method)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmmm.... Now that I see this, I realize that this cannot use run_in_transaction directly. The reason is that:

  1. This method batch.commit() in its original form (so without the changes in this pull request) creates and commits a single-use read/write transaction.
  2. run_in_transaction however creates a new read/write transaction and then executes the given method in the scope of that transaction.

So what happens now is that you create two transactions:

  1. run_in_transaction creates a transaction that is not being used.
  2. method that is passed in to run_in_transaction executes a Commit in a single-use read/write transaction.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for pointing that out. I've updated the PR with a new retry strategy, replicating the same retry behavior as in the run_in_transaction method.

except Aborted:
pass

# Verify that commit was called more than once (due to retry)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You should verify the exact number. Based on my comment above, my guess is that you are getting 1 more commit than you would expect, as run_in_transaction also creates a read/write transaction.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for the clarification. I’ve added updated assertions to track the number of times api.commit is called with the next revision

@product-auto-label product-auto-label bot added size: l Pull request size is large. and removed size: m Pull request size is medium. labels Jan 2, 2025
@product-auto-label product-auto-label bot added size: xl Pull request size is extra large. and removed size: l Pull request size is large. labels Jan 2, 2025
@aakashanandg
Copy link
Contributor Author

A new PR has been created to fix the commit history. Please refer to the new PR for the updated changes. This PR will be closed to keep the commit history clean and in line with the intended changes.
New PR: #1279

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
api: spanner Issues related to the googleapis/python-spanner API. size: xl Pull request size is extra large.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants