Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

test(spanner): fix flaky restore_database() sample test #4641

Merged
merged 6 commits into from
Sep 18, 2020

Conversation

larkee
Copy link
Contributor

@larkee larkee commented Sep 8, 2020

Fixes #4544

@larkee larkee added the api: spanner Issues related to the Spanner API. label Sep 8, 2020
@larkee larkee requested review from tmatsuo and skuruppu September 8, 2020 03:03
@larkee larkee self-assigned this Sep 8, 2020
@google-cla google-cla bot added the cla: yes This human has signed the Contributor License Agreement. label Sep 8, 2020
@larkee larkee added the kokoro:force-run Add this label to force Kokoro to re-run the tests. label Sep 8, 2020
@kokoro-team kokoro-team removed the kokoro:force-run Add this label to force Kokoro to re-run the tests. label Sep 8, 2020
@larkee larkee marked this pull request as ready for review September 8, 2020 04:21
@larkee larkee requested a review from a team as a code owner September 8, 2020 04:21
@product-auto-label product-auto-label bot added the samples Issues that are directly related to samples. label Sep 8, 2020
@larkee larkee changed the title test(spanner): skip flaky restore_database() test test(spanner): fix flaky restore_database() sample test Sep 11, 2020
@@ -67,6 +67,13 @@ def test_create_backup(capsys, database):
assert BACKUP_ID in out


@pytest.mark.skip(
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we mark the test with flaky?

You'll need to add

flaky==3.7.0

to requirements-test.txt.

Then mark the test:

@pytest.mark.flaky(max_runs=2, min_passes=1)

But the current timeout seems like 1200 seconds (20 minutes). This is likely problematic with the current test timeout with btlr.
@kurtisvg Is here a way to extend the timeout for a particular test?

I also think the restore operation for a small database should finish quicker. I would rather get annoyed with flaky build cop bot issues until the server side operations are quicker and more consistent.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@kurtisvg Is here a way to extend the timeout for a particular test?

Nope

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've used a RetryErrors decorator from the Cloud Spanner test_utils library which will retry the test on DEADLINE_EXCEEDED for a maximum of two attempts. Is this an acceptable alternative to flaky?

spanner/cloud-client/backup_sample_test.py Outdated Show resolved Hide resolved
@tmatsuo tmatsuo merged commit cacc0c7 into GoogleCloudPlatform:master Sep 18, 2020
@tmatsuo
Copy link
Contributor

tmatsuo commented Sep 18, 2020

@larkee Thanks!

larkee added a commit to larkee/python-spanner that referenced this pull request Sep 24, 2020
…gleCloudPlatform/python-docs-samples#4641)

* test(spanner): fix flaky restore backup sample test

* fix: update timeout to match other languages

* test: import test_utils and set max retries to 2

Co-authored-by: larkee <larkee@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
api: spanner Issues related to the Spanner API. cla: yes This human has signed the Contributor License Agreement. samples Issues that are directly related to samples.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

spanner.cloud-client.backup_sample_test: test_restore_database failed
5 participants