Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[spanner] fix: set timeout for polling on operations #3488

Merged
merged 5 commits into from
Apr 25, 2020
Merged

[spanner] fix: set timeout for polling on operations #3488

merged 5 commits into from
Apr 25, 2020

Conversation

tmatsuo
Copy link
Contributor

@tmatsuo tmatsuo commented Apr 23, 2020

fixes #3471

Otherwise it will wait forever.

@tmatsuo tmatsuo requested review from mmmarklu and a team as code owners April 23, 2020 22:56
@googlebot googlebot added the cla: yes This human has signed the Contributor License Agreement. label Apr 23, 2020
@tmatsuo
Copy link
Contributor Author

tmatsuo commented Apr 24, 2020

The restore took more than 10 mins. I'll bump the deadline a bit.

@tmatsuo
Copy link
Contributor Author

tmatsuo commented Apr 24, 2020

This time, snippets.add_index(INSTANCE_ID, DATABASE_ID) is timing out.

@tmatsuo
Copy link
Contributor Author

tmatsuo commented Apr 24, 2020

It seems Spanner is having a bad day.

@tmatsuo
Copy link
Contributor Author

tmatsuo commented Apr 24, 2020

build
google.api_core.exceptions.ServiceUnavailable: 503 Socket closed

@tmatsuo tmatsuo added kokoro:run Add this label to force Kokoro to re-run the tests. kokoro:force-run Add this label to force Kokoro to re-run the tests. and removed kokoro:run Add this label to force Kokoro to re-run the tests. labels Apr 24, 2020
@kokoro-team kokoro-team removed the kokoro:force-run Add this label to force Kokoro to re-run the tests. label Apr 24, 2020
@tmatsuo tmatsuo requested a review from larkee April 24, 2020 22:37
larkee
larkee approved these changes Apr 25, 2020
View reviewed changes
Copy link
Contributor

@larkee larkee left a comment •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These timeouts LGTM. I'll investigate the above build failure on Monday. Thanks for this!

@tmatsuo tmatsuo merged commit 1e072f2 into GoogleCloudPlatform:master Apr 25, 2020
larkee pushed a commit to larkee/python-spanner that referenced this pull request Jul 23, 2020
[spanner] fix: set timeout for polling on operations [(#3488)](Google…
2ecd360 
…CloudPlatform/python-docs-samples#3488)

* [spanner] fix: set timeout for polling on operations

fixes #3471

* bump the deadline to 1200 for backup and restore

* bumped the deadline to 120
larkee added a commit to googleapis/python-spanner that referenced this pull request Jul 29, 2020
@larkee @jasonmorton
@ryanmats @dpebot @waprin @SurferJeffAtGoogle @jmdobry @kurtisvg @frankyn @jabubake @jsimonweb @RobinRH @theacodes @hegemonic @tonioshikanlu @wchargin @gguuss @mmmarklu @dmahugh @lgruen @skuruppu @renovate-bot @gcf-merge-on-green @leahecole @zx-zx
docs: add samples from spanner/cloud-client (#117)
8910771 
* Add spanner samples [(#804)](GoogleCloudPlatform/python-docs-samples#804)

* Update snippets.py [(#815)](GoogleCloudPlatform/python-docs-samples#815)

* Changed read_write minimum amount to 300,000 as per bug [(#818)](GoogleCloudPlatform/python-docs-samples#818)

* Remove cloud config fixture [(#887)](GoogleCloudPlatform/python-docs-samples#887)

* Remove cloud config fixture

* Fix client secrets

* Fix bigtable instance

* Fix reference to our testing tools

* Auto-update dependencies. [(#914)](GoogleCloudPlatform/python-docs-samples#914)

* Auto-update dependencies.

* xfail the error reporting test

* Fix lint

* Re-generate all readmes

* Auto-update dependencies. [(#922)](GoogleCloudPlatform/python-docs-samples#922)

* Auto-update dependencies.

* Fix pubsub iam samples

* Update spanner sample to use bind parameters [(#928)](GoogleCloudPlatform/python-docs-samples#928)

* Fix default arguments

* Fix README rst links [(#962)](GoogleCloudPlatform/python-docs-samples#962)

* Fix README rst links

* Update all READMEs

* Auto-update dependencies. [(#992)](GoogleCloudPlatform/python-docs-samples#992)

* Auto-update dependencies. [(#1004)](GoogleCloudPlatform/python-docs-samples#1004)

* Auto-update dependencies.

* Fix natural language samples

* Fix pubsub iam samples

* Fix language samples

* Fix bigquery samples

* Swap the album titles to be consistent with other samples [(#1035)](GoogleCloudPlatform/python-docs-samples#1035)

* Auto-update dependencies. [(#1055)](GoogleCloudPlatform/python-docs-samples#1055)

* Auto-update dependencies.

* Explicitly use latest bigtable client

Change-Id: Id71e9e768f020730e4ca9514a0d7ebaa794e7d9e

* Revert language update for now

Change-Id: I8867f154e9a5aae00d0047c9caf880e5e8f50c53

* Remove pdb. smh

Change-Id: I5ff905fadc026eebbcd45512d4e76e003e3b2b43

* Pass multi_use=True to spanner read-only transaction [(#1063)](GoogleCloudPlatform/python-docs-samples#1063)

Change-Id: Ied9d9f519edd572d79dc95d2812c1b98f5a92794

* fix typo

Change-Id: I887507fa33ea30f5859707063326934e5c11208f

* Auto-update dependencies. [(#1093)](GoogleCloudPlatform/python-docs-samples#1093)

* Auto-update dependencies.

* Fix storage notification poll sample

Change-Id: I6afbc79d15e050531555e4c8e51066996717a0f3

* Fix spanner samples

Change-Id: I40069222c60d57e8f3d3878167591af9130895cb

* Drop coverage because it's not useful

Change-Id: Iae399a7083d7866c3c7b9162d0de244fbff8b522

* Try again to fix flaky logging test

Change-Id: I6225c074701970c17c426677ef1935bb6d7e36b4

* Add spanner stale data sample [(#1107)](GoogleCloudPlatform/python-docs-samples#1107)

* Update all generated readme auth instructions [(#1121)](GoogleCloudPlatform/python-docs-samples#1121)

Change-Id: I03b5eaef8b17ac3dc3c0339fd2c7447bd3e11bd2

* Added Link to Python Setup Guide [(#1158)](GoogleCloudPlatform/python-docs-samples#1158)

* Update Readme.rst to add Python setup guide

As requested in b/64770713.

This sample is linked in documentation https://cloud.google.com/bigtable/docs/scaling, and it would make more sense to update the guide here than in the documentation.

* Update README.rst

* Update README.rst

* Update README.rst

* Update README.rst

* Update README.rst

* Update install_deps.tmpl.rst

* Updated readmegen scripts and re-generated related README files

* Fixed the lint error

* Auto-update dependencies. [(#1138)](GoogleCloudPlatform/python-docs-samples#1138)

* Auto-update dependencies. [(#1186)](GoogleCloudPlatform/python-docs-samples#1186)

* Fixed failed tests on Kokoro (Spanner + Translate) [(#1192)](GoogleCloudPlatform/python-docs-samples#1192)

* Fixed failed tests on Kokoro (Spanner + Translate)

* Update quickstart_test.py

* Bump spanner stale read from 10 to 15 seconds. [(#1207)](GoogleCloudPlatform/python-docs-samples#1207)

At the request of the spanner team.

* Added "Open in Cloud Shell" buttons to README files [(#1254)](GoogleCloudPlatform/python-docs-samples#1254)

* Auto-update dependencies. [(#1316)](GoogleCloudPlatform/python-docs-samples#1316)

* Auto-update dependencies. [(#1354)](GoogleCloudPlatform/python-docs-samples#1354)

* Add Spanner region tags. [(#1376)](GoogleCloudPlatform/python-docs-samples#1376)

* Auto-update dependencies. [(#1377)](GoogleCloudPlatform/python-docs-samples#1377)

* Auto-update dependencies.

* Update requirements.txt

* Spanner Batch Query Sample [(#1402)](GoogleCloudPlatform/python-docs-samples#1402)

* Auto-update dependencies. [(#1406)](GoogleCloudPlatform/python-docs-samples#1406)

* Spanner Commit Timestamp Sample [(#1425)](GoogleCloudPlatform/python-docs-samples#1425)

* Regenerate the README files and fix the Open in Cloud Shell link for some samples [(#1441)](GoogleCloudPlatform/python-docs-samples#1441)

* Update READMEs to fix numbering and add git clone [(#1464)](GoogleCloudPlatform/python-docs-samples#1464)

* Adding Spanner STRUCT param samples [(#1519)](GoogleCloudPlatform/python-docs-samples#1519)

* Adding Spanner STRUCT param samples

* Fix python Cloud Spanner tests. [(#1548)](GoogleCloudPlatform/python-docs-samples#1548)

* Fix python Cloud Spanner tests.
* Lint.

* Cleanup spanner tests. [(#1633)](GoogleCloudPlatform/python-docs-samples#1633)

* Cleanup spanner tests.

* Update requirements.

* Added Spanner DML/PDML samples.  [(#1742)](GoogleCloudPlatform/python-docs-samples#1742)

* Added Spanner DML/PDML samples.

* Fixed lint issues and bumped version.

* Update method name  to match action. [(#1836)](GoogleCloudPlatform/python-docs-samples#1836)

* Auto-update dependencies. [(#1846)](GoogleCloudPlatform/python-docs-samples#1846)

ACK, merging.

* Add sample to delete data. [(#1872)](GoogleCloudPlatform/python-docs-samples#1872)

* Update snippets.py

* Update snippets_test.py

* Auto-update dependencies. [(#1980)](GoogleCloudPlatform/python-docs-samples#1980)

* Auto-update dependencies.

* Update requirements.txt

* Update requirements.txt

* Add Cloud Spanner Batch DML sample [(#2068)](GoogleCloudPlatform/python-docs-samples#2068)

* Add Cloud Spanner Batch DML sample

* Fix test.

* Lint.

* More Lint.

* Add queryWithParameter to Cloud Spanner sample. [(#2153)](GoogleCloudPlatform/python-docs-samples#2153)

* Add queryWithParameter to Cloud Spanner sample.

* Lint.

* Update to fix test.

* Deflake bigtable and spanner tests. [(#2224)](GoogleCloudPlatform/python-docs-samples#2224)

* Spanner doesn't actually promise the order of the results, so make the assertion work regardless of ordering.
* Bigtable might need some more time to scale, so retry the assertion up to 10 times.

* Improve and fix Cloud Spanner samples that transfer marketing budget [(#2198)](GoogleCloudPlatform/python-docs-samples#2198)

The samples that transfer part of an album's marketing budget had some issues:

+ `read_write_transaction`: Compared `second_album_budget` with an arbitrary integer, rather than explicitly checking against `transfer_amount`.
+ `write_with_dml_transaction`: Moved money from album 1 to album 2, even though `read_write_transaction` was the other way around. Also retrieved album 1's budget where it should have retrieved album 2's budget.

This change fixes those issues and updates the tests accordingly.

* Add Datatypes examples to Spanner sample. [(#2251)](GoogleCloudPlatform/python-docs-samples#2251)

* Add Datatypes examples to Spanner sample.

* Lint.

* Lint.

* Fix test.

* Add bulk loading Python Sample [(#2295)](GoogleCloudPlatform/python-docs-samples#2295)

Adds the following functionality:

* Create bulk_load_csv
* Delete bulk_load_csv
* Create schema.ddl

* Fix Spanner `BOOL` example after upstream typo fix [(#2356)](GoogleCloudPlatform/python-docs-samples#2356)

Summary:
This code used to be correct, when the Spanner Python API had a typo in
the parameter name, but that typo was fixed in an upstream pull request:
<googleapis/google-cloud-python#7295>

Test Plan:
Running `git grep BOOE` now returns no results.

wchargin-branch: bool-not-booe

* Updates to spanner version with BOOL correctly spelled. [(#2392)](GoogleCloudPlatform/python-docs-samples#2392)

* Adds updates for samples profiler ... vision [(#2439)](GoogleCloudPlatform/python-docs-samples#2439)

* update filenames to match the CSV files [(#2535)](GoogleCloudPlatform/python-docs-samples#2535)

fixes GoogleCloudPlatform/python-docs-samples#2532

* Auto-update dependencies. [(#2005)](GoogleCloudPlatform/python-docs-samples#2005)

* Auto-update dependencies.

* Revert update of appengine/flexible/datastore.

* revert update of appengine/flexible/scipy

* revert update of bigquery/bqml

* revert update of bigquery/cloud-client

* revert update of bigquery/datalab-migration

* revert update of bigtable/quickstart

* revert update of compute/api

* revert update of container_registry/container_analysis

* revert update of dataflow/run_template

* revert update of datastore/cloud-ndb

* revert update of dialogflow/cloud-client

* revert update of dlp

* revert update of functions/imagemagick

* revert update of functions/ocr/app

* revert update of healthcare/api-client/fhir

* revert update of iam/api-client

* revert update of iot/api-client/gcs_file_to_device

* revert update of iot/api-client/mqtt_example

* revert update of language/automl

* revert update of run/image-processing

* revert update of vision/automl

* revert update testing/requirements.txt

* revert update of vision/cloud-client/detect

* revert update of vision/cloud-client/product_search

* revert update of jobs/v2/api_client

* revert update of jobs/v3/api_client

* revert update of opencensus

* revert update of translate/cloud-client

* revert update to speech/cloud-client

Co-authored-by: Kurtis Van Gent <31518063+kurtisvg@users.noreply.github.com>
Co-authored-by: Doug Mahugh <dmahugh@gmail.com>

* Delete spanner/cloud-client/bulk_load_csv. [(#2721)](GoogleCloudPlatform/python-docs-samples#2721)

This has been moved to https://github.com/cloudspannerecosystem/sampledb.

Co-authored-by: Kurtis Van Gent <31518063+kurtisvg@users.noreply.github.com>

* spanner: add query options versioning samples [(#3093)](GoogleCloudPlatform/python-docs-samples#3093)

* spanner: add query versioning samples

* update test assertions

* Fixed the region tags.

* Removed extra whitespace.

* update required spanner version

Co-authored-by: larkee <larkee@users.noreply.github.com>
Co-authored-by: skuruppu <skuruppu@google.com>

* spanner: Add Cloud Spanner Backup samples [(#3101)](GoogleCloudPlatform/python-docs-samples#3101)

* add backup samples

* update required spanner version

* fix lint errors

* run backup samples tests against a new instance

* fix lint

* wait for instance creation to complete

* Apply suggestions from code review

Co-Authored-By: skuruppu <skuruppu@google.com>

* add list_backups test

* fix lint

* add missing newline character in assert

* update samples to be consistent with other languages

* lint fix

* add pagination sample

* reorder tests

Co-authored-by: larkee <larkee@users.noreply.github.com>
Co-authored-by: skuruppu <skuruppu@google.com>

* Simplify noxfile setup. [(#2806)](GoogleCloudPlatform/python-docs-samples#2806)

* chore(deps): update dependency requests to v2.23.0

* Simplify noxfile and add version control.

* Configure appengine/standard to only test Python 2.7.

* Update Kokokro configs to match noxfile.

* Add requirements-test to each folder.

* Remove Py2 versions from everything execept appengine/standard.

* Remove conftest.py.

* Remove appengine/standard/conftest.py

* Remove 'no-sucess-flaky-report' from pytest.ini.

* Add GAE SDK back to appengine/standard tests.

* Fix typo.

* Roll pytest to python 2 version.

* Add a bunch of testing requirements.

* Remove typo.

* Add appengine lib directory back in.

* Add some additional requirements.

* Fix issue with flake8 args.

* Even more requirements.

* Readd appengine conftest.py.

* Add a few more requirements.

* Even more Appengine requirements.

* Add webtest for appengine/standard/mailgun.

* Add some additional requirements.

* Add workaround for issue with mailjet-rest.

* Add responses for appengine/standard/mailjet.

Co-authored-by: Renovate Bot <bot@renovateapp.com>

* Update dependency google-cloud-spanner to v1.15.1 [(#3377)](GoogleCloudPlatform/python-docs-samples#3377)

This PR contains the following updates:

| Package | Update | Change |
|---|---|---|
| [google-cloud-spanner](https://togithub.com/googleapis/python-spanner) | patch | `==1.15.0` -> `==1.15.1` |

---

### Release Notes

<details>
<summary>googleapis/python-spanner</summary>

### [`v1.15.1`](https://togithub.com/googleapis/python-spanner/blob/master/CHANGELOG.md#&#8203;1151-httpswwwgithubcomgoogleapispython-spannercomparev1150v1151-2020-04-08)

[Compare Source](https://togithub.com/googleapis/python-spanner/compare/v1.15.0...v1.15.1)

</details>

---

### Renovate configuration

:date: **Schedule**: At any time (no schedule defined).

:vertical_traffic_light: **Automerge**: Disabled by config. Please merge this manually once you are satisfied.

:recycle: **Rebasing**: Never, or you tick the rebase/retry checkbox.

:no_bell: **Ignore**: Close this PR and you won't be reminded about this update again.

---

 - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box

---

This PR has been generated by [WhiteSource Renovate](https://renovate.whitesourcesoftware.com). View repository job log [here](https://app.renovatebot.com/dashboard#GoogleCloudPlatform/python-docs-samples).

* [spanner] fix: bump the timeout for instance creation [(#3468)](GoogleCloudPlatform/python-docs-samples#3468)

fixes #3466

* [spanner] fix: set timeout for polling on operations [(#3488)](GoogleCloudPlatform/python-docs-samples#3488)

* [spanner] fix: set timeout for polling on operations

fixes #3471

* bump the deadline to 1200 for backup and restore

* bumped the deadline to 120

* fix: use DELETE FROM for consistency [(#3498)](GoogleCloudPlatform/python-docs-samples#3498)

Being consistent with [docs](https://cloud.google.com/spanner/docs/dml-syntax#delete_examples).

Co-authored-by: gcf-merge-on-green[bot] <60162190+gcf-merge-on-green[bot]@users.noreply.github.com>
Co-authored-by: Takashi Matsuo <tmatsuo@google.com>

* chore: pin new release [(#3688)](GoogleCloudPlatform/python-docs-samples#3688)

Co-authored-by: larkee <larkee@users.noreply.github.com>

* chore: some lint fixes [(#3749)](GoogleCloudPlatform/python-docs-samples#3749)

* chore(deps): update dependency google-cloud-spanner to v1.17.0 [(#3885)](GoogleCloudPlatform/python-docs-samples#3885)

* Improve Spanner delete_data sample coverage [(#3922)](GoogleCloudPlatform/python-docs-samples#3922)

* improve Spanner delete_data sample coverage

* specify end to stay consistent with other languages

Co-authored-by: larkee <larkee@users.noreply.github.com>

* chore(deps): update dependency google-cloud-spanner to v1.17.1 [(#4160)](GoogleCloudPlatform/python-docs-samples#4160)

Co-authored-by: Takashi Matsuo <tmatsuo@google.com>

* fix(spanner): use uuid for unique id [(#4198)](GoogleCloudPlatform/python-docs-samples#4198)

fixes #4197
(possibly)

* feat(spanner): add sample for create instance [(#4230)](GoogleCloudPlatform/python-docs-samples#4230)

Co-authored-by: larkee <larkee@users.noreply.github.com>
Co-authored-by: Leah E. Cole <6719667+leahecole@users.noreply.github.com>

* chore(deps): update dependency pytest to v5.4.3 [(#4279)](GoogleCloudPlatform/python-docs-samples#4279)

* chore(deps): update dependency pytest to v5.4.3

* specify pytest for python 2 in appengine

Co-authored-by: Leah Cole <coleleah@google.com>

* chore(deps): update dependency mock to v4 [(#4287)](GoogleCloudPlatform/python-docs-samples#4287)

* chore(deps): update dependency mock to v4

* specify mock version for appengine python 2

Co-authored-by: Leah Cole <coleleah@google.com>

* test(spanner): add sleep to fix flaky test [(#4289)](GoogleCloudPlatform/python-docs-samples#4289)

Co-authored-by: larkee <larkee@users.noreply.github.com>

* chore: update templates

* fix lint

Co-authored-by: Jon Wayne Parrott <jonwayne@google.com>
Co-authored-by: Jason Morton <jason.morton@gmail.com>
Co-authored-by: Ryan Matsumoto <ryanmats@google.com>
Co-authored-by: DPE bot <dpebot@google.com>
Co-authored-by: Bill Prin <waprin@gmail.com>
Co-authored-by: michaelawyu <chenyumic@google.com>
Co-authored-by: Jeffrey Rennie <rennie@google.com>
Co-authored-by: Jason Dobry <jmdobry@users.noreply.github.com>
Co-authored-by: Kurtis Van Gent <31518063+kurtisvg@users.noreply.github.com>
Co-authored-by: Frank Natividad <frankyn@users.noreply.github.com>
Co-authored-by: Jisha Abubaker <jishaa@google.com>
Co-authored-by: Jonathan Simon <jsimonweb@gmail.com>
Co-authored-by: Robin Reynolds-Haertle <RobinRH@users.noreply.github.com>
Co-authored-by: Thea Flowers <theaflowers@google.com>
Co-authored-by: Jeff Williams <jefesaurus@google.com>
Co-authored-by: Oluwatoni Oshikanlu <39501338+tonioshikanlu@users.noreply.github.com>
Co-authored-by: William Chargin <wchargin@gmail.com>
Co-authored-by: Gus Class <gguuss@gmail.com>
Co-authored-by: Mark <mmmarklu@gmail.com>
Co-authored-by: Doug Mahugh <dmahugh@gmail.com>
Co-authored-by: Leonhard Gruenschloss <leonhard.gruenschloss@gmail.com>
Co-authored-by: larkee <larkee@users.noreply.github.com>
Co-authored-by: skuruppu <skuruppu@google.com>
Co-authored-by: Renovate Bot <bot@renovateapp.com>
Co-authored-by: Takashi Matsuo <tmatsuo@google.com>
Co-authored-by: gcf-merge-on-green[bot] <60162190+gcf-merge-on-green[bot]@users.noreply.github.com>
Co-authored-by: Leah E. Cole <6719667+leahecole@users.noreply.github.com>
Co-authored-by: Leah Cole <coleleah@google.com>
Co-authored-by: root <root@google.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@larkee larkee larkee approved these changes

@mmmarklu mmmarklu Awaiting requested review from mmmarklu

Assignees

@crwilcox crwilcox

Labels
cla: yes This human has signed the Contributor License Agreement.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

[spanner] bug: sample code is waiting for the backup operations forever.
5 participants
@tmatsuo @larkee @crwilcox @googlebot @kokoro-team