Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[datalabeling] testing: wrap rpcs with backoff #3443

Merged
merged 8 commits into from
Apr 22, 2020

Conversation

tmatsuo
Copy link
Contributor

@tmatsuo tmatsuo commented Apr 20, 2020

fixes #3438
fixes #3439

We have been seeing flaky test results with the datalabeling test backends. This PR is wrapping all the rpc calls with backoff with max_time=60.

I believe this will significantly reduce the test flakiness. Here is an explanation.

If a test consists of multiple consecutive flaky rpc calls, the success rate of the test will become exponentially small as number of rpc grows.

Here is an example:

Let's say the success rate of a single rpc is always 0.9 (for simplicity) and
there is a test which has 8 consecutive rpc calls.

The success rate of the test as a whole:
0.9^8=~0.43
If we wrap this test with pytest.mark.flaky(max_runs=3), the success rate of it:
1 - 0.57^3 = 0.81

If we wrap each rpc calls with retries, the success rate will be significantly higher. Let's say we retry 3 times, then the success rate of a single rpc call (with 3 times retries):
1 - 0.1^3 = 0.999
The success rate of the test as a whole:
0.999^8 = ~0.992

The tests are still flaky, but significantly more robust without retrying the test itself. If we still see test failures, we may need to bump the max_time parameter on backoff.on_exception.

@tmatsuo tmatsuo requested review from leahecole and kurtisvg April 20, 2020 18:13
@tmatsuo tmatsuo requested review from sgreenberg and a team as code owners April 20, 2020 18:13
@googlebot googlebot added the cla: yes This human has signed the Contributor License Agreement. label Apr 20, 2020
@tmatsuo
Copy link
Contributor Author

tmatsuo commented Apr 20, 2020

The tests are still failing. According to the message, it seems like the test API backend is flaky.

The options came to mind are:

  1. Use backoff to retry the API call for a certain duration.
    Currently flaky doesn't use any delay, so using backoff is probably better. Also, some tests are errored during the setup phase and it doesn't seem flaky retried that case. We can use backoff in the setup code.
    This might mitigate the test flakiness, but no guarantee.

  2. Use the production endpoint, and somehow exclude our projects from manual review process (refer the comment).

  3. Use mock client for the tests.

@kurtisvg @sgreenberg
WDYT?

@tmatsuo
Copy link
Contributor Author

tmatsuo commented Apr 21, 2020

Ok, I'll go with the option 1 above first. A new commit is on the way.

@kurtisvg
Copy link
Contributor

Ok, I'll go with the option 1 above first. A new commit is on the way.

Option 1 seems good to me - maybe we should reach out to the eng team and verify that the backend is flakey?

@tmatsuo
Copy link
Contributor Author

tmatsuo commented Apr 21, 2020

Since it's not possible to run the tests locally, I'd let Kokoro run the tests with the said change only in one file.

@tmatsuo
Copy link
Contributor Author

tmatsuo commented Apr 21, 2020

The change seems to work. I'll make similar changes to other files too.

@tmatsuo
Copy link
Contributor Author

tmatsuo commented Apr 22, 2020

I think the code still has a problem around test_delete_dataset... I can ignore error in fixture teardown.

@tmatsuo tmatsuo changed the title [datalabeling] testing: mark some tests as flaky [datalabeling] testing: wrap rpcs with backoff Apr 22, 2020
@sgreenberg
Copy link
Contributor

@nnegrey , @dizcology - @tmatsuo asked me who knows about Data Labeling. I looked through the test history and it seems like you two might?

@nnegrey
Copy link
Contributor

nnegrey commented Apr 22, 2020

  1. Seems fine, potentially the test service is flaky.
  2. This is an option the product team would explicitly like us to avoid, because the Datalabeling is done by humans so the cost is higher and if our cancellation calls don't happen properly we can get a high bill with how many times all these tests run. The Test Service avoids this by mocking the completion, but I don't believe it runs fast enough for our tests which is why we cancel the jobs.

@tmatsuo, you can run the tests locally, you just have to set: https://github.com/GoogleCloudPlatform/python-docs-samples/blob/master/.kokoro/tests/run_tests.sh#L51

@tmatsuo tmatsuo requested review from nnegrey and removed request for sgreenberg and leahecole April 22, 2020 14:37
@tmatsuo
Copy link
Contributor Author

tmatsuo commented Apr 22, 2020

@nnegrey Yeah I know that envvar. When I set this, it errors something like "API not enabled", and the link provided for enabling the API doesn't work. If there's a way to enable the test API other than that link, it should be documented.

Also, I'll appreciate it if you can review, thanks!

@kurtisvg The PR is ready for review, take a look if you have time.

@nnegrey
Copy link
Contributor

nnegrey commented Apr 22, 2020

Are you using the python project? I believe it may need special white listing to hit the test service.

@tmatsuo
Copy link
Contributor Author

tmatsuo commented Apr 22, 2020

@nnegrey I'm using my own project. I understand I can use the shared project, but normally I'm using my own project for local testing to avoid contaminating the shared project.

Thanks for the review.

I'm going to merge this with my admin power.

@tmatsuo tmatsuo merged commit f877a9a into GoogleCloudPlatform:master Apr 22, 2020
@tmatsuo tmatsuo deleted the fix-datalabeling branch April 22, 2020 15:23
leahecole pushed a commit that referenced this pull request Apr 24, 2020
* wrap all the rpcs with backoff
* add a shared testing lib
* remove flaky
arithmetic1728 pushed a commit to googleapis/python-datalabeling that referenced this pull request Aug 7, 2020
…latform/python-docs-samples#3443)

* wrap all the rpcs with backoff
* add a shared testing lib
* remove flaky
arithmetic1728 added a commit to googleapis/python-datalabeling that referenced this pull request Aug 7, 2020
* Data Labeling Beta samples [(#2096)](GoogleCloudPlatform/python-docs-samples#2096)

* add files

* upate create_annotation_spec_set and test

* add requirements.txt

* update create_instruction and test

* update import data and test

* add label image and test

* add label_text test

* add label_video_test

* add manage dataset and tests

* flake

* fix

* add README

* Adds updates including compute [(#2436)](GoogleCloudPlatform/python-docs-samples#2436)

* Adds updates including compute

* Python 2 compat pytest

* Fixing weird \r\n issue from GH merge

* Put asset tests back in

* Re-add pod operator test

* Hack parameter for k8s pod operator

* Update datalabeling samples to hit test endpoint. [(#2641)](GoogleCloudPlatform/python-docs-samples#2641)

* Auto-update dependencies. [(#2005)](GoogleCloudPlatform/python-docs-samples#2005)

* Auto-update dependencies.

* Revert update of appengine/flexible/datastore.

* revert update of appengine/flexible/scipy

* revert update of bigquery/bqml

* revert update of bigquery/cloud-client

* revert update of bigquery/datalab-migration

* revert update of bigtable/quickstart

* revert update of compute/api

* revert update of container_registry/container_analysis

* revert update of dataflow/run_template

* revert update of datastore/cloud-ndb

* revert update of dialogflow/cloud-client

* revert update of dlp

* revert update of functions/imagemagick

* revert update of functions/ocr/app

* revert update of healthcare/api-client/fhir

* revert update of iam/api-client

* revert update of iot/api-client/gcs_file_to_device

* revert update of iot/api-client/mqtt_example

* revert update of language/automl

* revert update of run/image-processing

* revert update of vision/automl

* revert update testing/requirements.txt

* revert update of vision/cloud-client/detect

* revert update of vision/cloud-client/product_search

* revert update of jobs/v2/api_client

* revert update of jobs/v3/api_client

* revert update of opencensus

* revert update of translate/cloud-client

* revert update to speech/cloud-client

Co-authored-by: Kurtis Van Gent <31518063+kurtisvg@users.noreply.github.com>
Co-authored-by: Doug Mahugh <dmahugh@gmail.com>

* Update datalabeling to match lint. [(#2642)](GoogleCloudPlatform/python-docs-samples#2642)

* datalabeling: ensure all tests use test endpoint [(#2918)](GoogleCloudPlatform/python-docs-samples#2918)

* datalabeling: ensure all tests use test endpoint

* requires an input csv for text input, slight print statement cleanup

Co-authored-by: Leah E. Cole <6719667+leahecole@users.noreply.github.com>

* chore(deps): update dependency google-cloud-datalabeling to v0.4.0 [(#3081)](GoogleCloudPlatform/python-docs-samples#3081)

This PR contains the following updates:

| Package | Update | Change |
|---|---|---|
| [google-cloud-datalabeling](https://togithub.com/googleapis/python-datalabeling) | minor | `==0.3.0` -> `==0.4.0` |

---

### Release Notes

<details>
<summary>googleapis/python-datalabeling</summary>

### [`v0.4.0`](https://togithub.com/googleapis/python-datalabeling/blob/master/CHANGELOG.md#&#8203;040-httpswwwgithubcomgoogleapispython-datalabelingcomparev030v040-2020-01-31)

[Compare Source](https://togithub.com/googleapis/python-datalabeling/compare/v0.3.0...v0.4.0)

##### Features

-   **datalabeling:** undeprecate resource name helper methods (via synth) ([#&#8203;10039](https://www.github.com/googleapis/python-datalabeling/issues/10039)) ([88f8090](https://www.github.com/googleapis/python-datalabeling/commit/88f809008ee6a709c02c78b1d93af779fab19adb))

##### Bug Fixes

-   **datalabeling:** deprecate resource name helper methods (via synth) ([#&#8203;9832](https://www.github.com/googleapis/python-datalabeling/issues/9832)) ([e5f9021](https://www.github.com/googleapis/python-datalabeling/commit/e5f902154ebe7fcb139aa405babfe9993fd51319))

</details>

---

### Renovate configuration

:date: **Schedule**: At any time (no schedule defined).

:vertical_traffic_light: **Automerge**: Disabled by config. Please merge this manually once you are satisfied.

:recycle: **Rebasing**: Never, or you tick the rebase/retry checkbox.

:no_bell: **Ignore**: Close this PR and you won't be reminded about this update again.

---

 - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box

---

This PR has been generated by [WhiteSource Renovate](https://renovate.whitesourcesoftware.com). View repository job log [here](https://app.renovatebot.com/dashboard#GoogleCloudPlatform/python-docs-samples).

* Simplify noxfile setup. [(#2806)](GoogleCloudPlatform/python-docs-samples#2806)

* chore(deps): update dependency requests to v2.23.0

* Simplify noxfile and add version control.

* Configure appengine/standard to only test Python 2.7.

* Update Kokokro configs to match noxfile.

* Add requirements-test to each folder.

* Remove Py2 versions from everything execept appengine/standard.

* Remove conftest.py.

* Remove appengine/standard/conftest.py

* Remove 'no-sucess-flaky-report' from pytest.ini.

* Add GAE SDK back to appengine/standard tests.

* Fix typo.

* Roll pytest to python 2 version.

* Add a bunch of testing requirements.

* Remove typo.

* Add appengine lib directory back in.

* Add some additional requirements.

* Fix issue with flake8 args.

* Even more requirements.

* Readd appengine conftest.py.

* Add a few more requirements.

* Even more Appengine requirements.

* Add webtest for appengine/standard/mailgun.

* Add some additional requirements.

* Add workaround for issue with mailjet-rest.

* Add responses for appengine/standard/mailjet.

Co-authored-by: Renovate Bot <bot@renovateapp.com>

* testing: mark some tests as flaky [(#3288)](GoogleCloudPlatform/python-docs-samples#3288)

fixes #3138

* [datalabeling] testing: wrap rpcs with backoff [(#3443)](GoogleCloudPlatform/python-docs-samples#3443)

* wrap all the rpcs with backoff
* add a shared testing lib
* remove flaky

* [datalabeling] fix: clean up old datasets before the test [(#3707)](GoogleCloudPlatform/python-docs-samples#3707)

fixes #3710 
fixes #3711

* [datalabeling] testing: retry upon ServerError [(#3762)](GoogleCloudPlatform/python-docs-samples#3762)

fixes #3760

* Replace GCLOUD_PROJECT with GOOGLE_CLOUD_PROJECT. [(#4022)](GoogleCloudPlatform/python-docs-samples#4022)

* chore(deps): update dependency pytest to v5.4.3 [(#4279)](GoogleCloudPlatform/python-docs-samples#4279)

* chore(deps): update dependency pytest to v5.4.3

* specify pytest for python 2 in appengine

Co-authored-by: Leah Cole <coleleah@google.com>

* Update dependency pytest to v6 [(#4390)](GoogleCloudPlatform/python-docs-samples#4390)

* chore: update templates

* chore: fix docs error

* chore: skip unavailable samples

* chore: use staging endpoint for labeling tests

Co-authored-by: Rebecca Taylor <remilytaylor@gmail.com>
Co-authored-by: Gus Class <gguuss@gmail.com>
Co-authored-by: Kurtis Van Gent <31518063+kurtisvg@users.noreply.github.com>
Co-authored-by: DPEBot <dpebot@google.com>
Co-authored-by: Doug Mahugh <dmahugh@gmail.com>
Co-authored-by: Noah Negrey <nnegrey@users.noreply.github.com>
Co-authored-by: Leah E. Cole <6719667+leahecole@users.noreply.github.com>
Co-authored-by: WhiteSource Renovate <bot@renovateapp.com>
Co-authored-by: Takashi Matsuo <tmatsuo@google.com>
Co-authored-by: Leah Cole <coleleah@google.com>
Co-authored-by: Bu Sun Kim <busunkim@google.com>
ivanmkc pushed a commit that referenced this pull request Nov 3, 2022
* Data Labeling Beta samples [(#2096)](#2096)

* add files

* upate create_annotation_spec_set and test

* add requirements.txt

* update create_instruction and test

* update import data and test

* add label image and test

* add label_text test

* add label_video_test

* add manage dataset and tests

* flake

* fix

* add README

* Adds updates including compute [(#2436)](#2436)

* Adds updates including compute

* Python 2 compat pytest

* Fixing weird \r\n issue from GH merge

* Put asset tests back in

* Re-add pod operator test

* Hack parameter for k8s pod operator

* Update datalabeling samples to hit test endpoint. [(#2641)](#2641)

* Auto-update dependencies. [(#2005)](#2005)

* Auto-update dependencies.

* Revert update of appengine/flexible/datastore.

* revert update of appengine/flexible/scipy

* revert update of bigquery/bqml

* revert update of bigquery/cloud-client

* revert update of bigquery/datalab-migration

* revert update of bigtable/quickstart

* revert update of compute/api

* revert update of container_registry/container_analysis

* revert update of dataflow/run_template

* revert update of datastore/cloud-ndb

* revert update of dialogflow/cloud-client

* revert update of dlp

* revert update of functions/imagemagick

* revert update of functions/ocr/app

* revert update of healthcare/api-client/fhir

* revert update of iam/api-client

* revert update of iot/api-client/gcs_file_to_device

* revert update of iot/api-client/mqtt_example

* revert update of language/automl

* revert update of run/image-processing

* revert update of vision/automl

* revert update testing/requirements.txt

* revert update of vision/cloud-client/detect

* revert update of vision/cloud-client/product_search

* revert update of jobs/v2/api_client

* revert update of jobs/v3/api_client

* revert update of opencensus

* revert update of translate/cloud-client

* revert update to speech/cloud-client

Co-authored-by: Kurtis Van Gent <31518063+kurtisvg@users.noreply.github.com>
Co-authored-by: Doug Mahugh <dmahugh@gmail.com>

* Update datalabeling to match lint. [(#2642)](#2642)

* datalabeling: ensure all tests use test endpoint [(#2918)](#2918)

* datalabeling: ensure all tests use test endpoint

* requires an input csv for text input, slight print statement cleanup

Co-authored-by: Leah E. Cole <6719667+leahecole@users.noreply.github.com>

* chore(deps): update dependency google-cloud-datalabeling to v0.4.0 [(#3081)](#3081)

This PR contains the following updates:

| Package | Update | Change |
|---|---|---|
| [google-cloud-datalabeling](https://togithub.com/googleapis/python-datalabeling) | minor | `==0.3.0` -> `==0.4.0` |

---

### Release Notes

<details>
<summary>googleapis/python-datalabeling</summary>

### [`v0.4.0`](https://togithub.com/googleapis/python-datalabeling/blob/master/CHANGELOG.md#&#8203;040-httpswwwgithubcomgoogleapispython-datalabelingcomparev030v040-2020-01-31)

[Compare Source](https://togithub.com/googleapis/python-datalabeling/compare/v0.3.0...v0.4.0)

##### Features

-   **datalabeling:** undeprecate resource name helper methods (via synth) ([#&#8203;10039](https://www.github.com/googleapis/python-datalabeling/issues/10039)) ([88f8090](https://www.github.com/googleapis/python-datalabeling/commit/88f809008ee6a709c02c78b1d93af779fab19adb))

##### Bug Fixes

-   **datalabeling:** deprecate resource name helper methods (via synth) ([#&#8203;9832](https://www.github.com/googleapis/python-datalabeling/issues/9832)) ([e5f9021](https://www.github.com/googleapis/python-datalabeling/commit/e5f902154ebe7fcb139aa405babfe9993fd51319))

</details>

---

### Renovate configuration

:date: **Schedule**: At any time (no schedule defined).

:vertical_traffic_light: **Automerge**: Disabled by config. Please merge this manually once you are satisfied.

:recycle: **Rebasing**: Never, or you tick the rebase/retry checkbox.

:no_bell: **Ignore**: Close this PR and you won't be reminded about this update again.

---

 - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box

---

This PR has been generated by [WhiteSource Renovate](https://renovate.whitesourcesoftware.com). View repository job log [here](https://app.renovatebot.com/dashboard#GoogleCloudPlatform/python-docs-samples).

* Simplify noxfile setup. [(#2806)](#2806)

* chore(deps): update dependency requests to v2.23.0

* Simplify noxfile and add version control.

* Configure appengine/standard to only test Python 2.7.

* Update Kokokro configs to match noxfile.

* Add requirements-test to each folder.

* Remove Py2 versions from everything execept appengine/standard.

* Remove conftest.py.

* Remove appengine/standard/conftest.py

* Remove 'no-sucess-flaky-report' from pytest.ini.

* Add GAE SDK back to appengine/standard tests.

* Fix typo.

* Roll pytest to python 2 version.

* Add a bunch of testing requirements.

* Remove typo.

* Add appengine lib directory back in.

* Add some additional requirements.

* Fix issue with flake8 args.

* Even more requirements.

* Readd appengine conftest.py.

* Add a few more requirements.

* Even more Appengine requirements.

* Add webtest for appengine/standard/mailgun.

* Add some additional requirements.

* Add workaround for issue with mailjet-rest.

* Add responses for appengine/standard/mailjet.

Co-authored-by: Renovate Bot <bot@renovateapp.com>

* testing: mark some tests as flaky [(#3288)](#3288)

fixes #3138

* [datalabeling] testing: wrap rpcs with backoff [(#3443)](#3443)

* wrap all the rpcs with backoff
* add a shared testing lib
* remove flaky

* [datalabeling] fix: clean up old datasets before the test [(#3707)](#3707)

fixes #3710 
fixes #3711

* [datalabeling] testing: retry upon ServerError [(#3762)](#3762)

fixes #3760

* Replace GCLOUD_PROJECT with GOOGLE_CLOUD_PROJECT. [(#4022)](#4022)

* chore(deps): update dependency pytest to v5.4.3 [(#4279)](#4279)

* chore(deps): update dependency pytest to v5.4.3

* specify pytest for python 2 in appengine

Co-authored-by: Leah Cole <coleleah@google.com>

* Update dependency pytest to v6 [(#4390)](#4390)

* chore: update templates

* chore: fix docs error

* chore: skip unavailable samples

* chore: use staging endpoint for labeling tests

Co-authored-by: Rebecca Taylor <remilytaylor@gmail.com>
Co-authored-by: Gus Class <gguuss@gmail.com>
Co-authored-by: Kurtis Van Gent <31518063+kurtisvg@users.noreply.github.com>
Co-authored-by: DPEBot <dpebot@google.com>
Co-authored-by: Doug Mahugh <dmahugh@gmail.com>
Co-authored-by: Noah Negrey <nnegrey@users.noreply.github.com>
Co-authored-by: Leah E. Cole <6719667+leahecole@users.noreply.github.com>
Co-authored-by: WhiteSource Renovate <bot@renovateapp.com>
Co-authored-by: Takashi Matsuo <tmatsuo@google.com>
Co-authored-by: Leah Cole <coleleah@google.com>
Co-authored-by: Bu Sun Kim <busunkim@google.com>
ivanmkc pushed a commit that referenced this pull request Nov 4, 2022
* Data Labeling Beta samples [(#2096)](#2096)

* add files

* upate create_annotation_spec_set and test

* add requirements.txt

* update create_instruction and test

* update import data and test

* add label image and test

* add label_text test

* add label_video_test

* add manage dataset and tests

* flake

* fix

* add README

* Adds updates including compute [(#2436)](#2436)

* Adds updates including compute

* Python 2 compat pytest

* Fixing weird \r\n issue from GH merge

* Put asset tests back in

* Re-add pod operator test

* Hack parameter for k8s pod operator

* Update datalabeling samples to hit test endpoint. [(#2641)](#2641)

* Auto-update dependencies. [(#2005)](#2005)

* Auto-update dependencies.

* Revert update of appengine/flexible/datastore.

* revert update of appengine/flexible/scipy

* revert update of bigquery/bqml

* revert update of bigquery/cloud-client

* revert update of bigquery/datalab-migration

* revert update of bigtable/quickstart

* revert update of compute/api

* revert update of container_registry/container_analysis

* revert update of dataflow/run_template

* revert update of datastore/cloud-ndb

* revert update of dialogflow/cloud-client

* revert update of dlp

* revert update of functions/imagemagick

* revert update of functions/ocr/app

* revert update of healthcare/api-client/fhir

* revert update of iam/api-client

* revert update of iot/api-client/gcs_file_to_device

* revert update of iot/api-client/mqtt_example

* revert update of language/automl

* revert update of run/image-processing

* revert update of vision/automl

* revert update testing/requirements.txt

* revert update of vision/cloud-client/detect

* revert update of vision/cloud-client/product_search

* revert update of jobs/v2/api_client

* revert update of jobs/v3/api_client

* revert update of opencensus

* revert update of translate/cloud-client

* revert update to speech/cloud-client

Co-authored-by: Kurtis Van Gent <31518063+kurtisvg@users.noreply.github.com>
Co-authored-by: Doug Mahugh <dmahugh@gmail.com>

* Update datalabeling to match lint. [(#2642)](#2642)

* datalabeling: ensure all tests use test endpoint [(#2918)](#2918)

* datalabeling: ensure all tests use test endpoint

* requires an input csv for text input, slight print statement cleanup

Co-authored-by: Leah E. Cole <6719667+leahecole@users.noreply.github.com>

* chore(deps): update dependency google-cloud-datalabeling to v0.4.0 [(#3081)](#3081)

This PR contains the following updates:

| Package | Update | Change |
|---|---|---|
| [google-cloud-datalabeling](https://togithub.com/googleapis/python-datalabeling) | minor | `==0.3.0` -> `==0.4.0` |

---

### Release Notes

<details>
<summary>googleapis/python-datalabeling</summary>

### [`v0.4.0`](https://togithub.com/googleapis/python-datalabeling/blob/master/CHANGELOG.md#&#8203;040-httpswwwgithubcomgoogleapispython-datalabelingcomparev030v040-2020-01-31)

[Compare Source](https://togithub.com/googleapis/python-datalabeling/compare/v0.3.0...v0.4.0)

##### Features

-   **datalabeling:** undeprecate resource name helper methods (via synth) ([#&#8203;10039](https://www.github.com/googleapis/python-datalabeling/issues/10039)) ([88f8090](https://www.github.com/googleapis/python-datalabeling/commit/88f809008ee6a709c02c78b1d93af779fab19adb))

##### Bug Fixes

-   **datalabeling:** deprecate resource name helper methods (via synth) ([#&#8203;9832](https://www.github.com/googleapis/python-datalabeling/issues/9832)) ([e5f9021](https://www.github.com/googleapis/python-datalabeling/commit/e5f902154ebe7fcb139aa405babfe9993fd51319))

</details>

---

### Renovate configuration

:date: **Schedule**: At any time (no schedule defined).

:vertical_traffic_light: **Automerge**: Disabled by config. Please merge this manually once you are satisfied.

:recycle: **Rebasing**: Never, or you tick the rebase/retry checkbox.

:no_bell: **Ignore**: Close this PR and you won't be reminded about this update again.

---

 - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box

---

This PR has been generated by [WhiteSource Renovate](https://renovate.whitesourcesoftware.com). View repository job log [here](https://app.renovatebot.com/dashboard#GoogleCloudPlatform/python-docs-samples).

* Simplify noxfile setup. [(#2806)](#2806)

* chore(deps): update dependency requests to v2.23.0

* Simplify noxfile and add version control.

* Configure appengine/standard to only test Python 2.7.

* Update Kokokro configs to match noxfile.

* Add requirements-test to each folder.

* Remove Py2 versions from everything execept appengine/standard.

* Remove conftest.py.

* Remove appengine/standard/conftest.py

* Remove 'no-sucess-flaky-report' from pytest.ini.

* Add GAE SDK back to appengine/standard tests.

* Fix typo.

* Roll pytest to python 2 version.

* Add a bunch of testing requirements.

* Remove typo.

* Add appengine lib directory back in.

* Add some additional requirements.

* Fix issue with flake8 args.

* Even more requirements.

* Readd appengine conftest.py.

* Add a few more requirements.

* Even more Appengine requirements.

* Add webtest for appengine/standard/mailgun.

* Add some additional requirements.

* Add workaround for issue with mailjet-rest.

* Add responses for appengine/standard/mailjet.

Co-authored-by: Renovate Bot <bot@renovateapp.com>

* testing: mark some tests as flaky [(#3288)](#3288)

fixes #3138

* [datalabeling] testing: wrap rpcs with backoff [(#3443)](#3443)

* wrap all the rpcs with backoff
* add a shared testing lib
* remove flaky

* [datalabeling] fix: clean up old datasets before the test [(#3707)](#3707)

fixes #3710 
fixes #3711

* [datalabeling] testing: retry upon ServerError [(#3762)](#3762)

fixes #3760

* Replace GCLOUD_PROJECT with GOOGLE_CLOUD_PROJECT. [(#4022)](#4022)

* chore(deps): update dependency pytest to v5.4.3 [(#4279)](#4279)

* chore(deps): update dependency pytest to v5.4.3

* specify pytest for python 2 in appengine

Co-authored-by: Leah Cole <coleleah@google.com>

* Update dependency pytest to v6 [(#4390)](#4390)

* chore: update templates

* chore: fix docs error

* chore: skip unavailable samples

* chore: use staging endpoint for labeling tests

Co-authored-by: Rebecca Taylor <remilytaylor@gmail.com>
Co-authored-by: Gus Class <gguuss@gmail.com>
Co-authored-by: Kurtis Van Gent <31518063+kurtisvg@users.noreply.github.com>
Co-authored-by: DPEBot <dpebot@google.com>
Co-authored-by: Doug Mahugh <dmahugh@gmail.com>
Co-authored-by: Noah Negrey <nnegrey@users.noreply.github.com>
Co-authored-by: Leah E. Cole <6719667+leahecole@users.noreply.github.com>
Co-authored-by: WhiteSource Renovate <bot@renovateapp.com>
Co-authored-by: Takashi Matsuo <tmatsuo@google.com>
Co-authored-by: Leah Cole <coleleah@google.com>
Co-authored-by: Bu Sun Kim <busunkim@google.com>
parthea pushed a commit to googleapis/google-cloud-python that referenced this pull request Jun 4, 2023
* Data Labeling Beta samples [(#2096)](GoogleCloudPlatform/python-docs-samples#2096)

* add files

* upate create_annotation_spec_set and test

* add requirements.txt

* update create_instruction and test

* update import data and test

* add label image and test

* add label_text test

* add label_video_test

* add manage dataset and tests

* flake

* fix

* add README

* Adds updates including compute [(#2436)](GoogleCloudPlatform/python-docs-samples#2436)

* Adds updates including compute

* Python 2 compat pytest

* Fixing weird \r\n issue from GH merge

* Put asset tests back in

* Re-add pod operator test

* Hack parameter for k8s pod operator

* Update datalabeling samples to hit test endpoint. [(#2641)](GoogleCloudPlatform/python-docs-samples#2641)

* Auto-update dependencies. [(#2005)](GoogleCloudPlatform/python-docs-samples#2005)

* Auto-update dependencies.

* Revert update of appengine/flexible/datastore.

* revert update of appengine/flexible/scipy

* revert update of bigquery/bqml

* revert update of bigquery/cloud-client

* revert update of bigquery/datalab-migration

* revert update of bigtable/quickstart

* revert update of compute/api

* revert update of container_registry/container_analysis

* revert update of dataflow/run_template

* revert update of datastore/cloud-ndb

* revert update of dialogflow/cloud-client

* revert update of dlp

* revert update of functions/imagemagick

* revert update of functions/ocr/app

* revert update of healthcare/api-client/fhir

* revert update of iam/api-client

* revert update of iot/api-client/gcs_file_to_device

* revert update of iot/api-client/mqtt_example

* revert update of language/automl

* revert update of run/image-processing

* revert update of vision/automl

* revert update testing/requirements.txt

* revert update of vision/cloud-client/detect

* revert update of vision/cloud-client/product_search

* revert update of jobs/v2/api_client

* revert update of jobs/v3/api_client

* revert update of opencensus

* revert update of translate/cloud-client

* revert update to speech/cloud-client

Co-authored-by: Kurtis Van Gent <31518063+kurtisvg@users.noreply.github.com>
Co-authored-by: Doug Mahugh <dmahugh@gmail.com>

* Update datalabeling to match lint. [(#2642)](GoogleCloudPlatform/python-docs-samples#2642)

* datalabeling: ensure all tests use test endpoint [(#2918)](GoogleCloudPlatform/python-docs-samples#2918)

* datalabeling: ensure all tests use test endpoint

* requires an input csv for text input, slight print statement cleanup

Co-authored-by: Leah E. Cole <6719667+leahecole@users.noreply.github.com>

* chore(deps): update dependency google-cloud-datalabeling to v0.4.0 [(#3081)](GoogleCloudPlatform/python-docs-samples#3081)

This PR contains the following updates:

| Package | Update | Change |
|---|---|---|
| [google-cloud-datalabeling](https://togithub.com/googleapis/python-datalabeling) | minor | `==0.3.0` -> `==0.4.0` |

---

### Release Notes

<details>
<summary>googleapis/python-datalabeling</summary>

### [`v0.4.0`](https://togithub.com/googleapis/python-datalabeling/blob/master/CHANGELOG.md#&#8203;040-httpswwwgithubcomgoogleapispython-datalabelingcomparev030v040-2020-01-31)

[Compare Source](https://togithub.com/googleapis/python-datalabeling/compare/v0.3.0...v0.4.0)

##### Features

-   **datalabeling:** undeprecate resource name helper methods (via synth) ([#&#8203;10039](https://www.github.com/googleapis/python-datalabeling/issues/10039)) ([88f8090](https://www.github.com/googleapis/python-datalabeling/commit/88f809008ee6a709c02c78b1d93af779fab19adb))

##### Bug Fixes

-   **datalabeling:** deprecate resource name helper methods (via synth) ([#&#8203;9832](https://www.github.com/googleapis/python-datalabeling/issues/9832)) ([e5f9021](https://www.github.com/googleapis/python-datalabeling/commit/e5f902154ebe7fcb139aa405babfe9993fd51319))

</details>

---

### Renovate configuration

:date: **Schedule**: At any time (no schedule defined).

:vertical_traffic_light: **Automerge**: Disabled by config. Please merge this manually once you are satisfied.

:recycle: **Rebasing**: Never, or you tick the rebase/retry checkbox.

:no_bell: **Ignore**: Close this PR and you won't be reminded about this update again.

---

 - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box

---

This PR has been generated by [WhiteSource Renovate](https://renovate.whitesourcesoftware.com). View repository job log [here](https://app.renovatebot.com/dashboard#GoogleCloudPlatform/python-docs-samples).

* Simplify noxfile setup. [(#2806)](GoogleCloudPlatform/python-docs-samples#2806)

* chore(deps): update dependency requests to v2.23.0

* Simplify noxfile and add version control.

* Configure appengine/standard to only test Python 2.7.

* Update Kokokro configs to match noxfile.

* Add requirements-test to each folder.

* Remove Py2 versions from everything execept appengine/standard.

* Remove conftest.py.

* Remove appengine/standard/conftest.py

* Remove 'no-sucess-flaky-report' from pytest.ini.

* Add GAE SDK back to appengine/standard tests.

* Fix typo.

* Roll pytest to python 2 version.

* Add a bunch of testing requirements.

* Remove typo.

* Add appengine lib directory back in.

* Add some additional requirements.

* Fix issue with flake8 args.

* Even more requirements.

* Readd appengine conftest.py.

* Add a few more requirements.

* Even more Appengine requirements.

* Add webtest for appengine/standard/mailgun.

* Add some additional requirements.

* Add workaround for issue with mailjet-rest.

* Add responses for appengine/standard/mailjet.

Co-authored-by: Renovate Bot <bot@renovateapp.com>

* testing: mark some tests as flaky [(#3288)](GoogleCloudPlatform/python-docs-samples#3288)

fixes #3138

* [datalabeling] testing: wrap rpcs with backoff [(#3443)](GoogleCloudPlatform/python-docs-samples#3443)

* wrap all the rpcs with backoff
* add a shared testing lib
* remove flaky

* [datalabeling] fix: clean up old datasets before the test [(#3707)](GoogleCloudPlatform/python-docs-samples#3707)

fixes #3710 
fixes #3711

* [datalabeling] testing: retry upon ServerError [(#3762)](GoogleCloudPlatform/python-docs-samples#3762)

fixes #3760

* Replace GCLOUD_PROJECT with GOOGLE_CLOUD_PROJECT. [(#4022)](GoogleCloudPlatform/python-docs-samples#4022)

* chore(deps): update dependency pytest to v5.4.3 [(#4279)](GoogleCloudPlatform/python-docs-samples#4279)

* chore(deps): update dependency pytest to v5.4.3

* specify pytest for python 2 in appengine

Co-authored-by: Leah Cole <coleleah@google.com>

* Update dependency pytest to v6 [(#4390)](GoogleCloudPlatform/python-docs-samples#4390)

* chore: update templates

* chore: fix docs error

* chore: skip unavailable samples

* chore: use staging endpoint for labeling tests

Co-authored-by: Rebecca Taylor <remilytaylor@gmail.com>
Co-authored-by: Gus Class <gguuss@gmail.com>
Co-authored-by: Kurtis Van Gent <31518063+kurtisvg@users.noreply.github.com>
Co-authored-by: DPEBot <dpebot@google.com>
Co-authored-by: Doug Mahugh <dmahugh@gmail.com>
Co-authored-by: Noah Negrey <nnegrey@users.noreply.github.com>
Co-authored-by: Leah E. Cole <6719667+leahecole@users.noreply.github.com>
Co-authored-by: WhiteSource Renovate <bot@renovateapp.com>
Co-authored-by: Takashi Matsuo <tmatsuo@google.com>
Co-authored-by: Leah Cole <coleleah@google.com>
Co-authored-by: Bu Sun Kim <busunkim@google.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
cla: yes This human has signed the Contributor License Agreement.
Projects
None yet
6 participants