Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

test: E2E test on endpoint versions only #10441

Merged
merged 5 commits into from
Aug 2, 2023
Merged

test: E2E test on endpoint versions only #10441

merged 5 commits into from
Aug 2, 2023

Conversation

kweinmeister
Copy link
Contributor

@kweinmeister kweinmeister commented Jul 20, 2023

To reduce testing overhead and costs, update default test matrix to endpoints. Currently these are 3.7 and 3.11, but since Python 3.7 is EOL on 6/27/2003, these endpoints have been set to 3.8 and 3.11.

This ensures that dependency conflicts are detected. The earliest and latest versions are checked.

Samples with non-standard test configurations are not affected by this change.

@product-auto-label product-auto-label bot added samples Issues that are directly related to samples. api: appengine Issues related to the App Engine Admin API API. api: asset api: auth Issues related to the Google Authentication API api: bigquery Issues related to the BigQuery API. api: cloudbuild Issues related to the Cloud Build API. api: cloudmedia Issues related to the Media Livestream API. api: compute Issues related to the Compute Engine API. api: contactcenterinsights Issues related to the Contact Center AI Insights API. api: container Issues related to the Kubernetes Engine API API. api: datalabeling Issues related to the AI Platform Data Labeling Service API. api: tasks Issues related to the Tasks API API. api: translate Issues related to the Cloud Translation API. api: vision Issues related to the Cloud Vision API. labels Jul 20, 2023
@kweinmeister kweinmeister added the snippet-bot:force-run Force snippet-bot runs its logic label Jul 20, 2023
@snippet-bot snippet-bot bot removed the snippet-bot:force-run Force snippet-bot runs its logic label Jul 20, 2023
@kweinmeister kweinmeister marked this pull request as ready for review July 20, 2023 20:29
@kweinmeister kweinmeister requested review from a team and chalmerlowe as code owners July 20, 2023 20:29
@kweinmeister kweinmeister requested a review from a team July 20, 2023 20:29
@kweinmeister kweinmeister requested review from a team as code owners July 20, 2023 20:29
@kweinmeister
Copy link
Contributor Author

A couple test cases are failing on unrelated issues.

Copy link
Member

@rsamborski rsamborski left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

According to https://devguide.python.org/versions/, Python 3.7 is EOL from 2023-06-27. I suggest to turn off 3.7 testing completely and keep testing on 3.8 and 3.11 by default. I think it should be agreed with the language team before proceeding.

@kweinmeister
Copy link
Contributor Author

According to https://devguide.python.org/versions/, Python 3.7 is EOL from 2023-06-27. I suggest to turn off 3.7 testing completely and keep testing on 3.8 and 3.11 by default. I think it should be agreed with the language team before proceeding.

Updated to 3.8 & 3.11 based on feedback. Python team can review and provide feedback before merging. Thanks!

@dandhlee
Copy link
Collaborator

It might require some changes to how we run the tests, but would it be now then better to change ignored_versions option to test_additional_versions? Now we'd expect less products to run version specific tests other than the endpoint versions - which we can also control much easily when there's endpoint changes - and from a very quick skim it seems like all the changes are mostly for default version changes.

@kweinmeister
Copy link
Contributor Author

@dandhlee I like your proposal. How do you feel about approaching it in 2 steps?

  1. Updating the tested versions using the current noxfile config
  2. Refactoring noxfile config

We can then verify each step independently, including ensuring that step 2 tests the same versions as 1. Step 2 may also have some edge cases that we haven't thought about yet. I am open either way though, if you feel we should combine those in this PR. Thanks!

@kweinmeister
Copy link
Contributor Author

@rsamborski - I should update this PR to remove 3.7 references after we merge #10453? IIUC, they should just be noise now, since 3.7 won't be tested.

@rsamborski
Copy link
Member

@rsamborski - I should update this PR to remove 3.7 references after we merge #10453? IIUC, they should just be noise now, since 3.7 won't be tested.

Feel free to leave 3.7 as is (noise). We can remove it together with noxfile config refactoring (step 2 from your list above).

@kweinmeister kweinmeister added the kokoro:force-run Add this label to force Kokoro to re-run the tests. label Jul 31, 2023
@kokoro-team kokoro-team removed the kokoro:force-run Add this label to force Kokoro to re-run the tests. label Jul 31, 2023
@kweinmeister kweinmeister requested a review from a team as a code owner August 2, 2023 14:42
@kweinmeister
Copy link
Contributor Author

Approvals in place. A couple tests failed due to unrelated issues. Admin-merging.

@kweinmeister kweinmeister merged commit 3aa00a7 into main Aug 2, 2023
@kweinmeister kweinmeister deleted the test-endpoints branch August 2, 2023 15:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
api: appengine Issues related to the App Engine Admin API API. api: asset api: auth Issues related to the Google Authentication API api: bigquery Issues related to the BigQuery API. api: cloudbuild Issues related to the Cloud Build API. api: cloudmedia Issues related to the Media Livestream API. api: compute Issues related to the Compute Engine API. api: contactcenterinsights Issues related to the Contact Center AI Insights API. api: container Issues related to the Kubernetes Engine API API. api: datalabeling Issues related to the AI Platform Data Labeling Service API. api: tasks Issues related to the Tasks API API. api: translate Issues related to the Cloud Translation API. api: vision Issues related to the Cloud Vision API. samples Issues that are directly related to samples.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants