Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Deferrable mode for Custom Training Job operators #38584

Conversation

e-galan
Copy link
Contributor

@e-galan e-galan commented Mar 28, 2024

The PR adds the deferrable mode feature for the following Vertex AI operators:

  • CreateCustomContainerTrainingJobOperator
  • CreateCustomPythonPackageTrainingJobOperator
  • CreateCustomTrainingJobOperator

Other changes include:

  • Fix for VertexAIModelLink. The old link url did not work anymore, I replaced it with a working one.
  • New unit tests
  • Update example dags

^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named {pr_number}.significant.rst or {issue_number}.significant.rst, in newsfragments.

@VladaZakharova
Copy link
Contributor

Hi there!
@eladkal @Lee-W @potiuk
Can you please check the PR here? :)

@Lee-W Lee-W self-requested a review April 3, 2024 09:43
@e-galan e-galan force-pushed the add-defer-mode-to-custom-training-job-operators branch from cdba3ae to fd8d26e Compare April 4, 2024 10:33
@e-galan e-galan requested a review from Lee-W April 4, 2024 10:38
@e-galan e-galan force-pushed the add-defer-mode-to-custom-training-job-operators branch 2 times, most recently from 62877c7 to ce47e26 Compare April 4, 2024 12:52
Copy link
Member

@Lee-W Lee-W left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Left a few nitpicks. But overall looks good to me!

@e-galan e-galan force-pushed the add-defer-mode-to-custom-training-job-operators branch 3 times, most recently from 8ca1e8e to cd4e7f7 Compare April 9, 2024 20:58
@e-galan e-galan force-pushed the add-defer-mode-to-custom-training-job-operators branch from cd4e7f7 to fbb0d89 Compare April 10, 2024 08:10
@e-galan
Copy link
Contributor Author

e-galan commented Apr 11, 2024

Hi @Lee-W and @potiuk! Could anyone merge this PR please?

@Lee-W Lee-W merged commit 5ff2658 into apache:main Apr 11, 2024
40 checks passed
utkarsharma2 pushed a commit to astronomer/airflow that referenced this pull request Apr 22, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants