Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

SDK - Components - Refactoring - _resolve_graph_task now creates tasks directly #3448

Conversation

Ark-kun
Copy link
Contributor

@Ark-kun Ark-kun commented Apr 6, 2020

This PR builds on top of #3447

This change is a pure refactoring.
Previously _resolve_graph_task was creating task-like objects by materializing components and calling the resulting factory functions. One problem with this approach is that it prevented task options from reaching the _container_task_constructor. With this change, the _resolve_graph_task function directly calls the extracted _create_task_object_from_task_spec_and_arguments function and all task options are preserved.

…nd graph handlers

This is a refactoring PR.
The main goal of the PR is to make _components._container_task_constructor receive TaskSpec.execution_options, is_enabled and any options that might be added in the future. Otherwise, the customizations of tasks in a graph component are lost.

This PR partially reverses the previous refactoring which switched away from TaskSpec usage.

The interface for _components._container_task_constructor has changed from (component_spec, arguments, component_ref) to (task_spec, arguments).
The reason is that task_spec has additional attribues (execution_options) that should be passed in.
It looks weird to pass arguments separately (as task_spec can already hold arguments), but the reason for this is that the passed arguments may have types that are incompatible with TaskSpec.arguments. So the arguments are passed separately.
The interface is private, so it's fine to make a breaking change here as we control all implementations.
…s directly

This change is a pure refactoring.
Previously _resolve_graph_task was creating task-like objects by materializing components and calling the resulting factory functions. One problem with this approach is that it prevented task options from reaching the _container_task_constructor. With this change, the _resolve_graph_task function directly calls the extracted _create_task_object_from_task_spec_and_arguments function and all task options are preserved.
@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
To complete the pull request process, please assign ark-kun
You can assign the PR to them by writing /assign @ark-kun in a comment when ready.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@kubeflow-bot
Copy link

This change is Reviewable

@stale
Copy link

stale bot commented Jul 5, 2020

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

@stale stale bot added the lifecycle/stale The issue / pull request is stale, any activities remove this label. label Jul 5, 2020
@Ark-kun Ark-kun removed the lifecycle/stale The issue / pull request is stale, any activities remove this label. label Jul 6, 2020
@k8s-ci-robot
Copy link
Contributor

@Ark-kun: The following tests failed, say /retest to rerun all failed tests:

Test name Commit Details Rerun command
kubeflow-pipelines-sdk-python37 157fbbb link /test kubeflow-pipelines-sdk-python37
kubeflow-pipelines-sdk-python38 157fbbb link /test kubeflow-pipelines-sdk-python38
kubeflow-pipelines-sdk-python36 157fbbb link /test kubeflow-pipelines-sdk-python36
kubeflow-pipelines-sdk-python35 157fbbb link /test kubeflow-pipelines-sdk-python35

Full PR test history. Your PR dashboard. Please help us cut down on flakes by linking to an open issue when you hit one in your PR.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@stale
Copy link

stale bot commented Oct 19, 2020

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

@stale stale bot added the lifecycle/stale The issue / pull request is stale, any activities remove this label. label Oct 19, 2020
@stale
Copy link

stale bot commented Oct 28, 2020

This issue has been automatically closed because it has not had recent activity. Please comment "/reopen" to reopen it.

@stale stale bot closed this Oct 28, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/sdk/components lifecycle/stale The issue / pull request is stale, any activities remove this label. size/L
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants