Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

SDK - Compiler - Fixed handling of PipelineParams in artifact arguments #2042

Conversation

Ark-kun
Copy link
Contributor

@Ark-kun Ark-kun commented Sep 5, 2019

Previously only constant strings were supported and serialized PipelineParams were not resolved, producing incorrect workflows.


This change is Reviewable

Previously only constant strings were supported and serialized PipelineParams were not resolved, producing incorrect workflows.
@Ark-kun
Copy link
Contributor Author

Ark-kun commented Sep 5, 2019

/retest

@@ -138,6 +139,8 @@ def _inputs_to_json(
artifacts = []
for name, path in (input_artifact_paths or {}).items():
artifact = {'name': name, 'path': path}
if name in artifact_arguments: # The arguments should be compiled as DAG task arguments, not template's default values, but in the current DSL-compiler implementation it's too hard to make that work when passing artifact references.
artifact['raw'] = {'data': str(artifact_arguments[name])}

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry I'm not familiar with this. Are all artifacts guaranteed to be serializable?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good idea for the next PR.
Right now we only support two types of artifact arguments - constant string and PipelineParam reference to some task's output, which is also serializable.
In future I can put support for serializing complex types here so that it's on par with the component argument serialization support.

@numerology
Copy link

/lgtm

@Ark-kun
Copy link
Contributor Author

Ark-kun commented Sep 5, 2019

/approve

@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: Ark-kun

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@k8s-ci-robot k8s-ci-robot merged commit f911742 into kubeflow:master Sep 5, 2019
@Ark-kun Ark-kun deleted the SDK---Compiler---Fixed-handling-of-PipelineParams-in-artifact-arguments branch September 5, 2019 22:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants