Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

SDK - Fixed small bug in DSL code that generates unique names for ops #923

Conversation

Ark-kun
Copy link
Contributor

@Ark-kun Ark-kun commented Mar 6, 2019

Before the fix it would generate names as follows:
name
name-2
name-2-3
After the fix:
name
name-2
name-3


This change is Reviewable

Before the fix it would generate names as follows:
name
name-2
name-2-3
@Ark-kun Ark-kun changed the title Fixed small bug in code that generates unique names for ops Fixed small bug in DSL code that generates unique names for ops Mar 6, 2019
Copy link
Contributor

@hongye-sun hongye-sun left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/lgtm

@Ark-kun
Copy link
Contributor Author

Ark-kun commented Mar 6, 2019

/approve

@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: Ark-kun

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

1 similar comment
@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: Ark-kun

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@Ark-kun Ark-kun merged commit b68fbbd into kubeflow:master Mar 6, 2019
@Ark-kun Ark-kun deleted the Fixed-small-bug-in-code-that-generates-unique-names-for-ops branch March 6, 2019 22:24
@Ark-kun Ark-kun changed the title Fixed small bug in DSL code that generates unique names for ops SDK - Fixed small bug in DSL code that generates unique names for ops Mar 8, 2019
@gaoning777
Copy link
Contributor

Not in favor of creating dependency of the DSL on component. This name sanitization could happen in the second stage of the compiler.

cheyang pushed a commit to alibaba/pipelines that referenced this pull request Mar 28, 2019
…flow#923)

Before the fix it would generate names as follows:
name
name-2
name-2-3

After the fix:
name
name-2
name-3
@Ark-kun
Copy link
Contributor Author

Ark-kun commented Mar 29, 2019

@gaoning777

This name sanitization could happen in the second stage of the compiler.

Unfortunately, that's not the case. It's not about sanitization - it's about uniqueness. Currently, outputs refer to their task by name. If multiple tasks have the same name, the compiler has no idea which op to link to. And given that the PipelineParam can be serialized to string (which might not be a good feature), the problem becomes worse.

An example:

op1 = ContainerOp(name='bla', ...)
op2 = ContainerOp(name='bla', ...)

op3 = ContainerOp(arguments=[ "ouputs/%s" % op2.output ])

How would the compiler distinguish between str(op1.output) and str(op2.output)?

If you can improve this code, I have absolutely no problem with that.

@gaoning777
Copy link
Contributor

the name was unique, wasn't it?
your fix changed the unique name from name-2-3 to name-3 if I'm understanding correctly?

Linchin pushed a commit to Linchin/pipelines that referenced this pull request Apr 11, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants