Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

How to use a pipeline parameter containing spaces in Condition? #1950

Closed
haudren opened this issue Aug 26, 2019 · 3 comments · Fixed by #2043
Closed

How to use a pipeline parameter containing spaces in Condition? #1950

haudren opened this issue Aug 26, 2019 · 3 comments · Fixed by #2043
Assignees
Labels
area/sdk/dsl/compiler help wanted The community is welcome to contribute. kind/bug

Comments

@haudren
Copy link

haudren commented Aug 26, 2019

What happened:
I am using a pipeline parameter, and would like to use a Condition to switch on it existing / not existing. The condition I used is of the form parameter != '""'. When the parameter contains spaces (say parameter = "a b c"), it fails with:

This step is in Error state with this message: Invalid 'when' expression 'a b c != ""': Cannot transition token types from VARIABLE [a] to VARIABLE [b]

Another attempt with a, b, c results in:

This step is in Error state with this message: Expected boolean evaluation for 'a, b, c  != ""'. Got [a b true]

A quick look at the generated yaml shows that the parameter is compared unquoted which means that comparing it to a single string breaks...

What did you expect to happen:
I expect one of two things:

  • There is a simple way to check for a single parameter to be set / not set that does not depend on the input containing strings
  • There is a way to sanitize (quote) the input parameters before comparison

What steps did you take:
The pseudo-code:

import kfp.dsl as dsl
import kfp.components as comp
import os


def hello():
    print("Hello World")


hello_op = comp.func_to_container_op(hello)


@dsl.pipeline()
def my_pipeline(parameter="a b c"):
    with dsl.Condition(parameter == '!"'):
        hello_op()


if __name__ == "__main__":
    import kfp.compiler as compiler

    compiler.Compiler().compile(my_pipeline, os.path.splitext(__file__)[0] + ".yaml")

Anything else you would like to add:
I think this might be related to #1933 as this parameter is for all intents and purposes a list that I then pass to a shell script. I don't mind doing bits of parsing myself for the time being, but I'd like to not have the pipeline crash when spaces are passed in.

@Ark-kun Ark-kun self-assigned this Aug 26, 2019
@Ark-kun
Copy link
Contributor

Ark-kun commented Aug 26, 2019

This seems to be a quoting issue. I can try to fix it on our side, but it might be a bug in Argo.

@Ark-kun
Copy link
Contributor

Ark-kun commented Aug 26, 2019

Does it work when your constant string is simple?
with dsl.Condition(parameter == 'abc'):

@Ark-kun Ark-kun added area/sdk/dsl/compiler help wanted The community is welcome to contribute. kind/bug labels Aug 26, 2019
Ark-kun added a commit to Ark-kun/pipelines that referenced this issue Aug 26, 2019
@haudren
Copy link
Author

haudren commented Aug 27, 2019

No, it seems like for most purposes, the value is interpreted literally. Stopgap fix I used is to replace spaces with underscores (most other separators won't work, as -/%& are numerical operators, ,;: are also argo operators).

I think this problem is related to the more general problem of having typed parameters in Kubeflow but for the moment, quoting predicate operators should be a good fix!

magdalenakuhn17 pushed a commit to magdalenakuhn17/pipelines that referenced this issue Oct 22, 2023
Signed-off-by: Kim Caffeinism <make.dirty.code@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/sdk/dsl/compiler help wanted The community is welcome to contribute. kind/bug
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants