Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

dependency analysis issue with 'component.yaml'-based ops #941

Closed
amygdala opened this issue Mar 8, 2019 · 2 comments · Fixed by #943
Closed

dependency analysis issue with 'component.yaml'-based ops #941

amygdala opened this issue Mar 8, 2019 · 2 comments · Fixed by #943

Comments

@amygdala
Copy link
Contributor

amygdala commented Mar 8, 2019

I built this pipeline to play around with the new load_component_from_url functionality: https://gist.github.com/amygdala/f6ac2224c598e8355763a45ab502cb49
(The pipeline doesn't quite have correct functionality yet, but that's not important to this issue).
Note line 72, using the op2.after(op1) statement.

When I compile this file, I get the following error. To me it looks like the dependency analysis is trying to use the 'human name' for the component rather than its 'name'.
I'm using the 0.1.12 sdk. (If I comment out line 72, it compiles properly).

Traceback (most recent call last):
  File "cmle_train_deploy.py", line 77, in <module>
    compiler.Compiler().compile(cmle_pipeline, __file__ + '.tar.gz')
  File "/Users/amyu/miniconda3/envs/mlpipeline2/lib/python3.6/site-packages/kfp/compiler/compiler.py", line 572, in compile
    workflow = self._compile(pipeline_func)
  File "/Users/amyu/miniconda3/envs/mlpipeline2/lib/python3.6/site-packages/kfp/compiler/compiler.py", line 562, in _compile
    workflow = self._create_pipeline_workflow(args_list_with_defaults, p)
  File "/Users/amyu/miniconda3/envs/mlpipeline2/lib/python3.6/site-packages/kfp/compiler/compiler.py", line 454, in _create_pipeline_workflow
    templates = self._create_templates(pipeline)
  File "/Users/amyu/miniconda3/envs/mlpipeline2/lib/python3.6/site-packages/kfp/compiler/compiler.py", line 416, in _create_templates
    dependencies = self._get_dependencies(pipeline, new_root_group, op_groups)
  File "/Users/amyu/miniconda3/envs/mlpipeline2/lib/python3.6/site-packages/kfp/compiler/compiler.py", line 190, in _get_dependencies
    upstream_op = pipeline.ops[op_name]
KeyError: 'Train a model with Cloud ML Engine'
@Ark-kun
Copy link
Contributor

Ark-kun commented Mar 8, 2019

Thank you for discovering this DSL compiler bug. I've submitted PR fixing it.

@hongye-sun
Copy link
Contributor

Thanks for the quick fix, Alexey.

Linchin pushed a commit to Linchin/pipelines that referenced this issue Apr 11, 2023
Fix postsubmit PR link
HumairAK pushed a commit to red-hat-data-services/data-science-pipelines that referenced this issue Mar 11, 2024
…eflow#941)

* Loop ranges should handle start = 0 and end  = 0, should generate one iteration.

* When from = to, then generate one iteration. emulating how enumerate works.

* Fixed edge cases.

* if from and to are unset, we should not pick defaults either.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants