Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Feedback] docs/components/pipelines/user-guides/components/containerized-python-components.md #3871

Closed
stephenleo opened this issue Sep 7, 2024 · 0 comments

Comments

@stephenleo
Copy link
Contributor

stephenleo commented Sep 7, 2024

How do I import add in pipeline.py while using containerized python components?

Here's my directory structure

pipeline.py
src/
  |--math_utils.py
  |--my_component.py

Using code exactly from the documentation, runs into this error:

$ python -m pipeline
Traceback (most recent call last):
  File "/opt/conda/lib/python3.10/runpy.py", line 196, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/opt/conda/lib/python3.10/runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "/home/jupyter/mlops-on-gcp/test/pipeline.py", line 4, in <module>
    def addition_pipeline(x: int, y: int) -> int:
  File "/opt/conda/lib/python3.10/site-packages/kfp/dsl/pipeline_context.py", line 65, in pipeline
    return component_factory.create_graph_component_from_func(
  File "/opt/conda/lib/python3.10/site-packages/kfp/dsl/component_factory.py", line 673, in create_graph_component_from_func
    return graph_component.GraphComponent(
  File "/opt/conda/lib/python3.10/site-packages/kfp/dsl/graph_component.py", line 58, in __init__
    pipeline_outputs = pipeline_func(*args_list)
  File "/home/jupyter/mlops-on-gcp/test/pipeline.py", line 5, in addition_pipeline
    task1 = add(a=x, b=y)
NameError: name 'add' is not defined

Updating pipeline.py to the below code

from kfp import dsl, compiler
from src.my_component import add

@dsl.pipeline
def addition_pipeline(x: int, y: int) -> int:
    task1 = add(a=x, b=y)
    task2 = add(a=task1.output, b=x)
    return task2.output

compiler.Compiler().compile(addition_pipeline, 'pipeline.yaml')

runs into this error

$ python -m pipeline
Traceback (most recent call last):
  File "/opt/conda/lib/python3.10/runpy.py", line 196, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/opt/conda/lib/python3.10/runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "/home/jupyter/mlops-on-gcp/test/pipeline.py", line 2, in <module>
    from src.my_component import add
  File "/home/jupyter/mlops-on-gcp/test/src/my_component.py", line 2, in <module>
    from math_utils import add_numbers
ModuleNotFoundError: No module named 'math_utils'
stephenleo added a commit to stephenleo/website that referenced this issue Sep 14, 2024
Minor updates to make the documentation work

Signed-off-by: Marie Stephen Leo <35026793+stephenleo@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant