KubeFlow Pipeline example notebook is half way updated for 0.1.4

[post2web]

The notebook: KubeFlow Pipeline Using TFX OSS Components.ipynb is updated to have .apply(gcp.use_gcp_secret('user-gcp-sa')) to every container but all other content is written for 0.1.3-rc.2 release.

To me, this notebook is the most useful reference for the pipeline creation and execution process and it is currently broken.

Simply updating the pip3 install kfp to 0.1.4 do not fix the issues with that notebook.

[gaoning777]

What is the error? Does it happen during compile time or runtime?
Do you mind posting the exact errors? Thanks

[post2web]

The first error would come from:

import kfp.gcp as gcp

ModuleNotFoundError: No module named 'kfp.gcp'

because KFP_PACKAGE = 'https://storage.googleapis.com/ml-pipeline/release/0.1.3-rc.2/kfp.tar.gz'

There will be more errors on every .apply(gcp.use_gcp_secret('user-gcp-sa'))

After changing KFP_PACKAGE to use 0.1.4, the pipeline compiles. The run starts when I deploy it to a cluster with pipelines 0.1.4 but get authentification errors from within the containers. This is probably because I haven't set anything for the gcp.use_gcp_secret. A hint on how to set this up will be much appreciated.

The pipeline compiles and runs as expected when I remove all .apply(gcp.use_gcp_secret('user-gcp-sa')) references from the notebook and run it on 0.1.3-rc.2 piplenes cluster.

[hongye-sun]

Have you tried restarting the kernal?

…

On Tue, Dec 11, 2018 at 9:52 PM post2web ***@***.***> wrote: The first error would come from: import kfp.gcp as gcp ModuleNotFoundError: No module named 'kfp.gcp' because KFP_PACKAGE = ' https://storage.googleapis.com/ml-pipeline/release/0.1.3-rc.2/kfp.tar.gz' There will be more errors on every .apply(gcp.use_gcp_secret('user-gcp-sa')) After changing KFP_PACKAGE to use 0.1.4, the pipeline compiles. The run starts when I deploy it to a cluster with pipelines 0.1.4 but get authentification errors from within the containers. This is probably because I haven't set anything for the gcp.use_gcp_secret. A hint on how to set this up will be much appreciated. The pipeline compiles and runs as expected when I remove all .apply(gcp.use_gcp_secret('user-gcp-sa')) references from the notebook and run it on 0.1.3-rc.2 piplenes cluster. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#520 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ApvF90ZXXoPjiVGfARoGL8JrhJZdhT4lks5u4JmNgaJpZM4ZOllL> .

[post2web]

Yes, the above are errors raised after restarting of the kernel. To test:

KFP_PACKAGE = 'https://storage.googleapis.com/ml-pipeline/release/0.1.3-rc.2/kfp.tar.gz'
!pip3 install $KFP_PACKAGE --upgrade
import kfp.gcp as gcp

will raise ModuleNotFoundError: No module named 'kfp.gcp'

[hongye-sun]

Sorry. I meant to be 0.1.4 kfp package. It should have the gcp package. I will update the sample to use that package tomorrow.

…

On Tue, Dec 11, 2018 at 11:59 PM post2web ***@***.***> wrote: Yes, the above are errors raised after restarting of the kernel. To test: KFP_PACKAGE = 'https://storage.googleapis.com/ml-pipeline/release/0.1.3-rc.2/kfp.tar.gz' !pip3 install $KFP_PACKAGE --upgrade import kfp.gcp as gcp will raise ModuleNotFoundError: No module named 'kfp.gcp' — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#520 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ApvF90XgSpJePnA_Fd4DPi3zvm12TQtGks5u4LdigaJpZM4ZOllL> .

[Nick-Harvey]

For anyone who stumbles upon this between now and the sample getting updated, simply replace KFP_PACKAGE = 'https://storage.googleapis.com/ml-pipeline/release/0.1.3-rc.2/kfp.tar.gz' to KFP_PACKAGE = 'https://storage.googleapis.com/ml-pipeline/release/0.1.4/kfp.tar.gz'