End to end mnist pipeline use case

[hougangliu]

A mnist pipeline including katib, tfjob and kfserving

---

This change is 

[karlschriek]

Hey @hougangliu, thanks very much for this example, very useful!

Could I give a some input from a target audience perspective?

The first thing anyone looking at this example will do is to ask "how could I do this in my use case?". Which will come down to "how to I replace liuhougangxa/tf-estimator-mnist with my own image? What is it supposed to look like? How do I set it up?". It would be really useful if you could include the Dockerfile and the model.py file so that someone could use this as a template for setting up their own.

Might I also suggest moving the follwoing outside of the def mnist_pipeline(..) function scope?

• input dictionaries
• the def convert_mnist_experiment_result(...) function
• ops definitions convert_op = func_to_container_op(convert_mnist_experiment_result) and katib_experiment_launcher_op = components.load_component_from_url(...)

That would make the example much cleaner to read.

[hougangliu]

@karlschriek thanks for your comment.
This sample mainly aims to show how to integrate katib, Tfjob and inferenceservice in pipeline, as for better understanding this sample or even modify it for your use case, I think user need to know katib, tfjob and inferenceservice more. Then it is easy to implement your case just following this pipeline way

[Ark-kun]

This image is pretty big. Would you mind removing it from the notebook to keep the source repo smaller?

[hougangliu]

I can do it, but the picture can show how user can get inferenceservice name well. it can help user to run this example successfully.

[Ark-kun]

The notebook size seems to be quite big.
Other than that,
/lgtm

[karlschriek]

@karlschriek thanks for your comment.
This sample mainly aims to show how to integrate katib, Tfjob and inferenceservice in pipeline, as for better understanding this sample or even modify it for your use case, I think user need to know katib, tfjob and inferenceservice more. Then it is easy to implement your case just following this pipeline way

Thanks for the response.

Perhaps my point could be made for the Katib examples in general (and so probably should go on the Katib board), but one of the main difficulties I have with the examples is that they typically just call a pre-built Docker image from somewhere... but it is quite important to understand how that image was built in order to understand how the metric collector picks up its inputs! (In most cases I ended up running the Docker container and then manually copying out the model.py file in order to scrutinize what was happening. I did the same with this example)

[hougangliu]

Thanks for the response.

Perhaps my point could be made for the Katib examples in general (and so probably should go on the Katib board), but one of the main difficulties I have with the examples is that they typically just call a pre-built Docker image from somewhere... but it is quite important to understand how that image was built in order to understand how the metric collector picks up its inputs! (In most cases I ended up running the Docker container and then manually copying out the model.py file in order to scrutinize what was happening. I did the same with this example)

Yes, user need understand how katib metrics collector works and how metrics should print.
For now, in 0.7 release, katib in fact only can read two kinds of metric format:

1. tensorflow event file;
2. stdout or log file, but in each line, only one metrics can exist with format "$metricsName=$value"
   Anyway, I think here we should move on katib repo instead of here :)

[Ark-kun]

/approve

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: Ark-kun

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• samples/OWNERS [Ark-kun]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[hougangliu]

/retest

[hougangliu]

/test kubeflow-pipeline-e2e-test

[hougangliu]

/test kubeflow-pipeline-sample-test

[pretidav]

@hougangliu I am trying to use yout e2e mnist tutorial with both katib and tfjob launchers. However, the latter get stuck all the time and never meets "success condition" before going after the time limit (even using a huge limit).
Could you please suggest me something to do? Probably I am doing something wrong.