Render static graph when compiling a pipeline in a notebook

[amygdala]

Feature request: When compiling a pipeline in a notebook, render the static graph as output, same as the graph shown when you upload a pipeline.

[gaoning777]

Why is the compiler generating a static graph important when we already have the pipeline UI supporting it? Would it be a common scenario that users author the pipeline without a pipeline deployment(we do support on-prem minikube) such that the DSL generating the static graph is beneficial? Thanks

[yebrahim]

I can see that providing a nice feedback loop to iterate visually on the pipeline. You make a change, compile, see the output graph, then iterate. It's easier to debug high level issues just by looking at the graph than dozens of Python lines.

It won't be easy to support though, since the graph is built on the fly by our UI layer. It does make one more argument for enabling editing the pipeline source code in our UI though?
@Ark-kun @qimingj

[Ark-kun]

Yes, visual pipeline editing would be great.

As for the graph, I could generate it, but I'm, a bit hesitant to add more dependencies (to graphing packages) to the SDK.

[Ark-kun]

Another solution would be to add IFrame to Jupyter (easy - I've checked) that shows the run page (without navigation). See #182

[gaoning777]

@amygdala is this still a wanted feature?

[gaoning777]

/unassign @gaoning777
/assign @numerology @Ark-kun

[Bobgy]

This sounds useful. @Ark-kun Can you provide me an example python notebook that compiles a pipeline, uploads it and shows the iframe of pipeline detail page, so I can try to iterate UI on it.

[Bobgy]

It seems keeping uploading versions to the same pipeline and maybe auto refresh the pipeline detail page showing latest result is a good enough UX.

[Bobgy]

WDUT?

[amygdala]

Yeah, I think we can consider it lower-priority. (though if we get some breathing room it would be a 'nice to have').

[rmgogogo]

Middle/long term plan: this one can be handled together with IR topic. when we have an IR, it can help decouple the runner with visualizer.

In short term, we may bring the argo based render to jupyter world but I think it has lower priority as it will be replaced by IR based sooner or later. SG?

[Bobgy]

@rmgogogo I think @Ark-kun's main proposal is that putting something to jupyter is minimal cost, jupyter just takes a url to show something. If we have a page that only shows the content, it will work by itself.

So it doesn't really have any conflict with IR.

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[Bobgy]

/lifecycle frozen

[rimolive]

Closing this issue. No activity for more than a year.

/close

[google-oss-prow]

@rimolive: Closing this issue.

In response to this:

Closing this issue. No activity for more than a year.

/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.