[RFC][Tracking Issue] Pipeline Executor For Compute graph pipeline

[huajsj]

PR #7892
Discussion https://discuss.tvm.apache.org/t/rfc-compute-graph-pipeline-with-new-subgraph-executor/9839/

Steps

Following are the PR list for the feature plan

[Pipeline Executor] Infrastructure

• Pipeline configuration creation [Runtime] Pipeline Executor Initial patch. #8702
• Pipeline backend build and pipeline configuration export/load [Runtime] Pipeline Executor Second patch, configuration load and executor export/import. #9108
• Pipeline executor set_input implementation and input configuration loading. [Runtime][Pipeline Executor] Add the map logic of global input and subgraph input. #9751
• Pipeline executor set_params implementation and params configuration loading. [Runtime][Pipeline executor] Global parameters group name and runtime modules parameters map. #9846
• Pipeline executor runtime interface declare and implementation.[Runtime][PipelineExecutor] Add Pipeline Executor Interface #10010
• Pipeline executor pipe line sequence execution. [Runtime][PipelineExecutor] Pipeline Executor Sequential execution #10082
• Pipeline executor, pipeline execution, multiple threads management and the event notification system .[Runtime][Pipeline Executor] multiple threads management and the data forwarding notification mechanism. #10234
• Pipeline executor, pipeline execution, data forwarding for internal binding data [Runtime][PipelineExecutor] Add the pipeline internal forwarding logic. #10543
• Pipeline executor, pipeline execution, runtime CPU affinity configuration and applying.[Runtime][PipelineExecutor] Setting CPU affinity for the Runtime of pipeline. #10639
• Pipeline executor, pipeline execution, runtime asynchronous run, and getting output.[Runtime][PipelineExecutor] Getting the asynchronous output #10723
• Pipeline executor, pipeline execution, data forwarding for "on the fly" input data.[Runtime][PipelineExecutor]Add forwarding queue logic for set input. #10990
• Refactor PipelineExecutor.py and Add cross compile support for pipeline executor [Runtime][PipelineExecutor] Refactor PipelineExecutor.py and Add cross compile support for pipeline executor. #11133
• Add network split logic into unit test [Runtime][PipelineExecutor] Add graph manually splitting logic into the unit test. #11334
• Pipeline executor tutorial.[Runtime][PipelineExecutor] Tutorial of using pipeline executor. #11557

[Enable CPU Affinity]

• Binding the control flow/data flow thread of backend runtime on a/a group of CPU.[Runtime][ThreadPool]Refactor affinity function and support CPU affinity list setting. #9802

PR is not in this solution scope but related, separate issue used to track.

Compute graph splitting function	PR	Tracking Issue
Splitting compute graph into a list of subgraph in C++.

Compute graph automatic splitting	PR	Tracking Issue
compute graph automatic splitting infrastructure
compute graph auto splitting interface and example
compute graph auto tune and graph splitting algorithm implementation

non-DNN model function pipeline integration support	PR	Tracking Issue
Add darknet operator for yolov3 data preparation(pre-processing )
Add darknet operator for yolov3 detection result analysis(post processing)

[comaniac]

@huajsj please update this issue with a PR checklist. Please link all future PRs to this issue for better progress tracking.
You can refer to other RFC issues like #8473

[huajsj]

@comaniac , thanks for the follow up, just saw this comments, already done.

[comaniac]

@comaniac , thanks for the follow up, just saw this comments, already done.

Could you update the issue with a complete plan (e.g., milestones, expected PRs) instead of adding one line when filing a PR? Otherwise people won't have an idea about how many PRs you are going to send and what milestone you have achieved.

[huajsj]

@comaniac , sure, already add the future PR plan.

[huajsj]

@comaniac, I just updated this issue to move these planned PR not belong to this solution to sperate tables to avoid confuse, please check.

[rebel-shshin]

@huajsj Thanks for the nice job.

[masahi]

At one point can we expect a non-trivial demo?

[huajsj]

thanks @masahi for the follow up, I am also looking forward a chance to do a demo after all patch upstream done.

[rebel-shshin]

thanks @masahi for the follow up, I am also looking forward a chance to do a demo after all patch upstream done.

Hi @huajsj, do you have any plan to do a demo with compiling deep learning models such as [TF model to relay](https://tvm.apache.org/docs/how_to/compile_models/from_tensorflow.html#sphx-glr-how-to-compile-models-from-tensorflow-py](https://github.com/apache/tvm/issues/url) ?

And I think it would be helpful to the TVM users if you provide an example with BYOC also. Thank you very much :)

[huajsj]

thanks @masahi for the follow up, I am also looking forward a chance to do a demo after all patch upstream done.

Hi @huajsj, do you have any plan to do a demo with compiling deep learning models such as [TF model to relay](https://tvm.apache.org/docs/how_to/compile_models/from_tensorflow.html#sphx-glr-how-to-compile-models-from-tensorflow-py](https://tvm.apache.org/docs/how_to/compile_models/from_tensorflow.html#sphx-glr-how-to-compile-models-from-tensorflow-py%5D(https://github.com/apache/tvm/issues/url)) ?

And I think it would be helpful to the TVM users if you provide an example with BYOC also. Thank you very much :)

@rebel-shshin, thanks for the follow up, yes, a tutorial is a very good suggestion, actually we are working on the related document, I would add a tutorial task under this issue for tracking purpose.

[rebel-shshin]

thanks @masahi for the follow up, I am also looking forward a chance to do a demo after all patch upstream done.

Hi @huajsj, do you have any plan to do a demo with compiling deep learning models such as [TF model to relay](https://tvm.apache.org/docs/how_to/compile_models/from_tensorflow.html#sphx-glr-how-to-compile-models-from-tensorflow-py](https://tvm.apache.org/docs/how_to/compile_models/from_tensorflow.html#sphx-glr-how-to-compile-models-from-tensorflow-py%5D(https://github.com/apache/tvm/issues/url)) ?
And I think it would be helpful to the TVM users if you provide an example with BYOC also. Thank you very much :)

@rebel-shshin, thanks for the follow up, yes, a tutorial is a very good suggestion, actually we are working on the related document, I would add a tutorial task under this issue for tracking purpose.

Thank you very much :)

[huajsj]

thanks @masahi for the follow up, I am also looking forward a chance to do a demo after all patch upstream done.

Hi @huajsj, do you have any plan to do a demo with compiling deep learning models such as [TF model to relay](https://tvm.apache.org/docs/how_to/compile_models/from_tensorflow.html#sphx-glr-how-to-compile-models-from-tensorflow-py](https://tvm.apache.org/docs/how_to/compile_models/from_tensorflow.html#sphx-glr-how-to-compile-models-from-tensorflow-py%5D(https://github.com/apache/tvm/issues/url)) ?
And I think it would be helpful to the TVM users if you provide an example with BYOC also. Thank you very much :)

@rebel-shshin, thanks for the follow up, yes, a tutorial is a very good suggestion, actually we are working on the related document, I would add a tutorial task under this issue for tracking purpose.

Thank you very much :)

Hi @rebel-shshin , tutorial already there, #11557.

[huajsj]

all the task done, thanks @comaniac and @masahi for the kindly review, thanks @rebel-shshin for the follow up, close this issue now.

[rebel-shshin]

thanks @masahi for the follow up, I am also looking forward a chance to do a demo after all patch upstream done.

Hi @huajsj, do you have any plan to do a demo with compiling deep learning models such as [TF model to relay](https://tvm.apache.org/docs/how_to/compile_models/from_tensorflow.html#sphx-glr-how-to-compile-models-from-tensorflow-py](https://tvm.apache.org/docs/how_to/compile_models/from_tensorflow.html#sphx-glr-how-to-compile-models-from-tensorflow-py%5D(https://github.com/apache/tvm/issues/url)) ?
And I think it would be helpful to the TVM users if you provide an example with BYOC also. Thank you very much :)

@rebel-shshin, thanks for the follow up, yes, a tutorial is a very good suggestion, actually we are working on the related document, I would add a tutorial task under this issue for tracking purpose.

Thank you very much :)

Hi @rebel-shshin , tutorial already there, #11557.

Thank you very much : )