Extend Elyra tutorial with Neural Magic version.

[pacospace]

Is your feature request related to a problem? Please describe.
Extend Elyra tutorial with Neural Magic [1] version.

High-level Goals
Elyra tutorial has one version that focuses on the use of Neural Magic.

Describe the solution you'd like
https://docs.google.com/document/d/1GNICmV_EcI-PPygxvzTupBCIt9HG9MKiEUeVy6YVO2c/edit#heading=h.fbxyxnppymly

Describe alternatives you've considered
n/a

Additional context
This is a key result we would like to achieve in 21Q3

Related-To: AICoE/aicoe-ci#120
Related-To: AICoE/aicoe-ci#118

Tasks

• Extend docs for Neural Magic
• Create new notebooks
• Add new dataset
• Add new models
• Add Elyra pipeline for Neural Magic.

Acceptance Criteria

• demo extension can be run in Operate First
• STRETCH GOAL: feedback pipeline is able to show results from two deployments at least (Tf+Flask+gunicorn, Pytorch+Flask+gunicorn, neuralmagic+Flask+gunicorn): https://devconfus2021.sched.com/event/lkgM/feedback-about-deployment-of-an-intelligent-app

References

• [1] https://neuralmagic.com/

cc @goern @riekrh @TreeinRandomForest

[goern]

sounds good to me, thanks for writing this up. please take it to the next Tech Talk session to circulate it with the team

[pacospace]

Depends-On: neuralmagic/sparseml#386

[pacospace]

Depends-On: neuralmagic/sparseml#387 (onnx/tensorflow-onnx#1715 (tensorflow/tensorflow#52013))

[pacospace]

Related-To: neuralmagic/sparsify#83

[pacospace]

I was able to create an example for MNIST Classification using Pytorch + Neural Magic Tools. The output is a MNIST pruned ONNX model, optimized with Neural Magic Sparsify Recipe and optimizer.

The next step will be to deploy that ONNX model with NeuralMagic deepsparse inference engine and maybe compare results with a non-optimized model.

This example can be repeated once we get GPU and we can use some bigger model: ResNet or VGG.

[pacospace]

Nice-to-Have: neuralmagic/sparsify#84

[pacospace]

I was able to deploy locally deepsparse with MNIST example and got some very nice performance results. I will move to the gather metrics pipeline extension now for cluster deployment.

[pacospace]

Deployment in the cluster with deepsparse is currently failing because there are no node with flags avx2 or avx512. So I'm blocked on that.

[pacospace]

demo is also blocked until we get KFP in smaug: https://github.com/operate-first/SRE/issues/382

[pacospace]

Depends-On: operate-first/support#409
Depends-On: operate-first/support#408

[pacospace]

Deployment in the cluster with deepsparse is currently failing because there are no node with flags avx2 or avx512. So I'm blocked on that.

Neural magic deployment cannot be run on smaug cluster at the moment, but in rick cluster worked: neuralmagic/deepsparse#186

I added a note for users:

NOTE: Deepsparse deployment at the moments work only with CPU that supports avx2, avx512 flags (even better if avx512_vnni with VNNI is available: neuralmagic/deepsparse#186). Please check your environment running cat /proc/cpuinfo to identify flags and verify if your machine supports the deployment. In the deployment manifests in nm-inference overlay you need to set the env variable NM_ARCH=avx2|avx512.

[pacospace]

Related-To: thoth-station/prescriptions#18516
Related-To: thoth-station/prescriptions#18515

[pacospace]

Demo is ready to be tested, so waiting for https://github.com/operate-first/SRE/issues/382, operate-first/support#435

[goern]

what is the status of this?

[pacospace]

what is the status of this?

See comment above :) we are still waiting for KFP in Operate FIrst

[sesheta]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

/lifecycle stale

[sesheta]

Stale issues rot after 30d of inactivity.
Mark the issue as fresh with /remove-lifecycle rotten.
Rotten issues close after an additional 30d of inactivity.

If this issue is safe to close now please do so with /close.

/lifecycle rotten

[sesheta]

Rotten issues close after 30d of inactivity.
Reopen the issue with /reopen.
Mark the issue as fresh with /remove-lifecycle rotten.

/close

[sesheta]

@sesheta: Closing this issue.

In response to this:

Rotten issues close after 30d of inactivity.
Reopen the issue with /reopen.
Mark the issue as fresh with /remove-lifecycle rotten.

/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.