Skip to content

TorchServe IPEX Blog #2 #2079

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 45 commits into from
Oct 14, 2022
Merged
Changes from 1 commit
Commits
Show all changes
45 commits
Select commit Hold shift + click to select a range
b5488b9
Create torchserve_with_ipex_2.rst
min-jean-cho Oct 12, 2022
500dc2b
create torchserve-ipex-images-2
min-jean-cho Oct 12, 2022
51e405b
add torchserve-ipex-images-2 png
min-jean-cho Oct 12, 2022
5508e1b
add png
min-jean-cho Oct 12, 2022
b359eb7
update wording
min-jean-cho Oct 12, 2022
5df5bed
update wording
min-jean-cho Oct 12, 2022
9124a50
update wording
min-jean-cho Oct 12, 2022
e76c82e
update wording
min-jean-cho Oct 12, 2022
de70676
add tutorial to matrix and left nav
min-jean-cho Oct 12, 2022
a0b6350
Delete placeholder
min-jean-cho Oct 12, 2022
d06933a
link github
min-jean-cho Oct 12, 2022
d905bba
tutorial -> blog
min-jean-cho Oct 12, 2022
bece54b
tutorial -> blog
min-jean-cho Oct 12, 2022
8ce095a
grammar fix
min-jean-cho Oct 12, 2022
a04fa71
grammar fix
min-jean-cho Oct 12, 2022
c71fee4
grammar fix
min-jean-cho Oct 12, 2022
fe7dbf1
blog -> tutorial
min-jean-cho Oct 12, 2022
32cf25d
un-tuned -> untuned, submetircs -> sub-metrics
min-jean-cho Oct 12, 2022
07a6e8d
blog -> tutorial, we'll -> we will
min-jean-cho Oct 12, 2022
bfe11ae
with torch.autograd.profiler.emit_itt()
min-jean-cho Oct 12, 2022
aaff8cb
grammar fix
min-jean-cho Oct 12, 2022
d81e08f
grammar fix
min-jean-cho Oct 12, 2022
4e6ba4b
we'll -> we will
min-jean-cho Oct 12, 2022
8490135
we'll -> we will
min-jean-cho Oct 12, 2022
8824027
we'll -> we will
min-jean-cho Oct 12, 2022
697dfd1
we've -> we have
min-jean-cho Oct 12, 2022
bdd965c
we'll -> we will
min-jean-cho Oct 12, 2022
bca2544
we'll -> we will
min-jean-cho Oct 12, 2022
15717d8
2 -> two
min-jean-cho Oct 12, 2022
7c33548
we'll -> we will
min-jean-cho Oct 12, 2022
c936e12
e.g., -> for example, refer to -> see
min-jean-cho Oct 12, 2022
a276f35
etc -> and more
min-jean-cho Oct 12, 2022
129e933
we'll -> we will
min-jean-cho Oct 12, 2022
0661896
we'll -> we will
min-jean-cho Oct 12, 2022
22858fa
we'll we will
min-jean-cho Oct 12, 2022
0abdb79
un-tuned -> untuned
min-jean-cho Oct 12, 2022
ee1e3d7
take-aways -> conclusion
min-jean-cho Oct 12, 2022
bcc847d
blog -> tutorial, we've -> we have
min-jean-cho Oct 12, 2022
cb5a935
fix linking
min-jean-cho Oct 12, 2022
ddaa4fe
fix png sizes
min-jean-cho Oct 12, 2022
df74a9f
my lin <url>__
min-jean-cho Oct 13, 2022
15c6b3f
(1) add content under each heading (2) fix heading syntax
min-jean-cho Oct 14, 2022
4e189ba
update
min-jean-cho Oct 14, 2022
a1f45ae
Merge branch 'master' into minjean/torchserve_with_ipex_2
min-jean-cho Oct 14, 2022
786417e
blog -> tutorial
min-jean-cho Oct 14, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
my lin <url>__
  • Loading branch information
min-jean-cho authored Oct 13, 2022
commit df74a9fb5629681e86f820185900b337b5e11366
6 changes: 3 additions & 3 deletions intermediate_source/torchserve_with_ipex_2.rst
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ In this tutorial, we will demonstrate boosting performance with memory allocator
:width: 100%
:align: center

Throughout this tutorial, we will use `Top-down Microarchitecture Analysis (TMA) <https://www.intel.com/content/www/us/en/develop/documentation/vtune-cookbook/top/methodologies/top-down-microarchitecture-analysis-method.html>`_ to profile and show that the Back End Bound (Memory Bound, Core Bound) is often the primary bottleneck for under-optimized or under-tuned deep learning workloads, and we'll demonstrate optimization techniques via Intel® Extension for PyTorch* for improving Back End Bound. We will also use `Intel® VTune™ Profiler's Instrumentation and Tracing Technology (ITT) <https://github.com/pytorch/pytorch/issues/41001>`_ to profile at finer granularity.
Throughout this tutorial, we will use `Top-down Microarchitecture Analysis (TMA) <https://www.intel.com/content/www/us/en/develop/documentation/vtune-cookbook/top/methodologies/top-down-microarchitecture-analysis-method.html>`_ to profile and show that the Back End Bound (Memory Bound, Core Bound) is often the primary bottleneck for under-optimized or under-tuned deep learning workloads, and we'll demonstrate optimization techniques via Intel® Extension for PyTorch* for improving Back End Bound. We will also use `Intel® VTune™ Profiler's Instrumentation and Tracing Technology (ITT) <https://github.com/pytorch/pytorch/issues/41001>`__ to profile at finer granularity.

*****************
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We typically don't add a table of contents like this because it's autogenerated on the right hand side under Shortcuts. It is difficult to keep a manual TOC like this up-to-date with any future changes in the content.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm fine with either options, but I thought TOC would help when readers read the intro since this tutorial has lots of content. Let me know what you'd prefer !

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I suggest to remove it as we don't have it in any other tutorials.

Table of Contents
Expand Down Expand Up @@ -145,7 +145,7 @@ Let's profile PTMalloc vs. JeMalloc with TorchServe.

We will use `TorchServe apache-bench benchmarking <https://github.com/pytorch/serve/tree/master/benchmarks#benchmarking-with-apache-bench>`_ with ResNet50 FP32, batch size 32, concurrency 32, requests 8960. All other parameters are the same as the `default parameters <https://github.com/pytorch/serve/tree/master/benchmarks#benchmark-parameters>`_.

As in the previous exercise, we will use the launcher to designate the memory allocator, and to bind the workload to physical cores of the first socket. To do so, user simply needs to add a few lines in `config.properties <https://pytorch.org/serve/configuration.html#config-properties-file>`_:
As in the previous exercise, we will use the launcher to designate the memory allocator, and to bind the workload to physical cores of the first socket. To do so, user simply needs to add a few lines in `config.properties <https://pytorch.org/serve/configuration.html#config-properties-file>`__:

PTMalloc

Expand Down Expand Up @@ -397,7 +397,7 @@ Let's profile Intel® Extension for PyTorch* optimizations with TorchServe.

We will use `TorchServe apache-bench benchmarking <https://github.com/pytorch/serve/tree/master/benchmarks#benchmarking-with-apache-bench>`_ with ResNet50 FP32 TorchScript, batch size 32, concurrency 32, requests 8960. All other parameters are the same as the `default parameters <https://github.com/pytorch/serve/tree/master/benchmarks#benchmark-parameters>`_.

As in the previous exercise, we will use the launcher to bind the workload to physical cores of the first socket. To do so, user simply needs to add a few lines in `config.properties <https://github.com/pytorch/serve/tree/master/benchmarks#benchmark-parameters>`_:
As in the previous exercise, we will use the launcher to bind the workload to physical cores of the first socket. To do so, user simply needs to add a few lines in `config.properties <https://github.com/pytorch/serve/tree/master/benchmarks#benchmark-parameters>`__:

.. code:: python

Expand Down