Skip to content

Commit

Permalink
bug fix (#3252)
Browse files Browse the repository at this point in the history
  • Loading branch information
jingxu10 committed Sep 11, 2024
1 parent 0ae4560 commit 3703754
Showing 1 changed file with 3 additions and 3 deletions.
6 changes: 3 additions & 3 deletions xpu/2.3.110+xpu/tutorials/api_doc.html
Original file line number Diff line number Diff line change
Expand Up @@ -338,15 +338,15 @@ <h2>General<a class="headerlink" href="#general" title="Link to this heading">
</dl>
<div class="admonition warning">
<p class="admonition-title">Warning</p>
<p>Please invoke <code class="docutils literal notranslate"><span class="pre">optimize_transformers</span></code> function AFTER invoking DeepSpeed in Tensor Parallel
<p>Please invoke <code class="docutils literal notranslate"><span class="pre">ipex.llm.optimize</span></code> function AFTER invoking DeepSpeed in Tensor Parallel
inference scenario.</p>
</div>
<p class="rubric">Examples</p>
<div class="doctest highlight-default notranslate"><div class="highlight"><pre><span></span><span class="gp">&gt;&gt;&gt; </span><span class="c1"># bfloat16 generation inference case.</span>
<span class="gp">&gt;&gt;&gt; </span><span class="n">model</span> <span class="o">=</span> <span class="o">...</span>
<span class="gp">&gt;&gt;&gt; </span><span class="n">model</span><span class="o">.</span><span class="n">load_state_dict</span><span class="p">(</span><span class="n">torch</span><span class="o">.</span><span class="n">load</span><span class="p">(</span><span class="n">PATH</span><span class="p">))</span>
<span class="gp">&gt;&gt;&gt; </span><span class="n">model</span><span class="o">.</span><span class="n">eval</span><span class="p">()</span>
<span class="gp">&gt;&gt;&gt; </span><span class="n">optimized_model</span> <span class="o">=</span> <span class="n">ipex</span><span class="o">.</span><span class="n">optimize_transformers</span><span class="p">(</span><span class="n">model</span><span class="p">,</span> <span class="n">dtype</span><span class="o">=</span><span class="n">torch</span><span class="o">.</span><span class="n">bfloat16</span><span class="p">)</span>
<span class="gp">&gt;&gt;&gt; </span><span class="n">optimized_model</span> <span class="o">=</span> <span class="n">ipex</span><span class="o">.</span><span class="n">llm</span><span class="o">.</span><span class="n">optimize</span><span class="p">(</span><span class="n">model</span><span class="p">,</span> <span class="n">dtype</span><span class="o">=</span><span class="n">torch</span><span class="o">.</span><span class="n">bfloat16</span><span class="p">)</span>
<span class="gp">&gt;&gt;&gt; </span><span class="n">optimized_model</span><span class="o">.</span><span class="n">generate</span><span class="p">()</span>
</pre></div>
</div>
Expand Down Expand Up @@ -789,4 +789,4 @@ <h2>C++ API<a class="headerlink" href="#c-api" title="Link to this heading"><
</script>

</body>
</html>
</html>

0 comments on commit 3703754

Please sign in to comment.