…ner (#2052)

I believe, new code snippet annotation have more sense.

---------

Co-authored-by: G. Ramalingam <grama@microsoft.com>
Co-authored-by: Ti-Tai Wang <titaiwang@microsoft.com>

Co-authored-by: Ti-Tai Wang <titaiwang@microsoft.com>

The function does not seem to work inplace in all cases.

---------

Co-authored-by: Ti-Tai Wang <titaiwang@microsoft.com>
Co-authored-by: Justin Chu <justinchuby@users.noreply.github.com>

Fix issues reported in
#2050 (comment)

Enable extraction of rewritten subgraph as model-local function.

This will enable multi-step rewrite optimizations: eg., map subgraph G1
to new-op1, and then map subgraph G2 containing new-op1 to new-op2, and
then inlining can replace any remaining new-op1 (that was not rewritten)
by original G1.

---------

Co-authored-by: Justin Chu <justinchuby@users.noreply.github.com>

Fix #1998 

Basically, just follow the logic from symbolic_opset9.py:
https://github.com/pytorch/pytorch/blob/fdb1305ace9cd875611931983eada640ab837c4c/torch/onnx/symbolic_opset9.py#L2878.

The implementation from symbolic_opset12.py claims it fixes static
shapes issue, but I see no difference because dimension, size, step are
all int. I also tested
https://github.com/pytorch/pytorch/blob/fdb1305ace9cd875611931983eada640ab837c4c/test/onnx/test_pytorch_onnx_onnxruntime.py#L7300
with opset9 and it still works. So we don't need to capture the loop in
dynamic I think.

Trace ops discovered in pytorch/pytorch#147617
and simplify `repeat` implementation.

Register it to aten_pow instead because exponent may not be a tensor.

Fixes pytorch/pytorch#147606

The op was failing and not traced.

…250221 in /requirements/ci (#2072)

Co-authored-by: Justin Chu <justinchuby@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

#2076)

Add a couple of variants of patterns in ORT fusions (motivated by Phi4)

…#2078)

The float8_e4m3fnuz type was mistaken with float8_e4m3b11fnuz, which is
a different type: https://github.com/jax-ml/ml_dtypes#float8_e4m3b11fnuz

A recent fix to the translation of pytorch symints introduces a
Squeeze=>Reshape pattern that can be optimized away. This PR introduces
a rewrite-rule to do this optimization.

TODO (in a separate PR): for now, this optimization needs to be
explicitly invoked. This should be done by default. (But there are
several other such optimizations that need to be collected and included
in the default-rule list.)

Fixes #2084

This is required to land pytorch/pytorch#148140
in torch.export().


cc @angelayi @@justinchuby

Co-authored-by: Shangdi Yu <shanhdiy@meta.com>

* Extends the cos-sin-cache fusion to support 1D position-id (without
batch dimension)
* Make MatchingTracer a parameter of the rewriter to give users better
control over how to report stats (for successful or failing matches)
* Improve the tracer output

#2086)