Investigate failure of `ElementwiseReluModule` and `ResNet18StaticModule` TOSA tests

[ashay]

Both ElementwiseReluModule and ResNet18StaticModule TOSA tests fail after updating LLVM. See CI log for details.

    FAIL - "ElementwiseReluModule_basic"
        @ trace item #0 - call to "forward"
        @ output of call to "forward"
        ERROR: value (Tensor with shape=[4, 2], dtype=torch.float32, min=+3.403e+38, max=+3.403e+38, mean=+3.403e+38) is not close to golden value (Tensor with shape=[4, 2], dtype=torch.float32, min=+0.0, max=+0.3964, mean=+0.09984)

    FAIL - "ResNet18StaticModule_basic"
        @ trace item #0 - call to "forward"
        @ output of call to "forward"
        ERROR: value (Tensor with shape=[1, 1000], dtype=torch.float32, min=+nan, max=+nan, mean=+nan) is not close to golden value (Tensor with shape=[1, 1000], dtype=torch.float32, min=-1.815, max=+2.328, mean=+0.03187)

[sjarus]

Without looking at anything in detail, this sounds like something might have changed in TosaToLinAlg/LinAlgNamed/Arith that generates functionally wrong output, since the TOSA content is unchanged here.

[silvasean]

@sjarus do you have someone on your side that can investigate this?

[sjarus]

Adding @eric-k256 and @u99127, as well as @jpienaar who has been reviewing recent TosaToXXX PRs . I think this might just be a git bisect away from catching which TosaToXX change broke the functional correctness of these models.

I'm traveling over the next 24hrs so I don't have the ability to look right now unfortunately.

[jpienaar]

So this failed after the last LLVM update? Could you add the before and after rev here? (ReluN was removed 4 days ago upstream, and llvm/llvm-project@444b4fd seems likely candidate fix given replacement)

[eric-k256]

I can confirm that llvm/llvm-project@444b4fd fixes the problem.
I'm not sure what you do in torch-mlir. Does this wait until the next LLVM update, or should we try to move LLVM forward again?

[silvasean]

This seems reasonable to wait until the next update -- the XFAILs will pass and we can update them and close this bug.

[eric-k256]

Sounds good to me.

[ashay]

Thanks all for diagnosing this so quickly! Looking forward to next Monday when I can close this issue without doing any work ;)

[sjarus]

Thank you for your assistance, @jpienaar and @eric-k256 !

[ashay]

Resolved in LLVM tag update PR #1263.