From a412c85fd75c14d7f16d92938a4d2c686571b924 Mon Sep 17 00:00:00 2001 From: Sean Silva Date: Mon, 20 Mar 2023 10:41:21 -0700 Subject: [PATCH] [docs] Add changes to e2e testing to long-term roadmap --- docs/long_term_roadmap.md | 13 +++++++++++++ 1 file changed, 13 insertions(+) diff --git a/docs/long_term_roadmap.md b/docs/long_term_roadmap.md index 62c3b6f94171..d22194633578 100644 --- a/docs/long_term_roadmap.md +++ b/docs/long_term_roadmap.md @@ -246,3 +246,16 @@ for current LTC-based toolchains onto TorchDynamo. This migration will improve the end-user experience since TorchDynamo is more seamless, but it is a end-user-impacting migration nonetheless and we will want to phase it appropriately with the community. + +### End-to-end (E2E) testing + +Torch-MLIR currently maintains its own test suite with +[hundreds of end-to-end tests](https://github.com/llvm/torch-mlir/tree/main/python/torch_mlir_e2e_test/test_suite) +that verify the correctness and completeness of our op lowerings. +These tests are tedious to write, and also sometimes hit corners +of PyTorch's API that aren't usually reachable by user code. +PyTorch already has an [end-to-end op test suite](https://github.com/pytorch/pytorch/blob/ead51864622467acd6835b6da86a166c1a32aa55/torch/testing/_internal/common_methods_invocations.py#L1) +and we should just plug into it. Here is [an example](https://github.com/pytorch/pytorch/blob/ead51864622467acd6835b6da86a166c1a32aa55/test/test_proxy_tensor.py#L1573) of doing so. +Even better, it would be great if TorchDynamo/PyTorch 2.0 +directly provided a way to plug into this. +