Assertion fails when lowering a torch mlir to tosa

[MIONkb]

I got a vit model and I have already transform it to torch mlir file with print(mb.module) in python.
1st Problem is :
However, when I run "run_pipeline_with_repro_report( mb.module, "builtin.module(torch-backend-to-tosa-backend-pipeline)", "Lowering Torch Backend IR -> TOSA Backend IR")", it fails like this:

"python exception: Failure while executing pass pipeline:
error: unknown: failed to legalize operation 'torch.prim.NumToTensor.Scalar' that was explicitly marked illegal
note: unknown: see current operation: %3 = "torch.prim.NumToTensor.Scalar"(%2) : (!torch.int) -> !torch.tensor<[],si64>
error: "/home/tianyi/anaconda3/envs/torch-mlir/lib/python3.10/site-packages/torch/nn/modules/linear.py":114:0: aten.Linear called but weight rank not 2 or 3
note: "/home/tianyi/anaconda3/envs/torch-mlir/lib/python3.10/site-packages/torch/nn/modules/linear.py":114:0: see current operation: %3 = "torch.aten.linear"(%0, %2, %1) : (!torch.tensor<[4,4,192],f32>, !torch.tensor, !torch.tensor) -> !torch.tensor<[4,4,32],f32>
error: "/home/tianyi/anaconda3/envs/torch-mlir/lib/python3.10/site-packages/torch/nn/modules/linear.py":114:0: failed to legalize operation 'torch.aten.linear' that was explicitly marked illegal
note: "/home/tianyi/anaconda3/envs/torch-mlir/lib/python3.10/site-packages/torch/nn/modules/linear.py":114:0: see current operation: %3 = "torch.aten.linear"(%0, %2, %1) : (!torch.tensor<[4,4,192],f32>, !torch.tensor, !torch.tensor) -> !torch.tensor<[4,4,32],f32>
error: "/home/tianyi/anaconda3/envs/torch-mlir/lib/python3.10/site-packages/torch/nn/modules/linear.py":114:0: aten.Linear called but weight rank not 2 or 3
note: "/home/tianyi/anaconda3/envs/torch-mlir/lib/python3.10/site-packages/torch/nn/modules/linear.py":114:0: see current operation: %3 = "torch.aten.linear"(%0, %1, %2) : (!torch.tensor<[16,32],f32>, !torch.tensor, !torch.none) -> !torch.tensor<[16,576],f32>
error: "/home/tianyi/anaconda3/envs/torch-mlir/lib/python3.10/site-packages/torch/nn/modules/linear.py":114:0: failed to legalize operation 'torch.aten.linear' that was explicitly marked illegal
note: "/home/tianyi/anaconda3/envs/torch-mlir/lib/python3.10/site-packages/torch/nn/modules/linear.py":114:0: see current operation: %3 = "torch.aten.linear"(%0, %1, %2) : (!torch.tensor<[16,32],f32>, !torch.tensor, !torch.none) -> !torch.tensor<[16,576],f32>"

Then I use " torch-mlir-opt --convert-torch-to-tosa SimpleVIT.mlir" to lower this torch mlir file, and
2nd Problem comes:

"torch-mlir-opt: /home/tianyi/Torchmlir/torch-mlir-main/externals/llvm-project/llvm/include/llvm/Support/Casting.h:566: decltype(auto) llvm::cast(const From&) [with To = mlir::RankedTensorType; From = mlir::Type]: Assertion isa<To>(Val) && "cast<Ty>() argument of incompatible type!"' failed. PLEASE submit a bug report to https://github.com/llvm/llvm-project/issues/ and include the crash backtrace. torch-mlir-opt: /home/tianyi/Torchmlir/torch-mlir-main/externals/llvm-project/llvm/include/llvm/Support/Casting.h:566: decltype(auto) llvm::cast(const From&) [with To = mlir::RankedTensorType; From = mlir::Type]: Assertion isa(Val) && "cast() argument of incompatible type!"' failed.
Aborted (core dumped)"

The 1st problem occurs in python script and give more information, the 2nd problem occurs without too much information. So what is the reason for these 2 problems, and how can I debug in the 2nd problem?

[MIONkb]

MySimpleVIT.zip
This is the torch mlir for a simple VIT

[ramiro050]

The IR you're applying the torch-backend-to-tosa-backend-pipeline pass to in problem 1 does not satisfy the backend contract of torch-mlir. As shown by "torch.aten.linear"(%0, %2, %1) : (!torch.tensor<[4,4,192],f32>, !torch.tensor, !torch.tensor) -> !torch.tensor<[4,4,32],f32>, the tensors don't have value semantics and some don't have sizes and dtype.

How are you generating the original IR you're passing to torch-backend-to-tosa-backend-pipeline?

[mgehre-amd]

In addition, TOSA does not support 64-bit integer tensors, so ops like %3 = "torch.prim.NumToTensor.Scalar"(%2) : (!torch.int) -> !torch.tensor<[],si64> cannot be represented in TOSA.

[MIONkb]

The IR you're applying the torch-backend-to-tosa-backend-pipeline pass to in problem 1 does not satisfy the backend contract of torch-mlir. As shown by "torch.aten.linear"(%0, %2, %1) : (!torch.tensor<[4,4,192],f32>, !torch.tensor, !torch.tensor) -> !torch.tensor<[4,4,32],f32>, the tensors don't have value semantics and some don't have sizes and dtype.

How are you generating the original IR you're passing to torch-backend-to-tosa-backend-pipeline?

Thanks so much for your replying. This torch MLIR file is generated as a temp file which is in /tmp/ directory in my ubuntu system, and it is generated in the python function "run_pipeline_with_repro_report()".I run this function with following args:

run_pipeline_with_repro_report(
mb.module,
"builtin.module(torch-backend-to-tosa-backend-pipeline)",
"Lowering Torch Backend IR -> TOSA Backend IR")

Then I copy it("/tmp/SimpleVIT.mlir") to my workspace and run it with " torch-mlir-opt --convert-torch-to-tosa SimpleVIT.mlir", then I got error in the 2nd Problem.

Now I have already locate the wrong instruction : %3 = "torch.prim.NumToTensor.Scalar"(%2) : (!torch.int) -> !torch.tensor<[ ],si64>, and the error occurs in matchandrewrite() of NumToTensorOp. This is the error infomation and the dump stack:

torch-mlir-opt: /home/tianyi/Torchmlir/torch-mlir-main/externals/llvm-project/llvm/include/llvm/Support/Casting.h:566: decltype(auto) llvm::cast(const From&) [with To = mlir::RankedTensorType; From = mlir::Type]: Assertion `isa(Val) && "cast() argument of incompatible type!"' failed.
PLEASE submit a bug report to https://github.com/llvm/llvm-project/issues/ and include the crash backtrace.
Stack dump:

0.  Program arguments: torch-mlir-opt Operation.mlir --convert-torch-to-tosa
   

#0 0x00005581033a732f llvm::sys::PrintStackTrace(llvm::raw_ostream&, int) (/home/tianyi/Torchmlir/torch-mlir-main/build/bin/torch-mlir-opt+0x281232f)
#1 0x00005581033a4b74 SignalHandler(int) Signals.cpp:0:0
#2 0x00007f734e734420 __restore_rt (/lib/x86_64-linux-gnu/libpthread.so.0+0x14420)
#3 0x00007f734e1d700b raise /build/glibc-SzIz7B/glibc-2.31/signal/../sysdeps/unix/sysv/linux/raise.c:51:1
#4 0x00007f734e1b6859 abort /build/glibc-SzIz7B/glibc-2.31/stdlib/abort.c:81:7
#5 0x00007f734e1b6729 get_sysdep_segment_value /build/glibc-SzIz7B/glibc-2.31/intl/loadmsgcat.c:509:8
#6 0x00007f734e1b6729 _nl_load_domain /build/glibc-SzIz7B/glibc-2.31/intl/loadmsgcat.c:970:34
#7 0x00007f734e1c7fd6 (/lib/x86_64-linux-gnu/libc.so.6+0x33fd6)
#8 0x000055810157f43c (/home/tianyi/Torchmlir/torch-mlir-main/build/bin/torch-mlir-opt+0x9ea43c)
#9 0x00005581029db95b (anonymous namespace)::ConvertAtenOpmlir::torch::Torch::PrimNumToTensorScalarOp::matchAndRewrite(mlir::torch::Torch::PrimNumToTensorScalarOp, mlir::torch::Torch::PrimNumToTensorScalarOpAdaptor, mlir::ConversionPatternRewriter&) const TorchToTosa.cpp:0:0
#10 0x000055810297a74e mlir::OpConversionPatternmlir::torch::Torch::PrimNumToTensorScalarOp::matchAndRewrite(mlir::Operation*, llvm::ArrayRefmlir::Value, mlir::ConversionPatternRewriter&) const (/home/tianyi/Torchmlir/torch-mlir-main/build/bin/torch-mlir-opt+0x1de574e)
#11 0x00005581030daaba mlir::ConversionPattern::matchAndRewrite(mlir::Operation*, mlir::PatternRewriter&) const (/home/tianyi/Torchmlir/torch-mlir-main/build/bin/torch-mlir-opt+0x2545aba)
#12 0x000055810311b1e8 mlir::PatternApplicator::matchAndRewrite(mlir::Operation*, mlir::PatternRewriter&, llvm::function_ref<bool (mlir::Pattern const&)>, llvm::function_ref<void (mlir::Pattern const&)>, llvm::function_ref<mlir::LogicalResult (mlir::Pattern const&)>) (/home/tianyi/Torchmlir/torch-mlir-main/build/bin/torch-mlir-opt+0x25861e8)
#13 0x00005581030e8c37 (anonymous namespace)::OperationLegalizer::legalize(mlir::Operation*, mlir::ConversionPatternRewriter&) DialectConversion.cpp:0:0
#14 0x00005581030e95c8 (anonymous namespace)::OperationConverter::convertOperations(llvm::ArrayRefmlir::Operation*, llvm::function_ref<void (mlir::Diagnostic&)>) DialectConversion.cpp:0:0
#15 0x00005581030eb5b4 mlir::applyPartialConversion(mlir::Operation*, mlir::ConversionTarget&, mlir::FrozenRewritePatternSet const&, llvm::DenseSet<mlir::Operation*, llvm::DenseMapInfo<mlir::Operation*, void>>) (/home/tianyi/Torchmlir/torch-mlir-main/build/bin/torch-mlir-opt+0x25565b4)
#16 0x00005581029ee5dc (anonymous namespace)::ConvertTorchToTosa::runOnOperation() TorchToTosa.cpp:0:0
#17 0x0000558103157066 mlir::detail::OpToOpPassAdaptor::run(mlir::Pass, mlir::Operation*, mlir::AnalysisManager, bool, unsigned int) (/home/tianyi/Torchmlir/torch-mlir-main/build/bin/torch-mlir-opt+0x25c2066)
#18 0x000055810315761a mlir::detail::OpToOpPassAdaptor::runPipeline(mlir::OpPassManager&, mlir::Operation*, mlir::AnalysisManager, bool, unsigned int, mlir::PassInstrumentor*, mlir::PassInstrumentation::PipelineParentInfo const*) (/home/tianyi/Torchmlir/torch-mlir-main/build/bin/torch-mlir-opt+0x25c261a)
#19 0x00005581031581ec mlir::detail::OpToOpPassAdaptor::runOnOperationAsyncImpl(bool)::'lambda'(mlir::detail::OpToOpPassAdaptor::runOnOperationAsyncImpl(bool)::OpPMInfo&)::operator()(mlir::detail::OpToOpPassAdaptor::runOnOperationAsyncImpl(bool)::OpPMInfo&) const Pass.cpp:0:0
#20 0x00005581031564bf mlir::detail::OpToOpPassAdaptor::runOnOperationAsyncImpl(bool) (/home/tianyi/Torchmlir/torch-mlir-main/build/bin/torch-mlir-opt+0x25c14bf)
#21 0x0000558103156d7b mlir::detail::OpToOpPassAdaptor::run(mlir::Pass*, mlir::Operation*, mlir::AnalysisManager, bool, unsigned int) (/home/tianyi/Torchmlir/torch-mlir-main/build/bin/torch-mlir-opt+0x25c1d7b)
#22 0x000055810315761a mlir::detail::OpToOpPassAdaptor::runPipeline(mlir::OpPassManager&, mlir::Operation*, mlir::AnalysisManager, bool, unsigned int, mlir::PassInstrumentor*, mlir::PassInstrumentation::PipelineParentInfo const*) (/home/tianyi/Torchmlir/torch-mlir-main/build/bin/torch-mlir-opt+0x25c261a)
#23 0x0000558103158010 mlir::PassManager::run(mlir::Operation*) (/home/tianyi/Torchmlir/torch-mlir-main/build/bin/torch-mlir-opt+0x25c3010)
#24 0x0000558100da1c2b performActions(llvm::raw_ostream&, std::shared_ptrllvm::SourceMgr const&, mlir::MLIRContext*, mlir::MlirOptMainConfig const&) MlirOptMain.cpp:0:0
#25 0x0000558100da23dd processBuffer(llvm::raw_ostream&, std::unique_ptr<llvm::MemoryBuffer, std::default_deletellvm::MemoryBuffer>, mlir::MlirOptMainConfig const&, mlir::DialectRegistry&, llvm::ThreadPool*) MlirOptMain.cpp:0:0
#26 0x0000558100da25b4 mlir::LogicalResult llvm::function_ref<mlir::LogicalResult (std::unique_ptr<llvm::MemoryBuffer, std::default_deletellvm::MemoryBuffer>, llvm::raw_ostream&)>::callback_fn<mlir::MlirOptMain(llvm::raw_ostream&, std::unique_ptr<llvm::MemoryBuffer, std::default_deletellvm::MemoryBuffer>, mlir::DialectRegistry&, mlir::MlirOptMainConfig const&)::'lambda'(std::unique_ptr<llvm::MemoryBuffer, std::default_deletellvm::MemoryBuffer>, llvm::raw_ostream&)>(long, std::unique_ptr<llvm::MemoryBuffer, std::default_deletellvm::MemoryBuffer>, llvm::raw_ostream&) MlirOptMain.cpp:0:0
#27 0x000055810332b2b4 mlir::splitAndProcessBuffer(std::unique_ptr<llvm::MemoryBuffer, std::default_deletellvm::MemoryBuffer>, llvm::function_ref<mlir::LogicalResult (std::unique_ptr<llvm::MemoryBuffer, std::default_deletellvm::MemoryBuffer>, llvm::raw_ostream&)>, llvm::raw_ostream&, bool, bool) (/home/tianyi/Torchmlir/torch-mlir-main/build/bin/torch-mlir-opt+0x27962b4)
#28 0x0000558100d99697 mlir::MlirOptMain(llvm::raw_ostream&, std::unique_ptr<llvm::MemoryBuffer, std::default_deletellvm::MemoryBuffer>, mlir::DialectRegistry&, mlir::MlirOptMainConfig const&) (/home/tianyi/Torchmlir/torch-mlir-main/build/bin/torch-mlir-opt+0x204697)
#29 0x0000558100d9fee3 mlir::MlirOptMain(int, char**, llvm::StringRef, mlir::DialectRegistry&, bool) (/home/tianyi/Torchmlir/torch-mlir-main/build/bin/torch-mlir-opt+0x20aee3)
#30 0x0000558100d79bae main (/home/tianyi/Torchmlir/torch-mlir-main/build/bin/torch-mlir-opt+0x1e4bae)
#31 0x00007f734e1b8083 __libc_start_main /build/glibc-SzIz7B/glibc-2.31/csu/../csu/libc-start.c:342:3
#32 0x0000558100d8da3e _start (/home/tianyi/Torchmlir/torch-mlir-main/build/bin/torch-mlir-opt+0x1f8a3e)
Aborted (core dumped)

What can I do for it? Thanks!!!

[MIONkb]

In addition, TOSA does not support 64-bit integer tensors, so ops like %3 = "torch.prim.NumToTensor.Scalar"(%2) : (!torch.int) -> !torch.tensor<[],si64> cannot be represented in TOSA.

Thanks for your reply! So maybe I should fix the process of "model generated by ‘torch.jit.trace()’ to torch mlir file" and change the type of %3 = "torch.prim.NumToTensor.Scalar"(%2) : (!torch.int) -> !torch.tensor<[],si64>?

[ramiro050]

Thanks so much for your replying. This torch MLIR file is generated as a temp file which is in /tmp/ directory in my ubuntu system, and it is generated in the python function "run_pipeline_with_repro_report()".I run this function with following args:

run_pipeline_with_repro_report(
mb.module,
"builtin.module(torch-backend-to-tosa-backend-pipeline)",
"Lowering Torch Backend IR -> TOSA Backend IR")

Then I copy it("/tmp/SimpleVIT.mlir") to my workspace and run it with " torch-mlir-opt --convert-torch-to-tosa SimpleVIT.mlir", then I got error in the 2nd Problem.

Sorry, my question was not very clear. How are you generating the very first MLIR module mb.module? Before the pipeline torch-backend-to-tosa-backend-pipeline can be applied to an MLIR module, the module needs to go through the pipeline torchscript-module-to-torch-backend-pipeline. Given that your IR in the error messages does not have tensors with value semantics (!torch.vtensor<...>), this means the first pipeline failed. All the errors you're then seeing when doing torch-backend-to-tosa-backend-pipeline and --convert-torch-to-tosa are simply because the very first pipeline failed.

Also, you should not need to run run_pipeline_wth_repro_report directly. The usual flow for converting a torch module to tosa is:

class MyModule(torch.nn.Module):
    ...

my_module = MyModule()
tosa_module = torch_mlir.compile(my_module, example_inputs, output_type="tosa")

[MIONkb]

Sorry, my question was not very clear. How are you generating the very first MLIR module mb.module? Before the pipeline torch-backend-to-tosa-backend-pipeline can be applied to an MLIR module, the module needs to go through the pipeline torchscript-module-to-torch-backend-pipeline. Given that your IR in the error messages does not have tensors with value semantics (!torch.vtensor<...>), this means the first pipeline failed. All the errors you're then seeing when doing torch-backend-to-tosa-backend-pipeline and --convert-torch-to-tosa are simply because the very first pipeline failed.

Also, you should not need to run run_pipeline_wth_repro_report directly. The usual flow for converting a torch module to tosa is:

class MyModule(torch.nn.Module):
    ...

my_module = MyModule()
tosa_module = torch_mlir.compile(my_module, example_inputs, output_type="tosa")

At first, I actually compiled my VIT model with

module = torch_mlir.compile(v, img, output_type="tosa",use_tracing=True,verbose=True)

but it goes wrong, so I extract the codes in torch_mlir.compile() to my main() to check which step is wrong, that's why I run run_pipeline_wth_repro_report().

I attach model lib MySimpleVIT.py and my main.py in the attachment (I give some annotation about where I get mb.module). Now I have already locate the question:
rearrange() function from einops lib will lead to following torch mlir code segment:

    %88 = torch.tensor_static_info_cast %arg1 : !torch.tensor to !torch.tensor<[3,32,32],f32>
    %int0 = torch.constant.int 0
    %89 = torch.aten.size.int %88, %int0 : !torch.tensor<[3,32,32],f32>, !torch.int -> !torch.int
    %90 = torch.prim.NumToTensor.Scalar %89 : !torch.int -> !torch.tensor<[],si64>

which leads to the second problem:

torch-mlir-opt: /home/tianyi/Torchmlir/torch-mlir-main/externals/llvm-project/llvm/include/llvm/Support/Casting.h:566: decltype(auto) llvm::cast(const From&) [with To = mlir::RankedTensorType; From = mlir::Type]: Assertion `isa(Val) && "cast() argument of incompatible type!"' failed.
.......
 Program arguments: torch-mlir-opt Operation.mlir --convert-torch-to-tosa
.......
https://github.com/llvm/torch-mlir/pull/9 0x00005581029db95b (anonymous namespace)::ConvertAtenOpmlir::torch::Torch::PrimNumToTensorScalarOp::matchAndRewrite(mlir::torch::Torch::PrimNumToTensorScalarOp, mlir::torch::Torch::PrimNumToTensorScalarOpAdaptor, mlir::ConversionPatternRewriter&) const TorchToTosa.cpp:0:0

Maybe I should try to change the model or add a right conversion method for rearrange() in torchmlir ?

MySimpleVIT.zip

[MIONkb]

The IR you're applying the torch-backend-to-tosa-backend-pipeline pass to in problem 1 does not satisfy the backend contract of torch-mlir. As shown by "torch.aten.linear"(%0, %2, %1) : (!torch.tensor<[4,4,192],f32>, !torch.tensor, !torch.tensor) -> !torch.tensor<[4,4,32],f32>, the tensors don't have value semantics and some don't have sizes and dtype.

How are you generating the original IR you're passing to torch-backend-to-tosa-backend-pipeline?

I generate a simple linear layer:

  func.func private @__torch__.torch.nn.modules.linear.Linear.forward(%arg0: !torch.nn.Module<"__torch__.torch.nn.modules.linear.Linear">, %arg1: !torch.tensor) -> !torch.tensor {
    %9 = torch.tensor_static_info_cast %arg1 : !torch.tensor to !torch.tensor<[16,8],f32>
    %10 = torch.prim.GetAttr %arg0["weight"] : !torch.nn.Module<"__torch__.torch.nn.modules.linear.Linear"> -> !torch.tensor
    %none_0 = torch.constant.none
    %11 = torch.aten.linear %9, %10, %none_0 : !torch.tensor<[16,8],f32>, !torch.tensor, !torch.none -> !torch.tensor<[16,6],f32>
    %12 = torch.tensor_static_info_cast %11 : !torch.tensor<[16,6],f32> to !torch.tensor
    return %12 : !torch.tensor
  }

Following error for %11 will occur when I run " torch-mlir-opt --convert-torch-to-tosa simpleLinear.mlir":`

 #9 0x000055e2581b19c5 (anonymous namespace)::ConvertAtenLinearOp<mlir::torch::Torch::AtenLinearOp>::matchAndRewrite(mlir::torch::Torch::AtenLinearOp, mlir::torch::Torch::AtenLinearOpAdaptor, mlir::ConversionPatternRewriter&) const TorchToTosa.cpp:0:0

then I change %10's and %11's tensor shape to:

  func.func private @__torch__.torch.nn.modules.linear.Linear.forward(%arg0: !torch.nn.Module<"__torch__.torch.nn.modules.linear.Linear">, %arg1: !torch.tensor) -> !torch.tensor {
    %9 = torch.tensor_static_info_cast %arg1 : !torch.tensor to !torch.tensor<[16,8],f32>
    %10 = torch.prim.GetAttr %arg0["weight"] : !torch.nn.Module<"__torch__.torch.nn.modules.linear.Linear"> -> !torch.tensor
    %tmp = torch.tensor_static_info_cast %10 : !torch.tensor to !torch.tensor<[8,6],f32>
    %none_0 = torch.constant.none
    %11 = torch.aten.linear %9, %tmp, %none_0 : !torch.tensor<[16,8],f32>, !torch.tensor<[8,6],f32>, !torch.none -> !torch.tensor<[16,6],f32>
    %12 = torch.tensor_static_info_cast %11 : !torch.tensor<[16,6],f32> to !torch.tensor
    return %12 : !torch.tensor
  }

But the same error still occurs.

[ramiro050]

At first, I actually compiled my VIT model with

module = torch_mlir.compile(v, img, output_type="tosa",use_tracing=True,verbose=True)

but it goes wrong

What error do you get? When torch_mlir.compile fails, it should generate a temporary *.mlir file and the error message should contain the torch-mlir-opt command to run to reproduce the error from the command line. Can you paste the error here and also attach the *.mlir temporary file generated by torch_mlir.compile?

[MIONkb]

What error do you get? When torch_mlir.compile fails, it should generate a temporary *.mlir file and the error message should contain the torch-mlir-opt command to run to reproduce the error from the command line. Can you paste the error here and also attach the *.mlir temporary file generated by torch_mlir.compile?

Thanks for your reply. The error is showed in my first comment:

python exception: Failure while executing pass pipeline:
error: unknown: failed to legalize operation 'torch.prim.NumToTensor.Scalar' that was explicitly marked illegal
note: unknown: see current operation: %3 = "torch.prim.NumToTensor.Scalar"(%2) : (!torch.int) -> !torch.tensor<[],si64>
error: "/home/tianyi/anaconda3/envs/torch-mlir/lib/python3.10/site-packages/torch/nn/modules/linear.py":114:0: aten.Linear called but weight rank not 2 or 3
note: "/home/tianyi/anaconda3/envs/torch-mlir/lib/python3.10/site-packages/torch/nn/modules/linear.py":114:0: see current operation: %3 = "torch.aten.linear"(%0, %2, %1) : (!torch.tensor<[4,4,192],f32>, !torch.tensor, !torch.tensor) -> !torch.tensor<[4,4,32],f32>
error: "/home/tianyi/anaconda3/envs/torch-mlir/lib/python3.10/site-packages/torch/nn/modules/linear.py":114:0: failed to legalize operation 'torch.aten.linear' that was explicitly marked illegal
note: "/home/tianyi/anaconda3/envs/torch-mlir/lib/python3.10/site-packages/torch/nn/modules/linear.py":114:0: see current operation: %3 = "torch.aten.linear"(%0, %2, %1) : (!torch.tensor<[4,4,192],f32>, !torch.tensor, !torch.tensor) -> !torch.tensor<[4,4,32],f32>
error: "/home/tianyi/anaconda3/envs/torch-mlir/lib/python3.10/site-packages/torch/nn/modules/linear.py":114:0: aten.Linear called but weight rank not 2 or 3
note: "/home/tianyi/anaconda3/envs/torch-mlir/lib/python3.10/site-packages/torch/nn/modules/linear.py":114:0: see current operation: %3 = "torch.aten.linear"(%0, %1, %2) : (!torch.tensor<[16,32],f32>, !torch.tensor, !torch.none) -> !torch.tensor<[16,576],f32>
error: "/home/tianyi/anaconda3/envs/torch-mlir/lib/python3.10/site-packages/torch/nn/modules/linear.py":114:0: failed to legalize operation 'torch.aten.linear' that was explicitly marked illegal
note: "/home/tianyi/anaconda3/envs/torch-mlir/lib/python3.10/site-packages/torch/nn/modules/linear.py":114:0: see current operation: %3 = "torch.aten.linear"(%0, %1, %2) : (!torch.tensor<[16,32],f32>, !torch.tensor, !torch.none) -> !torch.tensor<[16,576],f32>

and the temp mlir file is the same with the mlir file I attached above:
#2096 (comment)

[ramiro050]

The main issue is that the model uses the ops

%68 = torch.operator "aten.meshgrid.indexing"(%67, %str) : (!torch.list<tensor>, !torch.str) -> !torch.list<tensor>
%119 = torch.operator "aten.chunk"(%118, %int3, %int-1) : (!torch.tensor<[16,576],f32>, !torch.int, !torch.int) -> !torch.list<tensor>

which we currently don't have support for in torch-mlir. Because torch-mlir does not know how to handle these, they prevent torch-mlir from fully converting the graph to value semantics, which is why you see some ops with !torch.tensor args rather than !torch.vtensor args.

[MIONkb]

which we currently don't have support for in torch-mlir. Because torch-mlir does not know how to handle these, they prevent torch-mlir from fully converting the graph to value semantics, which is why you see some ops with !torch.tensor args rather than !torch.vtensor args.

Thanks for your answering and helping! I just make up another issue about nn.linear() and rearrange() seconds ago.The problem of rearrange() is caused by the reason you here said. Thanks again!