'NotImplementedError: The following operators are not implemented:' errors when converting PyTorch Object Detection models

[achalhoubmeaa]

I am trying to work with the SSD300_VGG16 object detection model and convert it to a serialized and optimized format to eventually run on an embedded system. The script that I use to create and save the traced model from TorchScript is the following:

import torch
import torchvision

def do_trace(model, in_size=500):
    model_trace = torch.jit.trace(model, torch.rand(1, 3, in_size, in_size))
    model_trace.eval()
    return model_trace


def dict_to_tuple(out_dict):
    if "masks" in out_dict.keys():
        return (out_dict["boxes"], out_dict["scores"], out_dict["labels"], out_dict["masks"])
    return (out_dict["boxes"], out_dict["scores"], out_dict["labels"])


class TraceWrapper(torch.nn.Module):
    def __init__(self, model):
        super().__init__()
        self.model = model

    def forward(self, inp):
        out = self.model(inp)
        return dict_to_tuple(out[0])


model_funcs = [torchvision.models.detection.ssd300_vgg16]

names = ["ssd300_vgg16"]

for name, model_func in zip(names, model_funcs):

    model = TraceWrapper(model_func(num_classes=50, pretrained_backbone=False))

    model.eval()
    in_size = 500
    inp = torch.rand(1, 3, in_size, in_size)

    with torch.no_grad():
        out = model(inp)
        
        script_module = do_trace(model)
        script_out = script_module(inp)

        assert len(out[0]) > 0 and len(script_out[0]) > 0
        torch._C._jit_pass_inline(script_module.graph)
        torch.jit.save(script_module, name + ".pt")

After that I would like to convert the saved model to another format that I am working with, but I am getting the error below:

File "/home/achalhoub/tvm/python/tvm/relay/frontend/pytorch.py", line 3091, in report_missing_conversion
raise NotImplementedError(msg)
NotImplementedError: The following operators are not implemented: ['aten::clamp_min', 'aten::copy_']

From the research that I've done on this, it seems like TVM doesn't support the conversion of some OPs in certain model architectures?

I would appreciate help with this issue!

Thanks.

I am using the following versions:

torch 1.10.1+cu102
torchvision 0.11.2+cu102
tvm 0.9.dev0

[masahi]

aten::copy_ cannot convert to relay since this is an in-place op. So the model needs to be modified to remove mutation on PyTorch side.

[achalhoubmeaa]

Thank you for your response. Would you mind briefly explaining what an in-place op is? Or maybe guide me to some documentation so I can better understand?

[masahi]

so aten::copy_ apparently comes from these lines https://github.com/pytorch/vision/blob/d367a01a18a3ae6bee13d8be3b63fd6a581ea46f/torchvision/models/detection/anchor_utils.py#L267-L268

This is an example of in-place op (values are mutated in-place). We need to replace this with "pure" operation for conversion to TVM.

[masahi]

FYI I sent a PR to torchvision to remove the in-place op in question pytorch/vision#5279 (comment)

[achalhoubmeaa]

Thank you for explaining and sending a PR.

What about the ['aten::clamp_min'] error as well? How can I know what section/part of the network caused this error?

[masahi]

I've just added clamp_min support in https://github.com/apache/tvm/compare/main...masahi:torch-clamp-min?expand=1

With the two patches, the script https://github.com/masahi/torchscript-to-tvm/blob/master/ssd-vgg/test.py should work. But currently the outputs from TVM are very strange, I'll take a look.

[achalhoubmeaa]

I will try to convert the model again with these two patches and let you know if I have any other issues. Thank you!

[achalhoubmeaa]

@masahi is there any guidance you could give me on how you were able to identify what part of the model code were responsible for producing those errors please? Especially the aten::copy_ error. I understand them once you explained them but I couldn't have been able to identify them myself, and it would be useful to me to have some knowledge of how to debug for those kinds of errors since I will be working with multiple PyTorch models in the future.

I would really appreciate it!

[masahi]

Yeah that's a good question. The way I did it was, first put print(graph) after

tvm/python/tvm/relay/frontend/pytorch.py

Line 3913 in 095b639

_run_jit_passes(graph)

and find aten::copy_. The dump shows for each aten:: op its corresponding location in the python source code. That's how I found that code in anchor_utils.py.

[achalhoubmeaa]

Thank you so much! This seems like a really helpful tool for debugging such issues. I appreciate your help :)

[DzAvril]

aten::copy_ cannot convert to relay since this is an in-place op. So the model needs to be modified to remove mutation on PyTorch side.

@masahi May I ask why the in-place ops cannot be converted to relay? In pytorch front pytorch.py there are in-place ops like aten::add_ has been converted to relay. So what is the difference between aten::copy_ and aten::add_？