RuntimeError: pytorch_half_pixel mode unsupported

[johannesSX]

Hello,

I want to convert a model from onnx to pb format in tensorflow.
The following error occurs here:

File "/Users/.../convert_model.py", line 72, in <module>
    convert_onnx2pb()
  File "/Users/.../convert_model.py", line 44, in convert_onnx2pb
    tf_rep = prepare(onnx_model, strict=False, logging_level='DEBUG')
  File "/Users/.../opt/anaconda3/envs/big_convert_3/lib/python3.6/site-packages/onnx_tf-1.5.0-py3.6.egg/onnx_tf/backend.py", line 65, in prepare
    return cls.onnx_model_to_tensorflow_rep(model, strict)
  File "/Users/.../opt/anaconda3/envs/big_convert_3/lib/python3.6/site-packages/onnx_tf-1.5.0-py3.6.egg/onnx_tf/backend.py", line 85, in onnx_model_to_tensorflow_rep
    return cls._onnx_graph_to_tensorflow_rep(model.graph, opset_import, strict)
  File "/Users/.../opt/anaconda3/envs/big_convert_3/lib/python3.6/site-packages/onnx_tf-1.5.0-py3.6.egg/onnx_tf/backend.py", line 143, in _onnx_graph_to_tensorflow_rep
    onnx_node, tensor_dict, handlers, opset=opset, strict=strict)
  File "/Users/.../opt/anaconda3/envs/big_convert_3/lib/python3.6/site-packages/onnx_tf-1.5.0-py3.6.egg/onnx_tf/backend.py", line 245, in _onnx_node_to_tensorflow_op
    return handler.handle(node, tensor_dict=tensor_dict, strict=strict)
  File "/Users/.../opt/anaconda3/envs/big_convert_3/lib/python3.6/site-packages/onnx_tf-1.5.0-py3.6.egg/onnx_tf/handlers/handler.py", line 60, in handle
    cls.args_check(node, **kwargs)
  File "/Users/.../opt/anaconda3/envs/big_convert_3/lib/python3.6/site-packages/onnx_tf-1.5.0-py3.6.egg/onnx_tf/handlers/backend/resize.py", line 50, in args_check
    coordinate_transformation_mode, "Tensorflow")
  File "/Users/.../opt/anaconda3/envs/big_convert_3/lib/python3.6/site-packages/onnx_tf-1.5.0-py3.6.egg/onnx_tf/common/exception.py", line 50, in __call__
    raise self._func(self.get_message(op, framework))
RuntimeError: Resize coordinate_transformation_mode=pytorch_half_pixel is not supported in Tensorflow.

When executing the following code (prepare function):

from onnx_tf.backend import prepare
tf_rep = prepare(onnx_model, logging_level='DEBUG')

I use the follwing software/python packages:

• python 3.6.10
• TensorFlow 2.2.0
• onnx 1.7.0
• onnx-tf 1.5.0, but the resize11 branch from @winnietsang
  • if i use the master branch, the resize error mentioned here occurs. thats why i use the resize11 branch. Thank you @winnietsang for implementing this!
• onnx model have opset_version 11

How can i solve the coordinate_transformation_mode=pytorch_half_pixel problem? Any help would be greatly appreciated :)

[winnietsang]

@johannesSX Currently only limited modes/options introduced in Resize opset 11 are supported. More investigation needed for other modes/options due to framework incompatibility.

[jf99]

@johannesSX Currently only limited modes/options introduced in Resize opset 11 are supported.

Can you say which modes/options exactly are currently supported? Maybe I can modify my ONNX file to match onnx-tf's requirements.

Edit: Sorry, I was blind. The error message is quite clear and alternative coordinate_transformation_modes can be found in the ONNX doc.

[winnietsang]

@jf99 Resize required 4D input in Tensorflow. For opset 11, only the following attributes and inputs combination are supported in onnx-tf:

mode=nearest, coordinate_transformation_mode=align_corners, nearest_mode=round_prefer_ceil, can use scales(*) or sizes.
mode=nearest, coordinate_transformation_mode=asymmetric, nearest_mode=floor, can use scales(*) or sizes.
mode=nearest, coordinate_transformation_mode=tf_half_pixel_for_nn, nearest_mode=floor, can use scales(*) or sizes.
mode=linear, coordinate_transformation_mode=align_corners, can use scales(*) or sizes.
mode=linear, coordinate_transformation_mode=asymmetric, can use scales(*) or sizes.
mode=linear, coordinate_transformation_mode=half_pixel, can use scales(*) or sizes.
mode=cubic, coordinate_transformation_mode=align_corners, cubic_coeff_a=-0.5, exclude_outside=1, can use scales(*) or sizes.
mode=cubic, coordinate_transformation_mode=asymmetric, cubic_coeff_a=-0.5, exclude_outside=1, can use scales(*) or sizes.
mode=cubic, coordinate_transformation_mode=half_pixel, cubic_coeff_a=-0.5, exclude_outside=1, can use scales(*) or sizes.
mode=nearest, coordinate_transformation_mode=tf_crop_and_resize, extrapolation_value=any_float_value, nearest_mode=round_prefer_ceil, can use scales or sizes.
mode=linear, coordinate_transformation_mode=tf_crop_and_resize, extrapolation_value=any_float_value, can use scales or sizes.

Note (*): The accuracy of your model will go down, if the height and the width of the new sizes(scales * origial sizes) are not in whole numbers.

[Tian14267]

Do you solve your problem? I get same problem, and don't know how to do ..... @johannesSX

[johannesSX]

Do you solve your problem? I get same problem, and don't know how to do ..... @johannesSX

I tried to solve it, but it is still not working. I implement a resize version, where the error not occurred. But the generated results are still different.

My goal was, to convert an pth pytorch model to tensorflow. Because i had access to the underlying pytorch model it was possible to rebuild it manuelly in tensorflow. Then i wrote an programm, that transfered the weights directly from pth file to tensorflow pb file. And it is working.

That may also work for you, but is very time intensive.

[Tian14267]

Oh My god. It seems too complicated .I still don't know why we get 'pytorch_half_pixel' in onnx model. It seems happened in upsample in torch to onnx.
My head goes blank....

[zshn25]

Using align_corners=True in Upsample solved this for me

[cenkbircanoglu]

+1

[omerferhatt]

Please check onnx/onnx#2784 (comment), setting opset_version=10 will solve the issue

[anilsathyan7]

When i tried setting opset_version=10 there was a warning during conversion process:

/usr/local/lib/python3.6/dist-packages/torch/nn/functional.py:3103: UserWarning: The default behavior for interpolate/upsample with float scale_factor changed in 1.6.0 to align with other frameworks/libraries, and now uses scale_factor directly, instead of relying on the computed output size. If you wish to restore the old behavior, please set recompute_scale_factor=True. See the documentation of nn.Upsample for details.
warnings.warn("The default behavior for interpolate/upsample with float scale_factor changed "
/usr/local/lib/python3.6/dist-packages/torch/onnx/symbolic_helper.py:267: UserWarning: You are trying to export the model with onnx:Resize for ONNX opset version 10. This operator might cause results to not match the expected results by PyTorch.
ONNX's Upsample/Resize operator did not match Pytorch's Interpolation until opset 11. Attributes to determine how to transform the input were added in onnx:Resize in opset 11 to support Pytorch's behavior (like coordinate_transformation_mode and nearest_mode).
We recommend using opset 11 and above for models using this operator.

The model was converted successfully; but the output from the onnx model was different from the original pytorch model.

The original model was trained with bilinear upsample and align corners=False
F.interpolate(..., scale_factor=2, mode='bilinear', align_corners=False)

Even if we change align_corners=True in the model definition(and load weights), the output is different from the original trained model. How can we convert such a model to tensorflow, with correct upsample behaviour?

The only option seems to be to retrain the original model with align_corners=True and mode='bilinear'

[Adhithya-tech]

@zshn25 @anilsathyan7 Could you tell me how o change the model definition for pretrained models?

[mohammedayub44]

Hi,
I'm doing text boundary detections using CRAFT and I get the same error as reported but only when I use opset_version=11 and then try to run the exported model in TensorFlow. When I don't give any version, I do get the warning but my model seems to run fine. I tested with couple of images and boundaries come out to be fairly okay.

I visualized both the models (with and without version 11) in Netron UpSample does get converted to Resize Op when using version 11. As reported by @winnietsang bilinear and half_pixel combination is not supported.

Let me know if there is a new version that supports bilinear upsample. Unfortunately cannot retrain the model, sticking to using the okayish one for now.

[mmeendez8]

Is there any new about this?
Followed @omerferhatt suggestion of opset_version=10 but I get an error:

return torch.max_pool2d(input, kernel_size, stride, padding, dilation, ceil_mode)
Traceback (most recent call last):
  File "pytorch2onnx_seg.py", line 383, in <module>
    pytorch2onnx(
  File "pytorch2onnx_seg.py", line 190, in pytorch2onnx
    torch.onnx.export(
  File "/opt/miniconda3/envs/model_exporter/lib/python3.8/site-packages/torch/onnx/__init__.py", line 275, in export
    return utils.export(model, args, f, export_params, verbose, training,
  File "/opt/miniconda3/envs/model_exporter/lib/python3.8/site-packages/torch/onnx/utils.py", line 88, in export
    _export(model, args, f, export_params, verbose, training, input_names, output_names,
  File "/opt/miniconda3/envs/model_exporter/lib/python3.8/site-packages/torch/onnx/utils.py", line 718, in _export
    _check_onnx_proto(proto)
RuntimeError: Node (Resize_145) has input size 4 not in range [min=2, max=2].

==> Context: Bad node spec: input: "572" input: "582" input: "590" input: "589" output: "591" name: "Resize_145" op_type: "Resize" attribute { name: "coordinate_transformation_mode" s: "asymmetric" type: STRING } attribute { name: "cubic_coeff_a" f: -0.75 type: FLOAT } attribute { name: "mode" s: "nearest" type: STRING } attribute { name: "nearest_mode" s: "floor" type: STRING }

Also tried with @anilsathyan7 idea of setting align_corners=True andmode='bilinear' but no success.

This is the script I am using for converting the model:
https://github.com/open-mmlab/mmsegmentation/blob/master/tools/pytorch2onnx.py