converting 'cat' operation

[Mayor2305]

🐞Describing the bug

• Make sure to only create an issue here for bugs in the coremltools Python package. If this is a bug with the Core ML Framework or Xcode, please submit your bug here: https://developer.apple.com/bug-reporting/
• Provide a clear and consise description of the bug.
  While converting from detectron2 model to coreml, got the following error:

ValueError: dtypes needs to be a list/tuple of at least 1 element

Stack Trace

ERROR - converting 'cat' op (located at: 'roi_heads/box_pooler'):

Converting PyTorch Frontend ==> MIL Ops: 75%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████▉ | 1426/1891 [00:00<00:00, 1685.47 ops/s]
Traceback (most recent call last):
File "/home/user/Desktop/drive/segmentation/model/torch2coreml.py", line 12, in
mlmodel = ct.converters.convert(
File "/home/user/anaconda3/envs/coreml/lib/python3.10/site-packages/coremltools/converters/_converters_entry.py", line 581, in convert
mlmodel = mil_convert(
File "/home/user/anaconda3/envs/coreml/lib/python3.10/site-packages/coremltools/converters/mil/converter.py", line 188, in mil_convert
return _mil_convert(model, convert_from, convert_to, ConverterRegistry, MLModel, compute_units, **kwargs)
File "/home/user/anaconda3/envs/coreml/lib/python3.10/site-packages/coremltools/converters/mil/converter.py", line 212, in _mil_convert
proto, mil_program = mil_convert_to_proto(
File "/home/user/anaconda3/envs/coreml/lib/python3.10/site-packages/coremltools/converters/mil/converter.py", line 288, in mil_convert_to_proto
prog = frontend_converter(model, **kwargs)
File "/home/user/anaconda3/envs/coreml/lib/python3.10/site-packages/coremltools/converters/mil/converter.py", line 108, in call
return load(*args, **kwargs)
File "/home/user/anaconda3/envs/coreml/lib/python3.10/site-packages/coremltools/converters/mil/frontend/torch/load.py", line 82, in load
return _perform_torch_convert(converter, debug)
File "/home/user/anaconda3/envs/coreml/lib/python3.10/site-packages/coremltools/converters/mil/frontend/torch/load.py", line 116, in _perform_torch_convert
prog = converter.convert()
File "/home/user/anaconda3/envs/coreml/lib/python3.10/site-packages/coremltools/converters/mil/frontend/torch/converter.py", line 581, in convert
convert_nodes(self.context, self.graph)
File "/home/user/anaconda3/envs/coreml/lib/python3.10/site-packages/coremltools/converters/mil/frontend/torch/ops.py", line 86, in convert_nodes
raise e # re-raise exception
File "/home/user/anaconda3/envs/coreml/lib/python3.10/site-packages/coremltools/converters/mil/frontend/torch/ops.py", line 81, in convert_nodes
convert_single_node(context, node)
File "/home/user/anaconda3/envs/coreml/lib/python3.10/site-packages/coremltools/converters/mil/frontend/torch/ops.py", line 134, in convert_single_node
add_op(context, node)
File "/home/user/anaconda3/envs/coreml/lib/python3.10/site-packages/coremltools/converters/mil/frontend/torch/ops.py", line 2115, in cat
values=promote_input_dtypes(xs), axis=axis, name=node.name
File "/home/user/anaconda3/envs/coreml/lib/python3.10/site-packages/coremltools/converters/mil/mil/ops/defs/_utils.py", line 452, in promote_input_dtypes
promoted_dtype = promote_dtypes([var.dtype for var in input_vars])
File "/home/user/anaconda3/envs/coreml/lib/python3.10/site-packages/coremltools/converters/mil/mil/types/type_mapping.py", line 264, in promote_dtypes
raise ValueError("dtypes needs to be a list/tuple of at least 1 element")
ValueError: dtypes needs to be a list/tuple of at least 1 element

System environment (please complete the following information):

• coremltools version: 7.2
• OS (e.g. MacOS version or Linux type): Ubuntu 20.04.6 LTS
• Any other relevant version information (e.g. PyTorch or TensorFlow version): torch==2.2.0, detectron2==0.6

Additional context

• also initially i was getting a KeyError: 4 for the full operation, to counter that, i copied code from the latest beta release for the full operation. the copied code:

@register_torch_op
def full(context, node):
    inputs = _get_inputs(context, node, min_expected=2)

    size = inputs[0]

    # dtype could be torch.dtype or an integer that maps to a numpy.dtype
    dtype = None
    if len(inputs) < 3 or inputs[2] is None:
        dtype = np.float32
    elif isinstance(inputs[2].val, torch.dtype):
        dtype = NUM_TO_NUMPY_DTYPE[TORCH_DTYPE_TO_NUM[inputs[2].val]]
    elif isinstance(inputs[2].val, (int, np.generic)):
        dtype = NUM_TO_NUMPY_DTYPE[inputs[2].val]
    else:
        raise ValueError(f"unsupported type {type(inputs[2].val)}.")

    val = dtype(inputs[1].val)

    result = _make_fill_op(size, val, node.name)
    context.add(result)

[junpeiz]

Could you provide a code snippet for reproducing this? Thanks!

[Mayor2305]

Here is a colab notebook to reproduce the issue. https://colab.research.google.com/drive/191HxvCwjIFlUIT3gOBPl7YGUZNnUZTJi?usp=sharing,

there are two issues:

1. when you run this notebook, you will get an error KeyError: 4. I soved this error by changing the code for full operation at coremltools/converters/mil/frontend/torch/ops.py (took it from coremltool 8 beta) to:

@register_torch_op
def full(context, node):
    inputs = _get_inputs(context, node, min_expected=2)

    size = inputs[0]

    # dtype could be torch.dtype or an integer that maps to a numpy.dtype
    dtype = None
    if len(inputs) < 3 or inputs[2] is None:
        dtype = np.float32
    elif isinstance(inputs[2].val, torch.dtype):
        dtype = NUM_TO_NUMPY_DTYPE[TORCH_DTYPE_TO_NUM[inputs[2].val]]
    elif isinstance(inputs[2].val, (int, np.generic)):
        dtype = NUM_TO_NUMPY_DTYPE[inputs[2].val]
    else:
        raise ValueError(f"unsupported type {type(inputs[2].val)}.")

    val = dtype(inputs[1].val)

    result = _make_fill_op(size, val, node.name)
    context.add(result)

PLEASE RESTART THE NOTEBOOK AFTER THIS CHANGE.

2. Then I got the error for cat operation: ValueError: dtypes needs to be a list/tuple of at least 1 element

I hope this will help reproduce. Please let me know if you need anything else.

[Mayor2305]

Any updates yet?

[junpeiz]

Thank you for providing the detailed steps to reproduce! I can reproduce the issue. We are investigating on the root cause of this bug.