Errors when uploading certain pictures from my cell phone

[liusida]

When I try to use ComfyUI on my cell phone (Android), sometimes, when I try to upload an image, it will give error like this:

!!! Exception during processing!!! Sizes of tensors must match except in dimension 0. Expected size 2688 but got size 1018 for tensor number 1 in the list.
Traceback (most recent call last):
  File "/mnt/data/ComfyUI-BS/ComfyUI/execution.py", line 151, in recursive_execute
    output_data, output_ui = get_output_data(obj, input_data_all)
                             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/mnt/data/ComfyUI-BS/ComfyUI/execution.py", line 81, in get_output_data
    return_values = map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True)
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/mnt/data/ComfyUI-BS/ComfyUI/execution.py", line 74, in map_node_over_list
    results.append(getattr(obj, func)(**slice_dict(input_data_all, i)))
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/mnt/data/ComfyUI-BS/ComfyUI/nodes.py", line 1480, in load_image
    output_image = torch.cat(output_images, dim=0)
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
RuntimeError: Sizes of tensors must match except in dimension 0. Expected size 2688 but got size 1018 for tensor number 1 in the list.

And when I paused the process right before the error happens, and I inspect the variable img, I saw this (with n_frames=2), but it's a JPG! I don't know why, and ChatGPT says that it's a MPO file.

Extracted with ChatGPT:

img: <PIL.MpoImagePlugin.MpoImageFile image mode=L size=572x1018 at 0x7F1E153924D0>

special variables
function variables

app: {'APP1': b'http://ns.adobe.com/xmp/extension/\x01\x00\x08\xAC0B6D96AB3A7066059E016278C9C8...\x00\x00\x92\x00\x00\x10\x01\x02\x00\x08\x00\x00\x00'}
applist: [('APP0', b'JFIF\x00\x01\x02\x00\x48\x00\x48\x00\x00'), ('APP1', b'http://ns.adobe.com/xap/1.0/\x00...\x00\x00\x92\x00\x00\x12\x01\x03...\x00')]
bits: 8
custom_mimetype: None
decoderconfig: {}
decodermaxblock: 65536
filename: '/mnt/data/ComfyUI-BS/ComfyUI/input/PXL_20240507_013340406 (5).jpg'
format: 'MPO'
format_description: 'MPO (CIPA DC-007)'
fp: None
has_transparency_data: False
height: 2688
huffman_ac: {}
huffman_dc: {}
icclist: []
im: <ImagingCore object at 0x7f1e1422d5e0>
info: {'dpi': (72.0, 72.0), 'jfif': 258, 'jfif_version': (1, 2)}
is_animated: True
layer: [(1, 2, 2, 0), (2, 1, 1, 1), (3, 1, 1, 1)]
layers: 3
map: None
mode: 'RGB'
mpinfo: {45056: b'0100', 45057: 2, 45058: [{...}, {...}]}
n_frames: 2
offset: 0
palette: None
pyaccess: None
quantization: {0: [3, 3, 2, 3, 5, 8, 10, 12, 2, 4, 4, ...], 1: [3, 3, 5, 7, 11, 11, 12, 3, 4, 6, 8, 10, 2, 4, 4, ...]}
readonly: 0
size: (1512, 2688)
tile: []

I never encounter this kind of JPGs on my Desktop, ether Ubuntu or Windows. I'm not sure why cell phones store some JPGs like that (no all JPGs have two frames, only some of them do). But this causes errors in ComfyUI's LoadImage node.

I'll try to upload one of the JPG files here in the comment. To reproduce the error, save the tomato photo and try to upload with ComfyUI's Load Image node.

[liusida]

[comfyanonymous]

Should be fixed now.

[liusida]

@shawnington I've just tested your update #3422. And interestingly, it works for PIL=10.2.0, but the very picture caused another error when using PIL=10.3.0.

I think the tomato picture is weird, because the second frame is not the same as the first frame, I don't know why my Pixel 6's default camera save the photo this way, but it's real.

Here is the error message:

Error occurred when executing LoadImage:

Sizes of tensors must match except in dimension 0. Expected size 2688 but got size 1018 for tensor number 1 in the list.

File "C:\ComfyUI\ComfyUI-origin\execution.py", line 151, in recursive_execute
output_data, output_ui = get_output_data(obj, input_data_all)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\ComfyUI\ComfyUI-origin\execution.py", line 81, in get_output_data
return_values = map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\ComfyUI\ComfyUI-origin\execution.py", line 74, in map_node_over_list
results.append(getattr(obj, func)(**slice_dict(input_data_all, i)))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\ComfyUI\ComfyUI-origin\nodes.py", line 1481, in load_image
output_image = torch.cat(output_images, dim=0)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

And I found that the dimension of two frames are different:

and I found that the 10.3.0 version ImageOps.exif_transpose returns a smaller dimension image.

==
Of course, the tomato photo is absolutely a weird one, here is the visual inspection of the second frame using PIL == 10.2.0:

[liusida]

I just tested all my photos taken using my Pixel 6's default camera, they all behave the same. (2 frames, and the second is not quite standard)
Here's another example:

[shawnington]

@shawnington I've just tested your update #3422. And interestingly, it works for PIL=10.2.0, but the very picture caused another error when using PIL=10.3.0.

I think the tomato picture is weird, because the second frame is not the same as the first frame, I don't know why my Pixel 6's default camera save the photo this way, but it's real.

Here is the error message:

Error occurred when executing LoadImage:

Sizes of tensors must match except in dimension 0. Expected size 2688 but got size 1018 for tensor number 1 in the list.

File "C:\ComfyUI\ComfyUI-origin\execution.py", line 151, in recursive_execute
output_data, output_ui = get_output_data(obj, input_data_all)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\ComfyUI\ComfyUI-origin\execution.py", line 81, in get_output_data
return_values = map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\ComfyUI\ComfyUI-origin\execution.py", line 74, in map_node_over_list
results.append(getattr(obj, func)(**slice_dict(input_data_all, i)))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\ComfyUI\ComfyUI-origin\nodes.py", line 1481, in load_image
output_image = torch.cat(output_images, dim=0)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

And I found that the dimension of two frames are different:

and I found that the 10.3.0 version ImageOps.exif_transpose returns a smaller dimension image.

== Of course, the tomato photo is absolutely a weird one, here is the visual inspection of the second frame using PIL == 10.2.0:

Its unfortunately going to be a tricky issue to keep up with, and Im sure our triage will break with different versions of PIL, as its not an actual comfy bug, its a work around for a fairly long standing PIL bug. Id recommend using the version that works, and maybe we should looking into including version 10.2.0 in the requirements.txt so long as it doesn't break additional functionality.

Im going to do some more digging, and see if there is a more robust way to solve this, as photos from phones are quite a common input, for myself as well. I take pictures with my phone that I wan't to re-imagine into something cool, so I understand your frustration. Can you by chance provide the make model and software version for the phone you are have errors with?

There is always a small chance that there is a known bug with the phone that is causing errors.

The commit took the fix Comfy contributed and merged it into a generalized fix for embedded ICC profiles in PNGs being interpreted as decompression attacks, as they both used the exact same solution to resolve the issue. So it just made it a wrapper function that you can pass any PIL function that is impacted by that kind of bug into.

I'm going to have to look into the differences between pillow 10.3.0 and 10.2.0, maybe it will elucidate an actual reason the bug is happening.

[ltdrdata]

Its unfortunately going to be a tricky issue to keep up with, and Im sure our triage will break with different versions of PIL, as its not an actual comfy bug, its a work around for a fairly long standing PIL bug. Id recommend using the version that works, and maybe we should looking into including version 10.2.0 in the requirements.txt so long as it doesn't break additional functionality.

Im going to do some more digging, and see if there is a more robust way to solve this, as photos from phones are quite a common input, for myself as well. I take pictures with my phone that I wan't to re-imagine into something cool, so I understand your frustration. Can you by chance provide the make model and software version for the phone you are have errors with?

There is always a small chance that there is a known bug with the phone that is causing errors.

Maybe we need to spec as Pillow!=10.3.0

[liusida]

For the app:
version 9.3.160.621982096.22
com.google.android.GoogleCamera

My phone is Pixel 6, manufactured in 2021 with hardware version MP1.0.

[shawnington]

Its unfortunately going to be a tricky issue to keep up with, and Im sure our triage will break with different versions of PIL, as its not an actual comfy bug, its a work around for a fairly long standing PIL bug. Id recommend using the version that works, and maybe we should looking into including version 10.2.0 in the requirements.txt so long as it doesn't break additional functionality.
Im going to do some more digging, and see if there is a more robust way to solve this, as photos from phones are quite a common input, for myself as well. I take pictures with my phone that I wan't to re-imagine into something cool, so I understand your frustration. Can you by chance provide the make model and software version for the phone you are have errors with?
There is always a small chance that there is a known bug with the phone that is causing errors.

Maybe we need to spec as Pillow!=10.3.0

Thats actually much more reasonable, as long as we confirm its just isolated to 10.3.0

Im not super familiar with the proper format to spec requirements.txt for something like this, would you want to commit the change? If not I'll figure it out and commit it.

[liusida]

I just digged into the camera's settings, and I found that if I disable the Ultra HDR feature (which was enabled by default), I get only one frame!

Here's the documentation of this feature (with it's format JPEG_R!):
https://source.android.com/docs/core/camera/ultra-hdr

[liusida]

And follow this lead, I found this issue from PIL:

python-pillow/Pillow#8036

Yes, they need to support that to solve this issue.

[shawnington]

I just digged into the camera's settings, and I found that if I disable the Ultra HDR feature (which was enabled by default), I get only one frame!

I actually was about to post a follow up question about that. There is an outstanding issue having to do with HDR images that looked very similar to yours, where a smaller image that appeared to be encoding exposure data was included in a similar format, it seems to be limited to android currently.

It's definitely a PIL issue, I'll update with the issue number when I find it.

[shawnington]

And follow this lead, I found this issue from PIL:

python-pillow/Pillow#8036

Yes, they need to support that to solve this issue.

You beat me to it, but so are you saying that it works with your Ultra HDR images in 10.2.0 and not 10.3.0? Or does it not work with the those images in either?

[liusida]

Its unfortunately going to be a tricky issue to keep up with, and Im sure our triage will break with different versions of PIL, as its not an actual comfy bug, its a work around for a fairly long standing PIL bug. Id recommend using the version that works, and maybe we should looking into including version 10.2.0 in the requirements.txt so long as it doesn't break additional functionality.
Im going to do some more digging, and see if there is a more robust way to solve this, as photos from phones are quite a common input, for myself as well. I take pictures with my phone that I wan't to re-imagine into something cool, so I understand your frustration. Can you by chance provide the make model and software version for the phone you are have errors with?
There is always a small chance that there is a known bug with the phone that is causing errors.

Maybe we need to spec as Pillow!=10.3.0

I'll downgrade my PIL for now. Everything works for 10.2.0. Thanks @shawnington and @ltdrdata .