Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add JPEG XL Open/Read support via libjxl #7848

Open
wants to merge 29 commits into
base: main
Choose a base branch
from

Conversation

olokelo
Copy link

@olokelo olokelo commented Mar 2, 2024

Helps #4247

This PR enables opening and reading JPEG XL images and animations.
Supported image modes are: RGB, RGBA, RGBa, L, LA, La.
A relatively recent libjxl version is needed to compile Pillow with libjxl support.
The main changes are the addition of _jxl.c and JxlImagePlugin.py.
I'm also the author of jxlpy so this PR was influenced by the work of contributors there. This PR is also largely based on WebPImagePlugin.py which had similar implementation.

Why?
JPEG XL has recently seen increased adoption especially in Apple ecosystem. A lot of users are requesting Pillow support for JPEG XL as their products use Pillow and need to be able to handle jxl files.

I'm open to suggestions and comments. I understand such change would need a lot of testing and probably changes. After all Pillow would need to become somewhat dependent on libjxl. Creating documentation will not be a big problem however I decided to wait for feedback from Pillow core developers.

src/PIL/features.py Outdated Show resolved Hide resolved
Tests/test_jxl_leaks.py Outdated Show resolved Hide resolved
src/PIL/JxlImagePlugin.py Outdated Show resolved Hide resolved
@olokelo olokelo mentioned this pull request Mar 2, 2024
@brunoais
Copy link

brunoais commented Mar 2, 2024

May you please commit the images as LFS?

@olokelo
Copy link
Author

olokelo commented Mar 2, 2024

May you please commit the images as LFS?

Do you mean those .jxl files under Tests/images I've committed? I'm not really sure how to do that.

@brunoais
Copy link

brunoais commented Mar 3, 2024

The explanation is outlined here: https://docs.github.com/en/repositories/working-with-files/managing-large-files/configuring-git-large-file-storage
Nevermind. I just noticed they are not putting the test images as lfs:
image

They already setup lfs and they only use for other larger stuff. Let it stay as you did.

@radarhere radarhere added the JPEG label Mar 11, 2024
@olokelo
Copy link
Author

olokelo commented Mar 11, 2024

Mac OS builds were failing because clang complained about goto labels being declared before variables in scope.
I also merged @radarhere's commits that remove the jxl feature which was causing troubles before.
Now it should be fine. Almost all checks pass except codecov.

elif n_frames > 0:
self.n_frames = n_frames
self._tps_dur_secs = tps_num / tps_denom
# TODO: handle libjxl timecods
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
# TODO: handle libjxl timecods
# TODO: handle libjxl time codes

?

@@ -47,6 +47,7 @@
"IptcImagePlugin",
"JpegImagePlugin",
"Jpeg2KImagePlugin",
"JxlImagePlugin",
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Naming things: like other plugins, shall we use the name of the format rather than the extension?

Suggested change
"JxlImagePlugin",
"JpegXlImagePlugin",

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's a good point. Not sure why I decided to go with "jxl" from the beginning.
Would you want me to change all the references to jxl (including tests, function/class names, feature.jxl) or only plugin and .c extension filename?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think let's just use jxl for the filename extension (so the Tests/images/jxl/) dir is fine too) and of course the MIME type, and change the rest.

Tests/test_file_jxl_metadata.py Outdated Show resolved Hide resolved
Comment on lines 14 to 16
# TODO: lower the limit, I'm not sure what is correct limit
# since I have libjxl debug system-wide
mem_limit = 16 * 1024 # kb
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How can we find out the correct limit?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tested it on release version of libjxl and I've set it to 6 megabytes which seem to work well, that's also similar to memory usage of djxl for decoding hopper.jxl.
I think libjxl memory usage could depend also on number of decoding threads used. By default we use all available cores.

Tests/test_jxl_leaks.py Outdated Show resolved Hide resolved
Tests/test_jxl_leaks.py Outdated Show resolved Hide resolved
src/_jxl.c Outdated Show resolved Hide resolved
src/PIL/JxlImagePlugin.py Outdated Show resolved Hide resolved
src/PIL/JxlImagePlugin.py Outdated Show resolved Hide resolved
# jpeg xl does some weird shenanigans when storing exif
# it omits first 6 bytes of tiff header but adds 4 byte offset instead
if len(exif) <= 4:
return None
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we add a test case for this and _getexit?


def seek(self, frame):
if not self._seek_check(frame):
return
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

And a test for this?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There's already test_file_jxl_animated.py test which also does seeking.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@hugovk
Copy link
Member

hugovk commented Mar 19, 2024

I've merged in main to include #7827 to fix a lot of the warnings I'm seeing on macOS 14.4 Sonoma with CommandLineTools installed.

And this is failing to build for me with:

      src/_jxl.c:174:12: error: incompatible integer to pointer conversion returning 'int' from a function with result type 'void *' [-Wint-conversion]
          return true;
                 ^~~~
      /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang/15.0.0/include/stdbool.h:21:14: note: expanded from macro 'true'
      #define true 1
                   ^

Full log: log2.txt

@olokelo
Copy link
Author

olokelo commented Mar 19, 2024

I've merged in main to include #7827 to fix a lot of the warnings I'm seeing on macOS 14.4 Sonoma with CommandLineTools installed.

And this is failing to build for me with:

      src/_jxl.c:174:12: error: incompatible integer to pointer conversion returning 'int' from a function with result type 'void *' [-Wint-conversion]
          return true;
                 ^~~~
      /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang/15.0.0/include/stdbool.h:21:14: note: expanded from macro 'true'
      #define true 1
                   ^

Full log: log2.txt

Thanks for reporting. The _jxl_decoder_count_frames function was declared to return void * instead of bool. That was an obvious mistake and I'm even not sure why it worked under gcc.

src/PIL/JxlImagePlugin.py Outdated Show resolved Hide resolved
Comment on lines 30 to 31
if is_jxl and not SUPPORTED:
return "image file could not be identified because JXL support not installed"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Exception instead of return string?

Copy link
Author

@olokelo olokelo Mar 19, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we raise SyntaxError like here in Jpeg2K plugin?
I based my work on WebP plugin which just returns string.

@develar
Copy link

develar commented Apr 23, 2024

It would be excellent to see some progress in this area. I am currently working on introducing JPEG XL support to https://immich.app for previews and thumbnails (as Python is used for ML). However, an additional plugin pillow-jpegxl-plugin complicates the deployment process.

@bgorissen
Copy link

Could you still add support for 16 bit single channel files? Such images are commonly used in fluorescent microscopy, and storing them in jxl instead of tiff or png would lead to substantial storage savings. Pillow is used by various relevant software including imageio and CellProfiler.

In _pil_jxl_get_mode you would have to add:

    if(bi->bits_per_sample == 16 && bi->num_color_channels == 1 && bi->alpha_bits == 0 && !bi->alpha_premultiplied) {
        return "I;16";
    }

and in _jxl_decoder_new, the line

if (decp->pixel_format.data_type != JXL_TYPE_UINT8) {

should be changed into:

if (decp->pixel_format.data_type != JXL_TYPE_UINT8 && decp->pixel_format.data_type != JXL_TYPE_UINT16) {

@aclark4life
Copy link
Member

Any relation-to or overlap-with #1888 here? 🤔

@bgorissen
Copy link

bgorissen commented Apr 24, 2024

Any relation-to or overlap-with #1888 here? 🤔

Not quite, fluorescent microscopy images are often stored in a single channel while #1888 is about multichannel files. Pillow already supports single channel images (as #1888 points out), so it makes sense that this JPEG XL reader supports them too.

@olokelo
Copy link
Author

olokelo commented Apr 27, 2024

It would be excellent to see some progress in this area. I am currently working on introducing JPEG XL support to https://immich.app for previews and thumbnails (as Python is used for ML). However, an additional plugin pillow-jpegxl-plugin complicates the deployment process.

@develar Sounds awesome, thank you for working on JPEG XL support in Immich. I presume that merging this PR is now up to Pillow core developers since all checks have passed. However by now we would probably need to resolve some merge conflicts.

Could you still add support for 16 bit single channel files? Such images are commonly used in fluorescent microscopy, and storing them in jxl instead of tiff or png would lead to substantial storage savings.

@bgorissen Yes, of course it's possible. Thank you for detailed description of changes. Could you provide some image(s) to test this mode?

@j99ca
Copy link

j99ca commented May 7, 2024

@olokelo I 2nd @bgorissen request around 16 bit single channel support. I work with imaging systems that produce 16 bit single channel images, like thermal and depth, and the JXL format provides some significant cost savings storage-wise compared to our 16 bit single channel PNG format we are currently using. I would love to see the JXL ecosystem mature and this plugin would go a long way towards our adoption of the format

@bgorissen
Copy link

bgorissen commented May 8, 2024

@olokelo I've attached a 16-bits grayscale jxl file (zipped because Github does not support jxl). You could crop it to create a smaller test file. The file was converted from a public tiff image from the dataset cpg0011 (Laber et al., 2023), available from the Cell Painting Gallery on the Registry of Open Data on AWS (https://registry.opendata.aws/cellpainting-gallery/), and released under CC0 1.0 Universal (which puts it in the public domain).

subcutaneous__D14__45adb24f-23b7-43f8-9612-0a7ecaa22ada__BR00101077__r02c03f01p01-ch1sk1fk1fl1.zip

@j99ca
Copy link

j99ca commented May 24, 2024

This is very good work @olokelo! Is there any plan to add support for writing to JXL from Pillow?

@olokelo
Copy link
Author

olokelo commented May 24, 2024

@j99ca Yes of course, I just wanted to add reading support first to see what Pillow maintainers think about it.
For now you can create JXL images using jxlpy.

@cmahnke
Copy link

cmahnke commented Aug 8, 2024

@hugovk: It certainly would be nice to have this as feature in Pillow 11, wouldn't it?
Maybe you can sketch out the required steps?

@bigcat88
Copy link
Contributor

The iPhone 16 Pro models will support the JPEG-XL file format, according to code found in iOS 18.

I really hope this will be the next standard in photography, I would really like to see support for this in Pillow 11 by default.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

10 participants