Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add HTJ2K Compressor #1883

Draft
wants to merge 11 commits into
base: main
Choose a base branch
from

Conversation

palemieux
Copy link

WIP DO NOT MERGE

This patch fulfills my action item from the September 19 TSC meeting.

Introduction

This patch proposes to add support for High-Throughput JPEG 2000 (HTJ2K) compression to OpenEXR -- HTJ2K is JPEG 2000 with the HT block coder standardized in Rec. ITU-T T.814 | ISO/IEC 15444-15. As detailed at Evaluating HTJ2K as a compressor option for OpenEXR, HTJ2K demonstrates significant improvements in both speed and file size over other OpenEXR compressors. Furthermore, HTJ2K is a worldwide standard that benefits from a diversity of implementations, builds on JPEG 2000, which is in broad use in studio workflows, is appropriate for long-term preservation, has both lossy and lossless modes and supports integer and floating point samples. Finally, the unique features of HTJ2K could be powerful additions to high-performance OpenEXR workflows where the full image may not always be viewed at its full resolution.

The patch currently defines four compressors:

  • HT: Lossless coding using HTJ2K. Each chunk is the full image frame and the open-source OpenJPH library is used.
  • HT256: Lossless coding using HTJ2K. Each chunk is 256 lines and the open-source OpenJPH library is used.
  • HTK: Lossless coding using HTJ2K. Each chunk is the full image frame and the commercial KDU library is used.
  • HTK256: Lossless coding using HTJ2K. Each chunk is 256 lines and the commercial KDU library is used.

Questions

  • can a compressor store configuration information once in a file, so that the same information is available for each chunk to both the compressor and decompressor?

Notes

  • HT and HTK Compressors are identical and use the open-source OpenJPH library is the Kakadu SDK is not found
  • Ultimately there will be a single lossless HTJ2K Compressor. The HT and HTK Compressors are currently offered to ensure interop, and facilitate comparisons, between OSS and commercial options.
  • The full-frame and 256-line chunk options are currently offered to evaluate the impact of sub-frame chunking on coding efficiency performance.

Todo

  • add support for 32-bit samples to OpenJPH
  • explore adding a progressive decoding API to OpenEXR
  • explore tradeoff of chunk-size (currently set to 256) on compression efficiency, single-threaded/multi-threaded throughput and partial decoding (low-resolution and/or region of interest)
    • can chunking can be replaced with full-frame coding and J2K accessibility features? This is more important in lossy coding to avoid edge effects.
    • select a chunk size, i.e., is 256-line a reasonable value?
    • can J2K tiles be used instead of independent codestreams to remove JPEG 2000 codestream header overhead?

Copy link

linux-foundation-easycla bot commented Oct 13, 2024

CLA Not Signed


// make image filename
char input_filename[512] = { '\0' };
sprintf(input_filename, args->input_filename, frame_index + args->start_frame);

Check failure

Code scanning / CodeQL

Uncontrolled format string Critical

The value of this argument may come from
a command-line argument
and is being used as a formatting argument to sprintf(__fmt).
The value of this argument may come from
a command-line argument
and is being used as a formatting argument to sprintf(__fmt), which calls __builtin___sprintf_chk((unnamed parameter 3)).
// make image filenames
char input_filename[512] = { '\0' };
char output_filename[512] = { '\0' };
sprintf(input_filename, args.input_filename, frame_index + args.start_frame);

Check failure

Code scanning / CodeQL

Uncontrolled format string Critical

The value of this argument may come from
a command-line argument
and is being used as a formatting argument to sprintf(__fmt).
The value of this argument may come from
a command-line argument
and is being used as a formatting argument to sprintf(__fmt), which calls __builtin___sprintf_chk((unnamed parameter 3)).
char input_filename[512] = { '\0' };
char output_filename[512] = { '\0' };
sprintf(input_filename, args.input_filename, frame_index + args.start_frame);
sprintf(output_filename, args.output_filename, frame_index + args.start_frame);

Check failure

Code scanning / CodeQL

Uncontrolled format string Critical

The value of this argument may come from
a command-line argument
and is being used as a formatting argument to sprintf(__fmt).
The value of this argument may come from
a command-line argument
and is being used as a formatting argument to sprintf(__fmt), which calls __builtin___sprintf_chk((unnamed parameter 3)).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants