Skip to content

feat: JPEG 2000 codec and PDF JPXDecode support - #45

Merged
Malcolmnixon merged 46 commits into
mainfrom
feature/jpeg2000-codec
Oct 10, 2026
Merged

Malcolmnixon merged 46 commits into
mainfrom
feature/jpeg2000-codec

Conversation

@Malcolmnixon

@Malcolmnixon Malcolmnixon commented Oct 10, 2026 •

Copy link
Copy Markdown
Member

Pull Request

Description

Adds a dependency-free, decode-only JPEG 2000 codec (Jpeg2000Codec) and wires it into the PDF renderer so the /JPXDecode image filter works.

  • Codec: ISO 15444-1 Part 1 — JP2 boxes (ihdr, bpcc, colr, pclr/cmap, cdef) and raw codestreams; tiles/tile-parts; all five progression orders with POC; precincts; code-block styles; EBCOT tier-1 with the MQ decoder; tier-2 packets with tag trees; 5/3 and 9/7 inverse DWT; RCT/ICT; subsampling; ROI max-shift; 1–16 bit and signed samples (output scaled to 8 bits). Part-2/HTJ2K and >4 colour channels fail closed with UnsupportedImageFeatureException; malformed input throws InvalidDataException; allocations and work are capped.
  • PDF: JPX images use the JP2 colour space unless /ColorSpace is given; /SMaskInData, /Decode, and JPX as an /SMask. Also adds a general /SMask path (previously unsupported for raw/DCT images).
  • Tests/docs: test-only encoder, codec, fixture (ImageMagick-generated) and PDF tests, plus hostile-input tests; decode-wide CPU budget and public Jpeg2000DecoderLimits; raw 1/2/4/8/16-bit sample support; SysML2, reqstream, design, verification, reviewmark, README/user guide updated.

Known limitations: ICC profiles reported but not applied; naive CMYK; /Mask//Matte unsupported; JPX must be the sole filter; no inline images; truncated tile data is rejected. Encoder and decoder share an author, so 5/3, RCT, bypass and ROI paths are verified only against the test encoder (MQ verified against the T.88 vector; 9/7 against real files, not committed).

Type of Change

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • Documentation update
  • Code quality improvement

Related Issues

Closes #

Pre-Submission Checklist

Build and Test

  • Code builds successfully and all tests pass: pwsh ./build.ps1 (12,552 tests, 0 failed)
  • Code produces zero warnings

Code Quality

  • New code has appropriate XML documentation comments
  • Static analyzer warnings have been addressed

Quality Checks

  • All linters pass: pwsh ./lint.ps1 (no errors)

Testing

  • Added unit tests for new functionality
  • Updated existing tests if behavior changed
  • All tests follow the AAA (Arrange, Act, Assert) pattern
  • Test coverage is maintained or improved

Documentation

  • Updated README.md (if applicable)
  • Updated docs/ documentation (if applicable)
  • Added code examples for new features (if applicable)
  • Updated requirements.yaml (if applicable)

Additional Notes

Real-world smoke check against a local third-party PDF with 7 JPX streams (not committed) decoded correctly, and a mutation fuzz run produced only the two allowed exception types.

Malcolm Nixon and others added 10 commits October 9, 2026 23:03
…et decoding

Work packages 1-2: markers, tile-parts, PPM/PPT, geometry, tag trees, all progression orders, POC, SOP/EPH.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…rse DWT

Work packages 3-4: MQ and raw decoders, all code-block styles, ROI maxshift, 5/3 and 9/7 inverse wavelets.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…00Codec API

Work package 5: RCT/ICT, subsampling, bit-depth scaling, colr/pclr/cmap/cdef, Load/GetInfo/Decode, Jpeg2000Image.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Tiles lost main-header POC entries; a tile-part POC now replaces the inherited list.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Includes T.88 MQ vector, round trips for all Part 1 features, JP2 boxes and hostile-input tests.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…rted-filter PDF test to JBIG2

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…ion and user docs

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot AI lite review requested due to automatic review settings October 10, 2026 03:59

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Unresolved codec compilation and decoding correctness issues remain.

9 open findings
What changed in this PR

Adds a dependency-free JPEG 2000 decoder and PDF /JPXDecode/soft-mask support.

Changes:

  • Added JP2 and raw codestream decoding.
  • Integrated JPEG 2000 images and masks into PDF rendering.
  • Added tests, documentation, requirements, and traceability updates.
File Description
test/​DemaConsulting.CanvasNet.Tests/​Codecs/​Jpeg2000TestEncoder.Markers.cs Test encoder marker and JP2 support
test/​DemaConsulting.CanvasNet.Pdf.Tests/​PdfDocumentTests.cs Updated filter coverage
test/​DemaConsulting.CanvasNet.Pdf.Tests/​PdfDocumentJpxTests.cs JPX and mask tests
test/​DemaConsulting.CanvasNet.Pdf.Tests/​DemaConsulting.CanvasNet.Pdf.Tests.csproj Linked test encoder sources
src/​DemaConsulting.CanvasNet/​Codecs/​NamespaceDoc.cs Codec documentation
src/​DemaConsulting.CanvasNet/​Codecs/​Jpeg2000Image.cs Decoded image model
src/​DemaConsulting.CanvasNet/​Codecs/​Jpeg2000ColorSpace.cs Color-space definitions
src/​DemaConsulting.CanvasNet/​Codecs/​Jpeg2000Codec.cs Public codec API
src/​DemaConsulting.CanvasNet/​Codecs/​Jpeg2000/​Jpeg2000Codec.Tier1.cs EBCOT tier-1 decoding
src/​DemaConsulting.CanvasNet/​Codecs/​Jpeg2000/​Jpeg2000Codec.Packets.cs Tier-2 packet decoding
src/​DemaConsulting.CanvasNet/​Codecs/​Jpeg2000/​Jpeg2000Codec.Mq.cs MQ arithmetic decoding
src/​DemaConsulting.CanvasNet/​Codecs/​Jpeg2000/​Jpeg2000Codec.Jp2.cs JP2 box and channel parsing
src/​DemaConsulting.CanvasNet/​Codecs/​Jpeg2000/​Jpeg2000Codec.Geometry.cs Tile and code-block geometry
src/​DemaConsulting.CanvasNet/​Codecs/​Jpeg2000/​Jpeg2000Codec.Dwt.cs Inverse wavelet transforms
src/​DemaConsulting.CanvasNet/​Codecs/​Jpeg2000/​Jpeg2000Codec.Decoder.cs Decode orchestration
src/​DemaConsulting.CanvasNet.Pdf/​PdfDocument.Images.cs JPX and soft-mask rendering
src/​DemaConsulting.CanvasNet.Pdf/​PdfDocument.Filters.cs Filter documentation
src/​DemaConsulting.CanvasNet.Pdf/​PdfDocument.cs PDF feature documentation
src/​DemaConsulting.CanvasNet.Pdf/​NamespaceDoc.cs PDF namespace documentation
requirements.yaml JPEG 2000 requirements
README.md Feature documentation
docs/​verification/​canvas-net/​codecs/​jpeg2000-codec.md Codec verification design
docs/​verification/​canvas-net/​codecs.md Codec verification overview
docs/​verification/​canvas-net.md System verification
docs/​verification/​canvas-net-pdf/​pdf-document.md PDF verification
docs/​user_guide/​introduction.md Public API documentation
docs/​sysml2/​model/​canvas-net/​codecs/​jpeg2000-codec.sysml JPEG 2000 model
docs/​sysml2/​model/​canvas-net/​codecs.sysml Codec subsystem model
docs/​reqstream/​canvas-net/​codecs/​jpeg2000-codec.yaml Codec requirements
docs/​reqstream/​canvas-net/​codecs.yaml Codec subsystem requirements
docs/​reqstream/​canvas-net.yaml Library requirements
docs/​reqstream/​canvas-net-pdf/​pdf-document.yaml PDF requirements
docs/​reqstream/​canvas-net-pdf.yaml PDF traceability
docs/​design/​introduction.md Design overview
docs/​design/​canvas-net/​codecs/​jpeg2000-codec.md JPEG 2000 design
docs/​design/​canvas-net/​codecs.md Codec subsystem design
docs/​design/​canvas-net.md System design
docs/​design/​canvas-net-pdf/​pdf-document.md PDF image design
docs/​design/​canvas-net-pdf.md PDF subsystem design
.reviewmark.yaml Review coverage configuration
.cspell.yaml JPEG 2000 terminology

🧠 Review effort: Lite


💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread src/DemaConsulting.CanvasNet.Pdf/PdfDocument.Images.cs Outdated
Comment thread src/DemaConsulting.CanvasNet.Pdf/PdfDocument.Images.cs Outdated
Comment thread docs/design/canvas-net/codecs/jpeg2000-codec.md Outdated
Comment thread docs/verification/canvas-net-pdf/pdf-document.md Outdated
Comment thread docs/verification/canvas-net/codecs.md Outdated
Add ImageMagick/OpenJPEG-generated fixtures from synthetic sources, exact-match
decode tests against PNG sources, a PDF JPXDecode test and a system integration
test, with requirement and verification links.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot AI lite review requested due to automatic review settings October 10, 2026 13:05
@Malcolmnixon

Copy link
Copy Markdown
Member Author

Added independent real-encoder evidence (07ede83): fixtures generated with ImageMagick 7.1.2/OpenJPEG from synthetic images (no third-party content), decoded pixel-exact against the source (5/3+RCT, all five progression orders, layers, gray, alpha, 16-bit, raw codestream), plus a system-integration and a PDF-level test. Also verified locally (not committed, licenses unclear) against ~38 jpylyzer-test-files: pixel-exact vs OpenJPEG on Kakadu/OpenJPEG/GraphicsMagick/ERDAS/Sentinel files, +/-1-3 levels on sYCC files; everything OpenJPEG rejects we reject. Not independently covered: 9/7 from committed fixtures, tiles, precincts, PPM/PPT (test encoder only).

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Comment thread src/DemaConsulting.CanvasNet.Pdf/PdfDocument.Images.cs Outdated
Comment thread src/DemaConsulting.CanvasNet/Codecs/Jpeg2000/Jpeg2000Codec.Jp2.cs Outdated
Comment thread src/DemaConsulting.CanvasNet/Codecs/Jpeg2000Codec.cs
Comment thread test/DemaConsulting.CanvasNet.Tests/Codecs/Jpeg2000CodecTests.cs
Malcolm Nixon and others added 12 commits October 10, 2026 09:18
…; fix doc counts

Expose Jpeg2000Image.BitDepth and HasPalette; PdfDocument rejects an
Indexed /ColorSpace over non-8-bit samples and any /ColorSpace override
on JP2 data with its own palette. Correct ordinal, test count and
codecs verification list.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…validation

- Add Jpeg2000DecoderLimits enforced during header validation before allocation
- Single decode-wide progression budget, POC entry cap and covered-volume skipping
- Reject codestreams with missing tiles; validate bit-plane counts and segmentation symbols
- Treat four channels as CMYK only when CMYK or unspecified
- Narrow the Guard backstop; tests assert specific exception types

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…s and limits to review set

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…nData order

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…ket caps

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…et, coverage, TPsot tests

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
… names, user guide limits

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…es, tests

Unpack sub-byte and 16-bit samples through the shared sample path so soft masks of
any depth work, apply /Decode exactly to 1- and 3-channel JPEGs, and add CCITT and
Indexed /Decode regression tests.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…t, document budget calibration

The per-input-byte term already bounds hostile input; the absolute ceiling
only needs to exceed the work of any image the default sample limit admits.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot AI lite review requested due to automatic review settings October 10, 2026 18:42

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Unresolved critical allocation/probing issues, moderate decoder and PDF correctness/resource-limit issues, and a documentation nit remain.

2 open findings
Previously missed (1)

In code that hasn't changed since last review

Medium severity Direct component mappings are skipped without a palette

src/​DemaConsulting.CanvasNet/​Codecs/​Jpeg2000/​Jpeg2000Codec.Jp2.cs:490

cmap is not limited to palette images: a legal mapping with Type == 0 directly selects or reorders codestream components even when there is no pclr box. This early return ignores every mapping whenever the palette is absent, so Decode, Load, and GetInfo can report/render channels in the wrong order for direct component mappings. Apply the direct mappings (and reject a type-1 mapping without a palette) instead of bypassing the box.

🧠 Review effort: Lite

Comment thread src/DemaConsulting.CanvasNet/Codecs/Jpeg2000/Jpeg2000Codec.Jp2.cs Outdated
Comment thread src/DemaConsulting.CanvasNet/Codecs/Jpeg2000/Jpeg2000Codec.Probe.cs Outdated
GetInfo walks jp2h children incrementally instead of buffering the superbox;
the ICC colr payload is kept as a slice and copied once when the image is built.
Pin cmap-without-pclr being ignored.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot AI lite review requested due to automatic review settings October 10, 2026 19:02

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

Moderate findings remain in PDF image handling and JP2 channel and alpha processing.

0 open findings

2 resolved since last review

🧠 Review effort: Lite

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot AI lite review requested due to automatic review settings October 10, 2026 19:12

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Unresolved critical and moderate findings affect malformed-input validation, PDF transparency correctness, and resource bounds.

2 open findings

🧠 Review effort: Lite

Comment thread src/DemaConsulting.CanvasNet/Codecs/Jpeg2000/Jpeg2000Codec.Jp2.cs
Malcolm Nixon and others added 3 commits October 10, 2026 15:52
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…es; document SOP leniency

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot AI lite review requested due to automatic review settings October 10, 2026 19:55

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Unresolved moderate findings remain in PDF image handling and JPEG 2000 decoding validation.

3 open findings
2 resolved since last review

🧠 Review effort: Lite

Comment thread docs/design/canvas-net/codecs/jpeg2000-codec.md Outdated
Comment thread docs/reqstream/canvas-net/codecs/jpeg2000-codec.yaml Outdated
Comment thread docs/verification/canvas-net/codecs/jpeg2000-codec.md Outdated
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot AI lite review requested due to automatic review settings October 10, 2026 20:07

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

Six unresolved findings remain, including five moderate issues affecting limits, validation, color handling, alpha conversion, and tile coverage.

0 open findings

3 resolved since last review
Previously missed (1)

In code that hasn't changed since last review

Low severity Correct JPX dimension-limit security boundary documentation

src/​DemaConsulting.CanvasNet.Pdf/​PdfDocument.Images.cs:509

These remarks say the JPX decode is bounded by the PDF dictionary's /Width and /Height, but the implementation deliberately does not use those entries as limits; JpxDecoderLimits only sets Surface.MaxDimension. This misstates the security boundary and should describe the renderer-wide dimension limits instead.

🧠 Review effort: Lite

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot AI lite review requested due to automatic review settings October 10, 2026 20:20

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Unresolved critical and moderate findings affect malformed-input handling, resource limits, PDF dimensions, and premultiplied color processing.

2 open findings

🧠 Review effort: Lite

Comment thread src/DemaConsulting.CanvasNet/Codecs/Jpeg2000/Jpeg2000Codec.Tier1.cs
@Malcolmnixon
Malcolmnixon merged commit 46c832d into main Oct 10, 2026
7 checks passed
@Malcolmnixon
Malcolmnixon deleted the feature/jpeg2000-codec branch October 10, 2026 22:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants