Skip to content

Conversation

@nhz2
Copy link
Member

@nhz2 nhz2 commented Aug 11, 2025

This PR changes the return type of try_encode!, try_decode!, and try_resize_decode! from Union{Nothing, Int64} to a new MaybeSize type. This new type is a simple wrapper of an Int64 where negative values represent not enough dst space, and positive or zero values represent regular integers.

MaybeSize supports converting to and from Int64 and will throw InexactError if a negative value is converted to an Int64, or a negative Int64 is converted to a MaybeSize.

There is a added constant NOT_SIZE = MaybeSize(typemin(Int64))

Aside from typemin(Int64), the other negative values can be used as a size hint, to improve error messages or performance.

For example:

using ChunkCodecLibLz4
using Chairmarks
u = zeros(UInt8, 2^24);
c = encode(LZ4BlockEncodeOptions(), u);
@b decode(LZ4BlockCodec(), c)

main: 4.907 ms (30 allocs: 36.358 MiB, 8.26% gc time)
PR: 2.359 ms (3 allocs: 16.000 MiB)

@nhz2 nhz2 requested a review from Copilot August 11, 2025 03:51
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR implements a breaking change to the ChunkCodecCore library by introducing a new MaybeSize type to replace Union{Nothing, Int64} return types across all codec interfaces. The change improves error handling by allowing decoders to provide size hints when operations fail.

Key changes:

  • Introduces MaybeSize type that can represent either a valid size (≥0) or failure with optional hints (<0)
  • Updates all codec interfaces to use MaybeSize instead of Union{Nothing, Int64}
  • Modifies error handling to leverage size hints for better debugging

Reviewed Changes

Copilot reviewed 49 out of 49 changed files in this pull request and generated 8 comments.

Show a summary per file
File Description
ChunkCodecCore/src/errors.jl Implements the new MaybeSize type and updates DecodedSizeError
ChunkCodecCore/src/interface.jl Updates core codec interfaces to use MaybeSize
ChunkCodecCore/src/types.jl Updates type documentation for new return types
ChunkCodecCore/src/noop.jl Updates NoOp codec to use MaybeSize
ChunkCodecCore/src/shuffle.jl Updates Shuffle codec to use MaybeSize
Various codec libraries Updates all codec implementations across LibZstd, LibZlib, LibSnappy, LibLz4, LibBzip2, LibBrotli, LibBlosc, LibAec, and Bitshuffle
Test files Updates tests to work with new MaybeSize type
Project.toml files Version bumps and dependency updates for breaking change

@codecov
Copy link

codecov bot commented Aug 11, 2025

Codecov Report

❌ Patch coverage is 98.74214% with 2 lines in your changes missing coverage. Please review.
✅ Project coverage is 95.11%. Comparing base (87f7447) to head (09eaa59).
⚠️ Report is 1 commits behind head on main.

Files with missing lines Patch % Lines
ChunkCodecCore/src/interface.jl 97.14% 1 Missing ⚠️
LibBzip2/src/encode.jl 66.66% 1 Missing ⚠️
Additional details and impacted files
@@             Coverage Diff             @@
##              main      #72      +/-   ##
===========================================
- Coverage   100.00%   95.11%   -4.89%     
===========================================
  Files            2       38      +36     
  Lines          291     1902    +1611     
===========================================
+ Hits           291     1809    +1518     
- Misses           0       93      +93     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@nhz2 nhz2 marked this pull request as ready for review August 26, 2025 03:46
@nhz2 nhz2 requested a review from Copilot August 26, 2025 04:14
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot encountered an error and was unable to review this pull request. You can try again by re-requesting a review.

@nhz2 nhz2 merged commit a56fccb into main Aug 26, 2025
61 of 63 checks passed
@nhz2 nhz2 deleted the nz/return-val branch August 26, 2025 05:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants