Skip to content

Conversation

@github-actions
Copy link
Contributor

@github-actions github-actions bot commented May 9, 2024

Backport of #98278 to release/8.0-staging

/cc @carlossanlop @edwardneal

Customer Impact

  • Customer reported
  • Found internally

Originally reported in dotnet/Open-XML-SDK.

.NET Framework allows consumers of System.IO.Packaging.ZipPackage to use CompressionOption to update bits 1 & 2 in the ZIP archive header's general purpose bitfield. This functionality was lost in .NET Core.

OpenXML is affected because when a document like XSLX is created with our APIs, it is not compliant with the OpenXML specification ISO/IEC 29500-2, as the standard bits 1 and 2 are mandatory and are not reflecting the applied zip compression. Full discussions in #88812 and dotnet/Open-XML-SDK#1443

Specifying the CompressionLevel of System.IO.Compression.ZipArchive will now change these bits, and the ZipArchive's CompressionLevel value will be read from the bitfield.

Regression

  • Yes
  • No

This PR also introduces a breaking change which is reported in detail here: dotnet/docs#40299 . Summarized, the CompressionOption values from System.IO.Packaging will map to different CompressionLevel levels in System.IO.Compression depending if using .NET Framework or .NET Core.

Testing

New tests were added to both System.IO.Compression and System.IO.Packaging to verify that setting the CompressionLevel sets the expected bits, that they can be re-read correctly and that the file data can still be decompressed correctly.

Risk

Low.

Thanks to @edwardneal for fixing the issue and @maedula for the detailed report of this problem.

edwardneal added 6 commits May 9, 2024 23:40
Also changed mapping of ZipPackage's compression options, such that CompressionOption.Maximum now sets the compression level to SmallestSize, and SuperFast now sets the compression level to NoCompression.
Both of these changes restore compatibility with the .NET Framework.
This test was intended to ensure that bit 11 isn't set. It was actually performing a blind comparison of the entire bit field. Other tests in System.IO.Packaging function properly.
* Updated the conditional compilation directives for the .NET Framework/Core package CompressionLevel mappings.
* Specifying a CompressionMethod other than Deflate or Deflate64 will now set the compression level to NoCompression, and will write zeros to the relevant general purpose bits.
* The CompressionLevel always defaulted to Optimal for new archives, but this is now explicitly set (rather than relying upon setting it to null and null-coalescing it to Optimal.) This removes a condition to test for.
* Updated the test data for the CreateArchiveEntriesWithBitFlags test. If the compression level is set to NoCompression, we should expect the CompressionMethod to become Stored (which unsets the general purpose bits, returning an expected result of zero.)
* Updated mapping between general purpose bit fields and CompressionLevel.
* Updated mapping from CompressionOption to CompressionLevel.
* Added test to verify round-trip of CompressionOption and its setting of the general purpose bit fields.
@dotnet-policy-service
Copy link
Contributor

Tagging subscribers to this area: @dotnet/area-system-io-compression
See info in area-owners.md if you want to be subscribed.

@carlossanlop carlossanlop self-assigned this May 10, 2024
@carlossanlop carlossanlop added this to the 8.0.x milestone May 10, 2024
@carlossanlop carlossanlop added the Servicing-consider Issue for next servicing release review label May 29, 2024
@carlossanlop
Copy link
Contributor

@ViktorHofer @ericstj can I get a review and a sign-off for this servicing fix, please?

@carlossanlop carlossanlop removed the Servicing-consider Issue for next servicing release review label May 29, 2024
@ericstj
Copy link
Member

ericstj commented May 30, 2024

As discussed offline I'd like to make sure we have a good case for taking this in servicing and that we understand the potential risk. Can we also talk about why this needs to go to both 8.0 and 6.0?

@edwardneal
Copy link
Contributor

The original request for a backport came from dotnet/Open-XML-SDK#1443. Re-reading the issue, I'm not sure whether the request is for the ability to set the compression level bits, the UTF8 bit or both in the general-purpose bitflags, and I've not personally got a strong opinion whether to backport or not. What's your view @maedula?

@carlossanlop
Copy link
Contributor

Closing while we get clarity on whether this bug's impact is severe enough to justify a backport.

@carlossanlop carlossanlop deleted the backport/pr-98278-to-release/8.0-staging branch May 31, 2024 20:56
@maedula
Copy link

maedula commented Jun 4, 2024

The original request for a backport came from dotnet/Open-XML-SDK#1443. Re-reading the issue, I'm not sure whether the request is for the ability to set the compression level bits, the UTF8 bit or both in the general-purpose bitflags, and I've not personally got a strong opinion whether to backport or not. What's your view @maedula?

@edwardneal The initial issue was about bit 11 (UTF8) being toggled and thus not being spec compliant. I consider this as a no-go and Open-XML-SDK should produce spec compliant results regardless of the .net core version. But I guess I do not understand the concern about the breaking change vs. not being spec compliant in the first place.

I personally consider the compression level bits as a rather cosmetic issue and inconsistent to what the SDK produces using .net framework.

@edwardneal
Copy link
Contributor

That makes perfect sense. It looks like we can leave this PR alone then - the actual PRs we care about are #87883 and #88978, which would need to be backported to .NET 6.0.

@github-actions github-actions bot locked and limited conversation to collaborators Jul 6, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants