Skip to content

Conversation

@edwardneal
Copy link
Contributor

Bugfix for the outer loop tests which failed after PR #103153.

The original Zip64ExtraField.WriteBlock method contained logic to conditionally write the compressed size, uncompressed size, local header offset and starting disk number to the output stream. When this was modified to use BinaryPrimitives, (link) this condition wasn't ported across properly - it conditionally wrote into the destination buffer, but used constant field offsets. There was also one error in ZipArchiveEntry.WriteDataDescriptor - that's now corrected.

I don't see this pattern repeated anywhere else in ZipBlocks or ZipArchiveEntry. ZipArchiveEntry does sometimes write a ZIP64 data descriptor (link) but this logic is reasonable.

Outer loop tests which failed as a result of this bug:

  • System.IO.Compression.Tests.zip_LargeFiles.CheckZIP64VersionIsSet_ForSmallFilesAfterBigFiles
  • System.IO.Compression.Tests.zip_LargeFiles.UnzipOver4GBZipFile
  • System.IO.Packaging.Tests.LargeFilesTests.CheckZIP64VersionIsSet_ForSmallFilesAfterBigFiles
    These were the only tests which failed because they're the only ones which generated large enough files to trigger the logic which wrote Zip64ExtraField. They're now passing locally.

Apologies again for missing this.

@carlossanlop - if you're happy with the diff, could you run the outer loop tests against this PR again please?

@dotnet-policy-service dotnet-policy-service bot added the community-contribution Indicates that the PR has been added by a community member label Jan 24, 2025
@dotnet-policy-service
Copy link
Contributor

Tagging subscribers to this area: @dotnet/area-system-io-compression
See info in area-owners.md if you want to be subscribed.

@carlossanlop

This comment was marked as resolved.

@azure-pipelines

This comment was marked as resolved.

@carlossanlop

This comment was marked as resolved.

@azure-pipelines

This comment was marked as resolved.

@carlossanlop

This comment was marked as resolved.

@azure-pipelines

This comment was marked as resolved.

@carlossanlop
Copy link
Contributor

/azp list

@azure-pipelines
Copy link

CI/CD Pipelines for this repository:

@carlossanlop
Copy link
Contributor

/azp run runtime-libraries-coreclr outerloop

@azure-pipelines
Copy link

Azure Pipelines successfully started running 1 pipeline(s).

Copy link
Contributor

@carlossanlop carlossanlop left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the quick fix submission. LGTM. Let's see what outerloop says.

@carlossanlop
Copy link
Contributor

  • These were the only tests which failed because they're the only ones which generated large enough files to trigger the logic which wrote Zip64ExtraField. They're now passing locally.

Just double checking: The outerloop tests get skipped by default AFAIK even when you run them locally, unless you pass the necessary msbuild global property in the command (if from CLI). Did you do this by any chance?

I think such suppression does not happen if you run the unit test from VS or VS Code instead. You can choose the specific test and it gets executed.

@edwardneal
Copy link
Contributor Author

The VS test runner skips outerloop tests, even if I specifically select them. Those tests remain in the "not executed" state, but when that state rolls up to the parent, it looks like the entire suite passed...

It's an odd problem, but I've brute-forced it for the moment - I just locally commented out the OuterLoop attributes and ran the entire test suite for System.IO.Compression and System.IO.Packaging, specifically confirming that the previously-failing tests pass.

@carlossanlop
Copy link
Contributor

Thanks for checking! So far the results don't show compression-related failures. I'm monitoring the remaining ones too.

@carlossanlop
Copy link
Contributor

  • runtime (Build browser-wasm linux Release LibraryTests) was cancelled already, but showing up as stuck.
  • I opened issues for all the unrelated outerloop failures. Didn't find any Zip related errors.

Merging now!

grendello added a commit to grendello/runtime that referenced this pull request Jan 27, 2025
* main: (22 commits)
  Clean up Stopwatch a bit (dotnet#111834)
  JIT: Fix embedded broadcast simd size (dotnet#111638)
  Revert potential UB due to aliasing + more WB removals (dotnet#111733)
  re-enable acceleration of Vector512<long>.op_Multiply (dotnet#111832)
  Handle OSSL 3.4 change to SAN:othername formatting
  JIT: Fix stack allocated arrays for NativeAOT (dotnet#111827)
  JIT: enhance RBO inference for similar compares to constants (dotnet#111766)
  JIT: Don't run optSetBlockWeights when we have PGO data (dotnet#111764)
  [Android] Make sure RuntimeFlavor=CoreCLR when clr subset is specified (dotnet#111821)
  Change empty subject test certificate to include a critical SAN.
  Fix reversed code offsets in GcInfo (dotnet#111792)
  Swap some libraries areas between leads (dotnet#111816)
  Add left-handed spherical and cylindrical billboards (dotnet#109605)
  JIT: revise `optRelopImpliesRelop` to always set `reverseSense` (dotnet#111803)
  Fix Zip64ExtraField handling (dotnet#111802)
  Add build support for Android+CoreCLR (dotnet#110471)
  arm64: Add bic(s) compact encoding (dotnet#111452)
  JIT: Ensure `BBF_PROF_WEIGHT` flag is set when we have PGO data (dotnet#111780)
  Add support for AVX10.2, Add AVX10.2 API surface and template tests (dotnet#111209)
  JIT: Preliminary for enabling inlining late devirted calls (dotnet#111782)
  ...
@carlossanlop carlossanlop mentioned this pull request Feb 13, 2025
15 tasks
@github-actions github-actions bot locked and limited conversation to collaborators Feb 24, 2025
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

area-System.IO.Compression community-contribution Indicates that the PR has been added by a community member

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants