Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Release][Packaging] Wheel content script does not work with local wheel verification #44459

Closed
raulcd opened this issue Oct 17, 2024 · 2 comments
Closed

[Release][Packaging] Wheel content script does not work with local wheel verification #44459

raulcd opened this issue Oct 17, 2024 · 2 comments
Assignees
@raulcd
Labels
backport-candidate Component: Packaging Component: Release Type: bug
Milestone
19.0.0

Comments

@raulcd
Copy link
Member

raulcd commented Oct 17, 2024

Describe the bug, including details regarding any error messages, version, and platform.

We added content verification for wheels for this release:

This works for individual wheel verification bus it currently failing when verifying all the wheels for the release.

The error:

+ python /tmp/arrow-18.0.0.UeQlC/apache-arrow-18.0.0/ci/scripts/python_wheel_validate_contents.py --path /tmp/arrow-18.0.0.UeQlC/apache-arrow-18.0.0/python/repaired_wheels
Traceback (most recent call last):
  File "/tmp/arrow-18.0.0.UeQlC/apache-arrow-18.0.0/ci/scripts/python_wheel_validate_contents.py", line 48, in <module>
    main()
  File "/tmp/arrow-18.0.0.UeQlC/apache-arrow-18.0.0/ci/scripts/python_wheel_validate_contents.py", line 44, in main
    validate_wheel(args.path)
  File "/tmp/arrow-18.0.0.UeQlC/apache-arrow-18.0.0/ci/scripts/python_wheel_validate_contents.py", line 28, in validate_wheel
    assert len(wheels) == 1, error_msg
           ^^^^^^^^^^^^^^^^
AssertionError: 0 wheels found but only 1 expected ([])
Failed to verify release candidate. See /tmp/arrow-18.0.0.UeQlC for details.
Error: `docker compose --file=/home/runner/work/crossbow/crossbow/arrow/docker-compose.yml run --rm -e VERIFY_VERSION=18.0.0 -e VERIFY_RC=0 -e TEST_DEFAULT=0 -e TEST_WHEELS=1 almalinux-verify-rc` exited with a non-zero exit code 1, see the process log above.

The docker-compose command was invoked with the following parameters:

Component(s)

Packaging, Release
@raulcd
Copy link
Member Author

raulcd commented Oct 17, 2024

We could try to validate the binary wheels modifying the folder to check the downloaded folder but we would have to also modify the script because we download all the wheels at once:

+ python /tmp/arrow-18.0.0.pfUwd/apache-arrow-18.0.0/ci/scripts/python_wheel_validate_contents.py --path /tmp/arrow-18.0.0.pfUwd/binaries/python-rc/18.0.0-rc0
Traceback (most recent call last):
  File "/tmp/arrow-18.0.0.pfUwd/apache-arrow-18.0.0/ci/scripts/python_wheel_validate_contents.py", line 48, in <module>
    main()
  File "/tmp/arrow-18.0.0.pfUwd/apache-arrow-18.0.0/ci/scripts/python_wheel_validate_contents.py", line 44, in main
    validate_wheel(args.path)
  File "/tmp/arrow-18.0.0.pfUwd/apache-arrow-18.0.0/ci/scripts/python_wheel_validate_contents.py", line 28, in validate_wheel
    assert len(wheels) == 1, error_msg
           ^^^^^^^^^^^^^^^^
AssertionError: 24 wheels found but only 1 expected ([PosixPath('/tmp/arrow-18.0.0.pfUwd/binaries/python-rc/18.0.0-rc0/pyarrow-18.0.0-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl'), PosixPath('/tmp/arrow-18.0.0.pfUwd/binaries/python-rc/18.0.0-rc0/pyarrow-18.0.0-cp313-cp313t-manylinux_2_17_x86_64.manylinux2014_x86_64.whl'), PosixPath('/tmp/arrow-18.0.0.pfUwd/binaries/python-rc/18.0.0-rc0/pyarrow-18.0.0-cp313-cp313-manylinux_2_28_aarch64.whl'), PosixPath('/tmp/arrow-18.0.0.pfUwd/binaries/python-rc/18.0.0-rc0/pyarrow-18.0.0-cp312-cp312-manylinux_2_17_aarch64.manylinux2014_aarch64.whl'), PosixPath('/tmp/arrow-18.0.0.pfUwd/binaries/python-rc/18.0.0-rc0/pyarrow-18.0.0-cp312-cp312-manylinux_2_28_x86_64.whl'), PosixPath('/tmp/arrow-18.0.0.pfUwd/binaries/python-rc/18.0.0-rc0/pyarrow-18.0.0-cp313-cp313t-manylinux_2_17_aarch64.manylinux2014_aarch64.whl'), PosixPath('/tmp/arrow-18.0.0.pfUwd/binaries/python-rc/18.0.0-rc0/pyarrow-18.0.0-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl'), PosixPath('/tmp/arrow-18.0.0.pfUwd/binaries/python-rc/18.0.0-rc0/pyarrow-18.0.0-cp39-cp39-manylinux_2_28_x86_64.whl'), PosixPath('/tmp/arrow-18.0.0.pfUwd/binaries/python-rc/18.0.0-rc0/pyarrow-18.0.0-cp311-cp311-manylinux_2_28_x86_64.whl'), PosixPath('/tmp/arrow-18.0.0.pfUwd/binaries/python-rc/18.0.0-rc0/pyarrow-18.0.0-cp313-cp313t-manylinux_2_28_aarch64.whl'), PosixPath('/tmp/arrow-18.0.0.pfUwd/binaries/python-rc/18.0.0-rc0/pyarrow-18.0.0-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl'), PosixPath('/tmp/arrow-18.0.0.pfUwd/binaries/python-rc/18.0.0-rc0/pyarrow-18.0.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl'), PosixPath('/tmp/arrow-18.0.0.pfUwd/binaries/python-rc/18.0.0-rc0/pyarrow-18.0.0-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl'), PosixPath('/tmp/arrow-18.0.0.pfUwd/binaries/python-rc/18.0.0-rc0/pyarrow-18.0.0-cp313-cp313-manylinux_2_17_aarch64.manylinux2014_aarch64.whl'), PosixPath('/tmp/arrow-18.0.0.pfUwd/binaries/python-rc/18.0.0-rc0/pyarrow-18.0.0-cp311-cp311-manylinux_2_28_aarch64.whl'), PosixPath('/tmp/arrow-18.0.0.pfUwd/binaries/python-rc/18.0.0-rc0/pyarrow-18.0.0-cp312-cp312-manylinux_2_28_aarch64.whl'), PosixPath('/tmp/arrow-18.0.0.pfUwd/binaries/python-rc/18.0.0-rc0/pyarrow-18.0.0-cp311-cp311-manylinux_2_17_aarch64.manylinux2014_aarch64.whl'), PosixPath('/tmp/arrow-18.0.0.pfUwd/binaries/python-rc/18.0.0-rc0/pyarrow-18.0.0-cp313-cp313-manylinux_2_17_x86_64.manylinux2014_x86_64.whl'), PosixPath('/tmp/arrow-18.0.0.pfUwd/binaries/python-rc/18.0.0-rc0/pyarrow-18.0.0-cp310-cp310-manylinux_2_28_aarch64.whl'), PosixPath('/tmp/arrow-18.0.0.pfUwd/binaries/python-rc/18.0.0-rc0/pyarrow-18.0.0-cp313-cp313-manylinux_2_28_x86_64.whl'), PosixPath('/tmp/arrow-18.0.0.pfUwd/binaries/python-rc/18.0.0-rc0/pyarrow-18.0.0-cp39-cp39-manylinux_2_17_aarch64.manylinux2014_aarch64.whl'), PosixPath('/tmp/arrow-18.0.0.pfUwd/binaries/python-rc/18.0.0-rc0/pyarrow-18.0.0-cp39-cp39-manylinux_2_28_aarch64.whl'), PosixPath('/tmp/arrow-18.0.0.pfUwd/binaries/python-rc/18.0.0-rc0/pyarrow-18.0.0-cp313-cp313t-manylinux_2_28_x86_64.whl'), PosixPath('/tmp/arrow-18.0.0.pfUwd/binaries/python-rc/18.0.0-rc0/pyarrow-18.0.0-cp310-cp310-manylinux_2_28_x86_64.whl')])
Failed to verify release candidate. See /tmp/arrow-18.0.0.pfUwd for details.
Error: `docker compose --file=/home/raulcd/code/arrow/docker-compose.yml run --rm -e VERIFY_VERSION=18.0.0 -e VERIFY_RC=0 -e TEST_DEFAULT=0 -e TEST_WHEELS=1 almalinux-verify-rc` exited with a non-zero exit code 1, see the process log above.

raulcd added a commit to raulcd/arrow that referenced this issue Oct 17, 2024
@raulcd
apacheGH-44459: [Release][Packaging] Do not check wheel content when …
6eb2557 
…verifying binary wheels
@github-actions github-actions bot mentioned this issue Oct 17, 2024
Merged
@raulcd raulcd added this to the 18.0.0 milestone Oct 17, 2024
@raulcd raulcd mentioned this issue Oct 17, 2024
Open
kou pushed a commit that referenced this issue Oct 17, 2024
@raulcd
GH-44459: [Release][Packaging] Do not check wheel content when verify…
3657c94 
…ing binary wheels (#44460)

### Rationale for this change

Currently the folder used to verify the contents of the wheels is not correct when doing a release verification of the wheels and the verification fails. The script is supposed to check wheels individually and we are already verifying the contents on the wheels individually when generating the wheels.

### What changes are included in this PR?

Stop checking wheel content on release verification as the wheel content has already been validated when generating the wheels.

There is also a minor fix on the platform tags for the macOS wheels that was missed when we updated the MACOSX_DEPLOYMENT_TARGET.

### Are these changes tested?

Yes via archery.

### Are there any user-facing changes?

No
* GitHub Issue: #44459

Authored-by: Raúl Cumplido <raulcumplido@gmail.com>
Signed-off-by: Sutou Kouhei <kou@clear-code.com>
@kou
Copy link
Member

kou commented Oct 17, 2024

Issue resolved by pull request 44460
#44460

@kou kou closed this as completed Oct 17, 2024
@raulcd raulcd modified the milestones: 18.0.0, 19.0.0 Oct 18, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@raulcd raulcd

Labels
backport-candidate Component: Packaging Component: Release Type: bug
Projects
None yet
Milestone
19.0.0
Development

No branches or pull requests
2 participants
@kou @raulcd