[Bug]: Python build RC wheels failure

[kileys]

What happened?

/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/_distutils_hack/__init__.py:19: UserWarning: Distutils was imported before Setuptools, but importing Setuptools also replaces the `distutils` module in `sys.modules`. This may lead to undesirable behaviors or errors. To avoid these issues, avoid using distutils directly, ensure that setuptools is installed in the traditional way (e.g. not an editable install), and/or make sure that setuptools is always imported before distutils.
    "Distutils was imported before Setuptools, but importing Setuptools "
  /Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/_distutils_hack/__init__.py:33: UserWarning: Setuptools is replacing distutils.
    warnings.warn("Setuptools is replacing distutils.")
  Traceback (most recent call last):
    File "/Users/runner/hostedtoolcache/Python/3.7.13/x64/lib/python3.7/site-packages/cibuildwheel/resources/get-pip.py", line 22318, in <module>
      main()
    File "/Users/runner/hostedtoolcache/Python/3.7.13/x64/lib/python3.7/site-packages/cibuildwheel/resources/get-pip.py", line 198, in main
      bootstrap(tmpdir=tmpdir)
    File "/Users/runner/hostedtoolcache/Python/3.7.13/x[64](https://github.com/apache/beam/runs/7701131404?check_suite_focus=true#step:11:66)/lib/python3.7/site-packages/cibuildwheel/resources/get-pip.py", line 120, in bootstrap
      import setuptools  # noqa
    File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/setuptools/__init__.py", line 8, in <module>
      import _distutils_hack.override  # noqa: F401
    File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/_distutils_hack/override.py", line 1, in <module>
      __import__('_distutils_hack').do_override()
    File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/_distutils_hack/__init__.py", line 77, in do_override
      ensure_local_distutils()
    File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/_distutils_hack/__init__.py", line 64, in ensure_local_distutils
      assert '_distutils' in core.__file__, core.__file__
  AssertionError: /Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/distutils/core.py
                                                              ✕ 2.18s
Error: Command ['python', PosixPath('/Users/runner/hostedtoolcache/Python/3.7.13/x64/lib/python3.7/site-packages/cibuildwheel/resources/get-pip.py'), '-c', PosixPath('/Users/runner/hostedtoolcache/Python/3.7.13/x64/lib/python3.7/site-packages/cibuildwheel/resources/constraints-python37.txt')] failed with code 1. None

https://github.com/apache/beam/runs/7701131404?check_suite_focus=true

Issue Priority

Priority: 1

Issue Component

Component: testing

[kileys]

@AnandInguva Potentially related to distutils change?
cc: @tvalentyn

[tvalentyn]

@AnandInguva is investigating. Thanks.

[AnandInguva]

.take-issue

[AnandInguva]

Recently the use of distutils was updated. The build RC wheels test has been skipped on the master branch so this test was missed while merging

Reason: cibuildwheel is used to build wheels for the Beam Python SDK. We use 1.11.0 version which is very old and with setuptools>=60, it causes an assertion error. More details can be found here pypa/setuptools#2993. One of the solutions is to update the cibuildwheels to the newest version

Working on a PR to create a fix for this bug.

[aaltay]

Thank you for the quick resolution!