Tracking issue for PEP 600 Perennial manylinux rollout

[mattip]

PEP 600 has been accepted to adopt the manylinux_2_?? standard, we need a tracking issue for implementation (I'm reusing structure and prose from gh-338, maintainers : please edit this so it makes sense).

I am pushing this forward so we can use a newer docker image for other architectures: the available arm64 compilers on CentOS 7 have a bug with copysign xref gh-494 and the glibc 2.18 versions of transcendental functions have bugs, the NumPy work-arounds are based on long double but ppc64 uses double double xref numpy/numpy#15763. Moving to an image based on glibc2.24 (debian stretch, centos8 uses 2.28) will help move the goalposts on these issues.

---

Client-side Support:

• Update packaging to generate the perennial tags (Support PEP 600 tags packaging#293)
• New release of packaging (Release 20.5 packaging#345)
• Update pip to use the new packaging (Implement PEP-600: Future 'manylinux' Platform Tags for Portable Linux Built Distributions pip#9077)
• New release of pip (pip 20.3 release (Q4 2020) pip#8936)
• Update wheel to use the new packaging
• New release of wheel

Publisher-side Support:

• Update Warehouse (PyPI) to accept perennial wheel uploads (Allow manylinux_x_y wheel upload (PEP600) pypi/warehouse#7853)
• Update auditwheel to emit perennial wheel archives (Add manylinux_2_24 policy auditwheel#264)
• New release of auditwheel (Release 3.3.0 auditwheel#270)
• Provide a manylinux_2_24 docker environment (Add manylinux_2_24 images #999)
  • Including a rewrite of pypa/manylinux branches so that master produces all the currrent manylinux images (thanks @mayeut)
• Upload the new images to quay.io
• Confirm that twine will upload perennial wheel archives

Documentation:

• Update https://packaging.python.org/specifications/platform-compatibility-tags/ to cover manylinux_2_X (Mention manylinux_x_y in platform tags specification packaging.python.org#832)

Additional projects to consider once core capability support has rolled out:

• Update pipenv to install perennial wheels (Upgrade version of vendored package 'packaging' to support PEP 600 pipenv#4591)
  • Needed due to pipenv vendoring pip, so the support depends on the version of pipenv being used, rather than the version of pip in the host or target environments)
• Add an option to target manylinux_2_X in cibuildwheel (Add manylinux_2_24 support cibuildwheel#595)
• Add an option to target manylinux_2_X in multibuild (matthew-brett/multibuild#396)
• Distinguish between images in dockcross
• Develop list of further relevant helper utilities and update them

[mattip]

The PR against packaging to allow PEP 600 tags (pypa/packaging#293) has been merged, so once that percolates out to pip and wheel people could, theoretically, begin producing PEP600 wheels.

[mattip]

Pip has been updated, setuptools and wheel still have not.

[mattip]

A version of wheel with packaging as a dependency has been released. I guess the next step is to decide on the first PEP 600 format, create images, and adapt auditwheel to the image. I propose using glibc 2.24 via debian stretch, since that will push NumPy past these issues:

• On ppc64, malloc gives misaligned buffers for glibc <= 2.23. numpy/numpy#10491
• BUG: ppc64le uses double-double for np.float128, routines need adjustment numpy/numpy#15763
• numpy #14886 (trig functions on glibc<=2.18 are not accurate enough)

[di]

I've merged pypi/warehouse#7853 (and checked the appropriate box here).