Why does warehouse allow linux_armv6l and linux_armv7l wheels?

[njsmith]

Historically we never allowed linux wheels on PyPI, and this was an intentional choice, because linux doesn't tell us enough to make any guesses about ABI compatibility. That's still true. But warehouse now allows specifically linux_armv6l and linux_armv7l as legal wheel platforms (but no other options).

This appears to have come in through #2003, #2010, #2012, but none of those have any discussion of why we suddenly changed this policy...

What's going on here? Why are we allowing linux wheels at all, and why is it specific to these two CPU architectures?

[njsmith]

This appears to be connected somehow to the piwheels project to provide wheels for Raspberry Pis running Raspbian, which is super cool. But as their FAQ says:

Will wheels provided on piwheels work on other ARM platforms?

Some wheels may work on other ARM platforms, but we can't guarantee this. Pure Python wheels will certainly work, but there's a much smaller speed increase installing from wheels over source distribution. ARM platform wheels on piwheels are tagged armv6l and armv7l but actually both contain ARMv6-compatible code, which may not work on true ARMv7 platforms.

[bennuttall]

I started the piwheels project as a solution for Raspberry Pi users, and around the same time, some google devs working on a Pi project wanted to be able to distribute wheels for Pi users. piwheels was just getting started so they requested warehouse supported uploading armv6 and armv7 wheels. This was accepted quite quickly and then warehouse's back-end became the new default for uploads (a while before the front-end).

Because Raspberry Pi's OS, Raspbian, is compatible with all Pi models (Pi 1/Zero are Armv6, Pi 2 is Armv7 and Pi 3 is Armv8), it's all Armv6 userland, despite Pi 2/3 presenting as Armv7. So wheels built on a Pi 3 will have the platform tag linux_armv7l, but are identical to a wheel built on a Pi 1 (just much faster build time), so we provide armv7 wheels built on a Pi 3, and symlink them to provide armv6 wheels for Pi 1/Zero users.

I wouldn't advise anyone upload these kind of wheels to PyPi (note you can't upload two identical files even with different filenames, so you could only upload an armv7 not both), because they're not truly Armv7. But I guess there's a use case for other Arm boards. piwheels doesn't pretend to be an Arm package repository, just a Raspberry Pi one. But if package maintainers want their packages to work for Pi users and other Arm board users, they need to be sure what they upload is compatible with everyone (and hope that doesn't cause piwheels to symlink armv7 to armv6 and break it for some people).

See piwheels/piwheels#66 for issues around people uploading true Armv7 wheels to PyPI.

[ncoghlan]

I'm assuming that we can't reinstate the prohibition on Linux wheel uploads for ARM at this point without causing UX problems for Raspberry Pi users. So let's not do that :)

Instead, my suggestion would be that we restrospectively declare linux_armv7l and linux_armv6l to be Raspbian specific compatibility tags, and update pip (and any compatibility tag helper libraries) to check /etc/os-release/ before considering them as candidates for downloading from PyPI.

If folks from the Raspberry Pi project were then interested in helping define distro-specific ABI compatibility tags (which would make it possible to distribute distro-specific wheels via PyPI without causing compatibility problems for users of other distros), that would be most welcome: https://mail.python.org/pipermail/distutils-sig/2018-April/032117.html is the latest write-up of a design that we believe would work for that purpose, and what might be involved in getting there.

[ncoghlan]

#3806 proposes a new comment on the list of permitted ABI tags noting that there's a lot more involved in adding a new ABI tag than just approving it there.

[njsmith]

Would removing support for uploading linux_armv?l wheels cause a problem for Raspberry Pi users? Or are they all using piwheels instead? How many linux_armv?l wheels actually ended up getting uploaded to PyPI?

(Agreed that it would be great to have a proper wheel tag for this.)

[ncoghlan]

Ah, I'd missed that detail (https://www.piwheels.org/ is the URL). In that case, yeah, it would be highly desirable to go back to preventing use of these tags on pypi.org itself, and have the Warehouse level feature be a config setting for "extra wheel compatibility tags".

Then @bennuttall could add these tags to the config setting for piwheels, and be in a similar situation to Galaxy Project where the ABI is implied by the repository you're using, rather than the compatibility tag.

[njsmith]

Here's all the linux_arm downloads for PyPI in March 2018: https://docs.google.com/spreadsheets/d/1rbdY2KT8t4o7BVEmwvwc9Fh7D7wpUjj6tiClEhrQ0lY/edit#gid=26881496

Looks like ~20k total, over a handful of packages, mostly from google (maybe the dependency stack for a single package?)

Query:

SELECT
  COUNT(*) AS downloads,
  file.filename
FROM
  TABLE_DATE_RANGE( [the-psf:pypi.downloads], TIMESTAMP("20180301"), TIMESTAMP("20180401") )
WHERE
  file.filename CONTAINS 'linux_arm'
GROUP BY
  file.filename
ORDER BY
  downloads DESC
LIMIT
  1000

[dstufft]

I don't particularly remember merging that PR, but generally we don't prevent people from uploading incompatible wheels (e.g. you can upload a manylinux wheel that is actually ubuntu specific) and there are benefits to allowing the generic tags. You may have additional files that only make sense on a Linux distribution (e.g. man files, etc) while still being a pure Python wheel. I believe that Armin Ronacher has been looking for a generic linux tag for quite some time as well for similar reasons.

Quite generally, I don't think that the linux_arvm?l tags are Pi specific, and if folks are using them that way, that's a bug in their packages.

[njsmith]

@dstufft the problem is that if you don't treat the linux_armv?l tags as being Pi specific, then you can't use them at all, because there is no generic "linux arm" ABI, there's only "this works on a Pi running this version of Raspbian, but not Android", or "this works on Android, but not on a Pi", etc. etc.

OK yes I guess if Armin is building fully statically linked rust binaries that don't depend on the system libc, then that is an actual generic linux wheel, but if we want to allow that then we should talk about it on distutils-sig and there's no reason it should be allowed only on ARM. And I'm 99% sure that none of the linux ARM wheels currently on PyPI are like this.

[bennuttall]

Quite generally, I don't think that the linux_arvm?l tags are Pi specific, and if folks are using them that way, that's a bug in their packages.

linux_armv?l tags aren't Pi specific, but if someone built a wheel on a Pi, it may be incompatible with other Arm boards.

Ah, I'd missed that detail (https://www.piwheels.org/ is the URL). In that case, yeah, it would be highly desirable to go back to preventing use of these tags on pypi.org itself, and have the Warehouse level feature be a config setting for "extra wheel compatibility tags".

👍

Then @bennuttall could add these tags to the config setting for piwheels, and be in a similar situation to Galaxy Project where the ABI is implied by the repository you're using, rather than the compatibility tag.

@ncoghlan Are you suggesting we force another tag on the wheels we build? Wouldn't that require a change in pip for requests from Raspbian to match the tag?

Does anyone consider the fact pip identifies the platform as armv7 on a Pi 2/3 a bug, when it's actually armv6 userland? It's a detail we can mostly work around but it's the only compatibility issue.

[njsmith]

Does anyone consider the fact pip identifies the platform as armv7 on a Pi 2/3 a bug, when it's actually armv6 userland? It's a detail we can mostly work around but it's the only compatibility issue.

Yeah, you should file a separate bug on pip for that. It already has similar code to work around a similar problem on x86 (64-bit processor but 32-bit userland): https://github.com/pypa/pip/blob/7b1f2a06d24bd90a28405e52e9184848d33576c7/src/pip/_internal/pep425tags.py#L135-L138

[dstufft]

if we want to allow that then we should talk about it on distutils-sig and there's no reason it should be allowed only on ARM.

I could have swore we already relaxed the restriction on x86 and amd64 wheels as well to allow them, but looking at the code that's not true. Maybe I intended to do it but never actually got around to doing it.

I'm 99% sure that none of the linux ARM wheels currently on PyPI are like this.

Wheel has bad defaults, we should fix them. There's no reason why a generic linux wheel is an unreasonable thing, the only real problem is that people will get it by default with opt-ing into it. Wheel should produce a hyper specific wheel by default, and require opting into the "generic" linux tags just like it does for manylinux. Beyond that, if people opt into a generic wheel but produce a specific wheel, that's a packaging bug on that package, the same as if they did it with a manylinux wheel.