-
Notifications
You must be signed in to change notification settings - Fork 253
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
packaging.tags._platform_tags() does not report current OS version #435
Comments
It seems like all 11.x are being normalised to 11.0 explicitly: Lines 402 to 411 in 7350746
Since this code is contributed by an Apple employee, I feel we should ask Apple for some clarification on this. @lawrence-danna-apple |
Who is generating that wheel file name? If my memory serves me, the flooring is because we have no idea what the maximum potential minor version of macOS is, and builds should be compatible for any major version of the OS. So if you have a wheel built on macOS 11.987 but you're running macOS 11.2, it should still work, but there's no way to really express that short of dropping the minor version number for macOS > 11. |
In this case, |
So are you getting the file name from pip and then naming your wheel file yourself? Pip is not a build tool (e.g. setuptools and wheel are a build tool), so I'm not quite following how pip is participating in the wheel file naming. |
Not saying it’s the case here, but shouldn’t developers be able to have legistimate reasons to require at least 11.2 and later? And if they do, the resulting wheel should be installable on 11.2+. So |
@brettcannon No, the wheel is being built during a pip install and then cached with the filename containing |
@uranusjr Maybe, but I believe @lawrence-danna-apple did this on purpose as macOS is making a promise that all 11.x wheels will be compatible. I don't have an issue working backwards for compatibility on a platform, but we should probably give @lawrence-danna-apple a chance to reply before we move forward with any potential change. @DanLipsitt if this is about numpy then the build tool will be (I believe) their custom setuptools setup plus wheel (pip is just calling those tools on numpy's behalf). So it's probably wheel that's making the macOS 11.2 wheel. |
@brettcannon It happens with I took a look at >>> from wheel.vendored.packaging import tags
>>> list(tags._platform_tags())
['macosx_11_0_arm64', 'macosx_11_0_universal2'] So that's not where I will take a closer look at how the packages are being built. |
@DanLipsitt did you happen to ever figure out why your builds are getting the @uranusjr since we have not heard from @lawrence-danna-apple I'm going to ask @ronaldoussoren and @ned-deily if they have an opinion about changing the logic. Otherwise I would loathe to assume I knew macOS better than an Apple employee who added that logic on purpose and change how it has been working. |
Yeah, there are definitely some issues here. What's driving this is that Apple changed macOS versioning as of macOS 11 (Big Sur), going from a three-level versioning system in previous macOS releases (similar to the scheme used by cPython) to a two-level one (similar to what Apple's iOS has been using). At the cPython level, we didn't quite grasp all the implications of that (other non-Python projects have been impacted by this as well) and I think we are still not handling things quite right. That also has an impact on packaging. I will look at this further and report back here. |
I don't have the bandwidth to look into this, but I agree with what Ned writes. It might be better to bite the bullet and switch to using just the major version for macOS 11 and later, that is "macosx_11_universal2". Apple can introduce new functionality in minor releases (such as macOS 10.15.3 or macOS 11.1), but so far I've never seen such changes in unix-level APIs. Frameworks do changes at times (new AppKit APIs for example), but those should affect only a small number of projects and those should handle weak-linking such new APIs anyway. For a project as numpy the wheel generated on macOS 11.2 should work fine on macOS 11.0 without such shenanigans. P.S. Please prefer universal2 wheels over singel architecture wheels when installing ;-) |
If we have to special-case macOS 11 to emit both @ronaldoussoren I see your push for #381 😉 |
To state the obvious: changing from 11_0 to 11 would require changes to package building tools (in this particular case likely setuptools). Fixing this on the pip side would require changing the list of compatible wheels. This already contains a long list of versions back to macOS 10.4, adding subreleases of macOS 11 to the list wouldn't be too bad. That said, those subreleases being bug fix releases does complicate thing as you wouldn't want to hardcode the latest bug fix release in every major release (that is, I'm on macOS 12.0, which 11.x wheels are considered compatible). I guess just assuming that those minor releases are single digit would be good enough (tm), Mac OS 10.4 was the last release that required 2 digits (10.4.10) and that was in a time before yearly releases. |
Overview
On MacOS 11.2 arm64, the list returned by
packaging.tags._platform_tags()
does not containmacosx_11_2_arm64
. The closest match ismacosx_11_0_arm64
.Consequences
Because of this issue, pip does not recognize cached wheels that have been built on the same machine and instead repeats long compile processes. For example, it builds and caches
numpy-1.20.3-cp39-cp39-macosx_11_2_arm64.whl
but then does not recognize it as a compatible wheel and rebuilds it.To Reproduce
The text was updated successfully, but these errors were encountered: