extend `sys` module on windows

[maxbachmann]

It would be useful to be able to detect:

1. the platform as 32-bit vs 64-bit vs ARM64.
2. the windows api partition
   • multiple partitions can be true at the same time, so this should probably be a named tuple

@zooba you mentioned adding them to sys.implementation. Since they are only available on windows, I assume this would mean adding them with an underscore prefix? The alternative would be a separate API similar to sys.winver.

This is a follow up to #102256.

Linked PRs

• gh-102536: Added _architecture field to sys.implementation #124582

[zooba]

Yeah, I don't think we want to bother with making them required attributes, so I'd add:

• sys.implementation._architecture and make it the value of $(ArchName) from python.props (this approx. matches platform.architecture)
• sys.implementation._api_partitions and make it a tuple of all the ones we test for. No need to add names, simple in checks will be the most common use

[ericsnowcurrently]

Is sys.implementation the right place? It should have things related only to the Python implementation, that other implementations also have. The architecture is a build characteristic and the api partitions seem like it as well. Where do we normally record build or platform (or OS) information?

[zooba]

The only real alternative is at the top level of the sys module. Maybe you could make a case for it going into sysconfig, but we still need an attribute on a native module somewhere, as it's the only way we're getting the field it at build time.

I assume this hasn't come up before because Windows is the only OS with transparent CPU emulation? So querying the current OS architecture isn't sufficient to determine the current process architecture.

[ericsnowcurrently]

sys._build.*?

[zooba]

Really it should be sys.platform_but_for_real_this_time as we can't just go updating sys.platform any more.

What if we added sys.architecture for all platforms? platform.architecture(sys.executable) is too limited in its return values to work here, but it ought to be known at compile time, right? And something like Jython or IronPython can use clr or jvm if they prefer. Maybe they've already got an additional field for that?

[vstinner]

the platform as 32-bit vs 64-bit vs ARM64.

There are multiple existing APIs to do that. Examples:

>>> platform.architecture()
('64bit', 'ELF')
>>> os.uname().machine
'x86_64'

I don't think that we should add information retrieved at runtime in sys. Usually, sys is more information computed during the build.

[zooba]

platform.architecture() doesn't distinguish between ARM64 and x64, and os.uname().machine doesn't reflect how CPython was compiled. So we need to store the target architecture at build time.

But you're correct that we don't need to retrieve it at runtime. (Though worth noting that a lot of sys is dynamic - path, modules, various hooks, etc.)

[rruuaanng]

Hmmm. Actually, the discussion in the PR failed because no one decided whether to merge it or not (it seems that the only supporter is @zooba).
The newly added arch is used to get the current build architecture in Windows (it comes from python.props). What do you think?

link:
#124582
https://discuss.python.org/t/regarding-whether-we-should-add-py-currentarch-or-py-archname-function/65191/17

[ericsnowcurrently]

My main concern was/is that this information doesn't belong on sys.implementation. That's meant for information about the Python implementation (e.g. cpython, pypy, jython, micropython), not about the build. See PEP 421.

Something like sys._architecture, sys.build.architecture, or platform.??? or on sysconfig (like @zooba suggested) would be more appropriate. (FWIW, I like the idea of sys.build.*, PEP-421-style, but its hypothetical contents might already be covered elsewhere.)

All that said, I haven't looked closely at what we're trying to actually accomplish here, so I don't have much feedback on that.

[zooba]

I'm often the only supporter of things involving Windows :-) Usually aggregated from a lot of discussions with a range of users, but not easily cited (many Windows users are scared off by the culture here, so they won't report stuff themselves).

In this case, we've been looking at adding support to project's build scripts to allow building on Windows ARM64, which requires the builder to know whether it should be targeted ARM64 or not. That's usually done by running in the ARM64 interpreter, and then to cross-compile you'd run in the x64 or x86 interpreter, but we don't currently have a reliable way to detect that. Historically we haven't needed it, because sys.maxsize (or sys.maxint) was sufficient to detect 64-bit vs 32-bit. But that can't tell the difference between ARM64 and AMD64.

More importantly, and relevant for this case, is that platform.architecture() can't return the difference, even if it could detect it, because it's defined as only returning "64" or "32". And PEP 421 explicitly suggests sys.implementation is the place to store information that would be returned from platform about the Python implementation. The architecture the current runtime was compiled for sure seems to be related to the implementation, and since platform would be the obvious place to return such information (as it already is, albeit insufficiently), adding the field to sys.implementation seems fine.

Having now re-read that PEP though,¹ I think we should drop the underscore. Make it sys.implementation.architecture and if it exists then platform.architecture(sys.executable) can return it rather than querying the actual binary (if it's within the range of allowable return values, and perhaps we can later add platform.architecture_but_for_real_this_time())

Footnotes

1. Specifically Non-required Attributes and Use Cases, the platform and Jython use cases being most relevant. ↩

[vstinner]

platform.machine() is another possible place for such information. It already has Windows specific code to detect the architecture.