problem calling variadic functions imported from dynamic libraries under macOS on Apple Silicon

[applebud]

Bug report

Bug description:

I see a problem calling variadic functions imported from dynamic libraries under macOS on Apple Silicon.
I’m using a bit of code sample from https://docs.python.org/3/library/ctypes.html#specifying-the-required-argument-types-function-prototypes
First with Intel macOS Python (running via Rosetta):

% python3
Python 3.11.6 (main, Nov  3 2023, 03:33:27) [Clang 14.0.0 (clang-1400.0.29.202)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import platform
>>> platform.version()
'Darwin Kernel Version 23.3.0: Wed Dec 20 21:31:00 PST 2023; root:xnu-10002.81.5~7/RELEASE_ARM64_T6020'
>>> platform.processor()
'i386'
>>> from ctypes import *
>>> libc = cdll.LoadLibrary("libc.dylib")
>>> printf = libc.printf
>>> printf.argtypes = [c_char_p, c_char_p, c_int, c_double]
>>> printf(b"String '%s', Int %d, Double %f\n", b"Hi", 10, 2.2)
String 'Hi', Int 10, Double 2.200000
37
>>> 

And now with native Apple Silicon Python:

% python3
Python 3.11.7 (main, Dec  4 2023, 18:10:11) [Clang 15.0.0 (clang-1500.1.0.2.5)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import platform
>>> platform.version()
'Darwin Kernel Version 23.3.0: Wed Dec 20 21:31:00 PST 2023; root:xnu-10002.81.5~7/RELEASE_ARM64_T6020'
>>> platform.processor()
'arm'
>>> from ctypes import *
>>> libc = cdll.LoadLibrary("libc.dylib")
>>> printf = libc.printf
>>> printf.argtypes = [c_char_p, c_char_p, c_int, c_double]
>>> printf(b"String '%s', Int %d, Double %f\n", b"Hi", 10, 2.2)
String '(null)', Int 0, Double 0.000000
40
>>> 

As you can see the results are incorrect.

CPython versions tested on:

3.11

Operating systems tested on:

macOS

[ronaldoussoren]

See also: #107844, and this part of the documentation: https://docs.python.org/3/library/ctypes.html#calling-variadic-functions

In this particular case you shouldn't have to use argtypes for printf, or rather only have to specify the first argument. That doesn't necessarily work in general though.

The issue here is that on macOS/arm64 the calling conventions for regular and "variadic" arguments are different. During the initial port of Python to macOS/arm64 we choose to deduce which arguments are regular and which are variadic by checking the length of argtypes. That works most of the time, but not always as you have noticed.

We should add some (optional) API that explicitly tells ctypes which arguments are regular and which are variadic (for example by adding an attribute to function pointer object that denotes the index of the first variadic argument).

That is an API change and can be included in 3.13 at the earliest.

[applebud]

Yes, thanks, that workaround does work, with the catch that I have to effectively cast the double by hand:

>>> printf.argtypes=[c_char_p]
>>> printf(b"String '%s', Int %d, Double %f\n", b"Hi", 10, 2.2)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ctypes.ArgumentError: argument 4: TypeError: Don't know how to convert parameter 4
>>> printf(b"String '%s', Int %d, Double %f\n", b"Hi", 10, c_double(2.2))
String 'Hi', Int 10, Double 2.200000
37