False positive with -fsanitize=function on macOS

[neildhar]

Seeing failures with -fsanitize=function when building on macOS for function calls where the type signature is actually correct. It seems like the type hash is not being preserved through linking.

Verified with:

• Apple Clang from Xcode 16 (only when building for x64)
• Clang from LLVM 18.1.4 (both x64 and arm64 macOS)

Minimised repro (note that the exact ordering is important to get things in the right order in the final binary):

foo.cpp:

void callPtr(void (*fn)());

void myFn();

template <typename T>
int myT(){ return 5; }

int main(){
  myT<void>();
  callPtr(&myFn);
  return 0;
}

void myFn(){}

foo2.cpp:

template <typename T>
int myT(){ return 5; }

void callPtr(void (*fn)()){
    myT<void>();
    fn();
}

Build and run (add -arch x86_64 if needed):

clang++ foo2.cpp foo.cpp -fsanitize=function && ./a.out

Result:

/Users/neildhar/foo2.cpp:6:5: runtime error: call to function myFn() through pointer to incorrect function type 'void (*)()'
(a.out:arm64+0x100003f38): note: myFn() defined here
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior /Users/neildhar/foo2.cpp:6:5

To my untrained eye, it looks like the linker may be treating the type hash placed before myFn as part of the preceding function, and when that function occurs in multiple object files (as is the case for myT<void>), only one copy is kept. As a result, the bytes preceding myFn no longer represent its correct type hash.

[MaskRay]

To my untrained eye, it looks like the linker may be treating the type hash placed before myFn as part of the preceding function, and when that function occurs in multiple object files (as is the case for myT), only one copy is kept. As a result, the bytes preceding myFn no longer represent its correct type hash.

Yes. Mach-O's .subsections_via_symbols mechanism conceptually divides a sections with atoms (non-temporary symbols that are not .alt_entry).
For the following assembly, the two .long conceptually belongs to the previous function.

.long ...
.long ...

_f:
  nop

Some linker features (-dead_strip, -order_file) might move the .long relative to the function (and its body).

Perhaps we should just make -fsanitize=function unsupported on Mach-O platforms, but there are some cases that it would work.

[neildhar]

Thank you for the background, that certainly explains the behaviour I observed.

Perhaps we should just make -fsanitize=function unsupported on Mach-O platforms, but there are some cases that it would work.

To me, it would be preferable to disable it for Mach-O. Having UBSan fail unpredictably on correct code seems like a serious issue.