Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix macOS build with all sanitizers enabled. #47939

Merged
merged 1 commit into from
Apr 16, 2021

Conversation

bruvzg
Copy link
Member

@bruvzg bruvzg commented Apr 15, 2021

Some sanitizer flag fixes for macOS (#40924 follow-up).

  • There's no use_llvm environment variable, macOS build is LLVM only.
  • -fsanitize=leak is not supported as standalone flag (it is integrated into address sanitizer, see: https://clang.llvm.org/docs/LeakSanitizer.html).
  • -fsanitize=implicit-integer-sign-change is not supported.

Also removes implicit-signed-integer-truncation and implicit-unsigned-integer-truncation.

@bruvzg bruvzg added this to the 4.0 milestone Apr 15, 2021
@bruvzg bruvzg requested a review from a team as a code owner April 15, 2021 21:25
@bruvzg
Copy link
Member Author

bruvzg commented Apr 15, 2021

Sanitizer messages on editor startup on macOS:

thirdparty/nanosvg/nanosvg.h:2190:49: runtime error: division by zero
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior thirdparty/nanosvg/nanosvg.h:2190:49 in

// Split arc into max 90 degree segments.
// The loop assumes an iteration per end point (including start and end), this +1.
ndivs = (int)(fabsf(da) / (NSVG_PI*0.5f) + 1.0f);
hda = (da / (float)ndivs) / 2.0f;
kappa = fabsf(4.0f / 3.0f * (1.0f - cosf(hda)) / sinf(hda));

servers/rendering/renderer_rd/forward_clustered/render_forward_clustered.cpp:2719:64: runtime error: division by zero
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior servers/rendering/renderer_rd/forward_clustered/render_forward_clustered.cpp:2719:64 in

float max_scale = MAX(model_scale_vec.x, MAX(model_scale_vec.y, model_scale_vec.z));
float min_scale = MIN(model_scale_vec.x, MIN(model_scale_vec.y, model_scale_vec.z));
ginstance->non_uniform_scale = max_scale >= 0.0 && (min_scale / max_scale) < 0.9;

@RevoluPowered
Copy link
Contributor

RevoluPowered commented Apr 15, 2021

@bruvzg You're totally right about the leak detection

I do think we should also consider that LLVM could be used on OSX directly:
https://stackoverflow.com/questions/53456304/mac-os-leaks-sanitizer

Apparently we can support leak detection if we use the LLVM/clang but not the one shipped from apple.

Also docs states OSX support:
https://clang.llvm.org/docs/LeakSanitizer.html

I think we should fix it, I see there are linker problems. The manual says to use clang++ to do the linking portion too.

To use LeakSanitizer in stand-alone mode, link your program with -fsanitize=leak flag. Make sure to use clang (not ld) for the link step, so that it would link in proper LeakSanitizer run-time library into the final executable.

@RevoluPowered
Copy link
Contributor

RevoluPowered commented Apr 15, 2021

OK for the short term we will have to merge this patch regardless, I swapped to the clang linker but the issue persisted with the link process, even after following the manual.

Also, I think the CI should have detected that these options are broken so I will follow up with a CI change to test we can compile ALL sanitisers, note: this won't mean they actually execute, just that they at least compile.

In order to test the leak system I changed the section here see LINK this is not in our current setup we are using LD.

        else:
            env["CC"] = "clang"
            env["CXX"] = "clang++"
            env["LINK"] = "clang++"

The log for the linker error was as follows

Undefined symbols for architecture x86_64:
"___ubsan_handle_add_overflow", referenced from:
List<String, DefaultAllocator>::push_back(String const&) in crash_handler_osx.osx.tools.64s.o
Vector<Logger*>::push_back(Logger*) in os_osx.osx.tools.64s.o
List<String, DefaultAllocator>::push_back(String const&) in os_osx.osx.tools.64s.o
List<String, DefaultAllocator>::push_back(String const&) in display_server_osx.osx.tools.64s.o
-[GodotContentView insertText:replacementRange:] in display_server_osx.osx.tools.64s.o
_push_to_key_event_buffer(DisplayServerOSX::KeyEvent const&) in display_server_osx.osx.tools.64s.o
Vector::push_back(String) in display_server_osx.osx.tools.64s.o
...
"___ubsan_handle_builtin_unreachable", referenced from:
handle_crash(int) in crash_handler_osx.osx.tools.64s.o
OS_Unix::debug_break() in libdrivers.osx.tools.64s.a(os_unix.osx.tools.64s.o)
_terminator_DestroySurfaceKHR in libdrivers.osx.tools.64s.a(wsi.osx.tools.64s.o)
_terminator_GetPhysicalDeviceSurfaceSupportKHR in libdrivers.osx.tools.64s.a(wsi.osx.tools.64s.o)
_terminator_GetPhysicalDeviceSurfaceCapabilitiesKHR in libdrivers.osx.tools.64s.a(wsi.osx.tools.64s.o)
_terminator_GetPhysicalDeviceSurfaceFormatsKHR in libdrivers.osx.tools.64s.a(wsi.osx.tools.64s.o)
_terminator_GetPhysicalDeviceSurfacePresentModesKHR in libdrivers.osx.tools.64s.a(wsi.osx.tools.64s.o)
...
"___ubsan_handle_divrem_overflow", referenced from:
-[GodotContentView firstRectForCharacterRange:actualRange:] in display_server_osx.osx.tools.64s.o
DisplayServerOSX::mouse_warp_to_position(Vector2i const&) in display_server_osx.osx.tools.64s.o
DisplayServerOSX::screen_get_dpi(int) const in display_server_osx.osx.tools.64s.o
DisplayServerOSX::_create_window(DisplayServer::WindowMode, Rect2i const&) in display_server_osx.osx.tools.64s.o
DisplayServerOSX::cursor_set_custom_image(Ref const&, DisplayServer::CursorShape, Vector2 const&) in display_server_osx.osx.tools.64s.o
Geometry2D::is_point_in_polygon(Vector2 const&, Vector const&) in display_server_osx.osx.tools.64s.o
DisplayServerOSX::set_icon(Ref const&) in display_server_osx.osx.tools.64s.o
...
"___ubsan_handle_dynamic_type_cache_miss", referenced from:
handle_crash(int) in crash_handler_osx.osx.tools.64s.o
OS_OSX::initialize_core() in os_osx.osx.tools.64s.o
OS_OSX::initialize_joypads() in os_osx.osx.tools.64s.o
OS_OSX::initialize() in os_osx.osx.tools.64s.o
OS_OSX::finalize() in os_osx.osx.tools.64s.o
OS_OSX::set_main_loop(MainLoop*) in os_osx.osx.tools.64s.o
OS_OSX::delete_main_loop() in os_osx.osx.tools.64s.o
...
"___ubsan_handle_float_cast_overflow", referenced from:
-[GodotWindowDelegate windowDidExitFullScreen:] in display_server_osx.osx.tools.64s.o
-[GodotWindowDelegate windowDidChangeBackingProperties:] in display_server_osx.osx.tools.64s.o
-[GodotWindowDelegate windowDidResize:] in display_server_osx.osx.tools.64s.o
_get_mouse_pos(DisplayServerOSX::WindowData&, CGPoint) in display_server_osx.osx.tools.64s.o
-[GodotContentView mouseMoved:] in display_server_osx.osx.tools.64s.o
DisplayServerOSX::mouse_get_absolute_position() const in display_server_osx.osx.tools.64s.o
DisplayServerOSX::_get_native_screen_position(int) const in display_server_osx.osx.tools.64s.o
...
"___ubsan_handle_function_type_mismatch_v1", referenced from:
_err_print_error(char const*, char const*, int, char const*, char const*, ErrorHandlerType) in libcore.osx.tools.64s.a(error_macros.osx.tools.64s.o)
ObjectDB::debug_objects(void ()(Object)) in libcore.osx.tools.64s.a(object.osx.tools.64s.o)
FileAccess::create(FileAccess::AccessType) in libcore.osx.tools.64s.a(file_access.osx.tools.64s.o)
DisplayServer::vsync_set_enabled(bool) in libservers.osx.tools.64s.a(display_server.osx.tools.64s.o)
DisplayServer::get_create_function_rendering_drivers(int) in libservers.osx.tools.64s.a(display_server.osx.tools.64s.o)
DisplayServer::create(int, String const&, DisplayServer::WindowMode, unsigned int, Vector2i const&, Error&) in libservers.osx.tools.64s.a(display_server.osx.tools.64s.o)
VulkanContext::_obtain_vulkan_version() in libdrivers.osx.tools.64s.a(vulkan_context.osx.tools.64s.o)
...
"___ubsan_handle_implicit_conversion", referenced from:
handle_crash(int) in crash_handler_osx.osx.tools.64s.o
load_address() in crash_handler_osx.osx.tools.64s.o
CowData<char32_t>::_ref(CowData<char32_t> const&) in crash_handler_osx.osx.tools.64s.o
CowData<char32_t>::_unref(void*) in crash_handler_osx.osx.tools.64s.o
unsigned int std::__1::__cxx_atomic_fetch_sub(std::__1::__cxx_atomic_base_impl, unsigned int, std::__1::memory_order) in crash_handler_osx.osx.tools.64s.o
unsigned int std::__1::__cxx_atomic_load(std::__1::__cxx_atomic_base_impl const
, std::__1::memory_order) in crash_handler_osx.osx.tools.64s.o
bool std::__1::__cxx_atomic_compare_exchange_weak(std::__1::__cxx_atomic_base_impl, unsigned int, unsigned int, std::__1::memory_order, std::__1::memory_order) in crash_handler_osx.osx.tools.64s.o
...
"___ubsan_handle_invalid_builtin", referenced from:
_ZSTD_highbit32 in libcore.osx.tools.64s.a(zstd_compress.osx.tools.64s.o)
_ZSTD_NbCommonBytes in libcore.osx.tools.64s.a(zstd_compress.osx.tools.64s.o)
_ZSTD_highbit32 in libcore.osx.tools.64s.a(zstd_decompress.osx.tools.64s.o)
RandomPCG::randf() in libcore.osx.tools.64s.a(random_number_generator.osx.tools.64s.o)
RandomPCG::randd() in libcore.osx.tools.64s.a(random_pcg.osx.tools.64s.o)
RandomPCG::randf() in libcore.osx.tools.64s.a(random_pcg.osx.tools.64s.o)
_BIT_highbit32 in libcore.osx.tools.64s.a(fse_compress.osx.tools.64s.o)
...
"___ubsan_handle_load_invalid_value", referenced from:
CrashHandler::disable() in crash_handler_osx.osx.tools.64s.o
handle_crash(int) in crash_handler_osx.osx.tools.64s.o
std::__1::__atomic_base<unsigned int, true>::fetch_sub(unsigned int, std::__1::memory_order) in crash_handler_osx.osx.tools.64s.o
unsigned int std::__1::__cxx_atomic_fetch_sub(std::__1::__cxx_atomic_base_impl, unsigned int, std::__1::memory_order) in crash_handler_osx.osx.tools.64s.o
std::__1::__atomic_base<unsigned int, false>::load(std::__1::memory_order) const in crash_handler_osx.osx.tools.64s.o
std::__1::__atomic_base<unsigned int, false>::compare_exchange_weak(unsigned int&, unsigned int, std::__1::memory_order) in crash_handler_osx.osx.tools.64s.o
unsigned int std::__1::__cxx_atomic_load(std::__1::__cxx_atomic_base_impl const
, std::__1::memory_order) in crash_handler_osx.osx.tools.64s.o
...
"___ubsan_handle_missing_return", referenced from:
CompressEtc2Rgba(unsigned int const*, unsigned long long*, unsigned int, unsigned long) in libmodule_etcpak.osx.tools.64s.a(ProcessRGB.osx.tools.64s.o)
ProcessAlpha_ETC2(unsigned char const*) in libmodule_etcpak.osx.tools.64s.a(ProcessRGB.osx.tools.64s.o)
GetMulSel(int) in libmodule_etcpak.osx.tools.64s.a(ProcessRGB.osx.tools.64s.o)
glslang::TInputScanner::scanVersion(int&, EProfile&, bool&) in libmodule_glslang.osx.tools.64s.a(Scan.osx.tools.64s.o)
glslang::TScanContext::tokenize(glslang::TPpContext*, glslang::TParserToken&) in libmodule_glslang.osx.tools.64s.a(Scan.osx.tools.64s.o)
glslang::TIntermediate::findLValueBase(glslang::TIntermTyped const*, bool) in libmodule_glslang.osx.tools.64s.a(Intermediate.osx.tools.64s.o)
glslang::TPpContext::scanHeaderName(glslang::TPpToken*, char) in libmodule_glslang.osx.tools.64s.a(Pp.osx.tools.64s.o)
...
"___ubsan_handle_mul_overflow", referenced from:
DisplayServerOSX::screen_get_position(int) const in display_server_osx.osx.tools.64s.o
DisplayServerOSX::screen_get_usable_rect(int) const in display_server_osx.osx.tools.64s.o
DisplayServerOSX::_create_window(DisplayServer::WindowMode, Rect2i const&) in display_server_osx.osx.tools.64s.o
DisplayServerOSX::window_get_position(int) const in display_server_osx.osx.tools.64s.o
DisplayServerOSX::window_set_position(Vector2i const&, int) in display_server_osx.osx.tools.64s.o
DisplayServerOSX::cursor_set_custom_image(Ref const&, DisplayServer::CursorShape, Vector2 const&) in display_server_osx.osx.tools.64s.o
DisplayServerOSX::set_icon(Ref const&) in display_server_osx.osx.tools.64s.o
...
"___ubsan_handle_negate_overflow", referenced from:
axis_correct(int, int, int) in joypad_osx.osx.tools.64s.o
AudioServer::_driver_process(int, int*) in libservers.osx.tools.64s.a(audio_server.osx.tools.64s.o)
String::num_int64(long long, int, bool) in libcore.osx.tools.64s.a(ustring.osx.tools.64s.o)
String::to_int() const in libcore.osx.tools.64s.a(ustring.osx.tools.64s.o)
String::to_int(char const*, int) in libcore.osx.tools.64s.a(ustring.osx.tools.64s.o)
String::to_int(wchar_t const*, int) in libcore.osx.tools.64s.a(ustring.osx.tools.64s.o)
double built_in_strtod(char const*, char**) in libcore.osx.tools.64s.a(ustring.osx.tools.64s.o)
...
"___ubsan_handle_nonnull_arg", referenced from:
OpusVorbisDecoder::getPCMS16(WebMFrame&, short*, int&) in libmodule_webm.osx.tools.64s.a(OpusVorbisDecoder.osx.tools.64s.o)
OpusVorbisDecoder::getPCMF(WebMFrame&, float*, int&) in libmodule_webm.osx.tools.64s.a(OpusVorbisDecoder.osx.tools.64s.o)
"___ubsan_handle_nullability_arg", referenced from:
OS_OSX::get_unique_id() const in os_osx.osx.tools.64s.o
OS_OSX::shell_open(String) in os_osx.osx.tools.64s.o
OS_OSX::move_to_trash(String const&) in os_osx.osx.tools.64s.o
-[GodotApplication sendEvent:] in display_server_osx.osx.tools.64s.o
-[GodotApplicationDelegate forceUnbundledWindowActivationHackStep1] in display_server_osx.osx.tools.64s.o
-[GodotApplicationDelegate forceUnbundledWindowActivationHackStep2] in display_server_osx.osx.tools.64s.o
-[GodotApplicationDelegate applicationDidFinishLaunching:] in display_server_osx.osx.tools.64s.o
...
"___ubsan_handle_nullability_return_v1", referenced from:
-[GodotContentView init] in display_server_osx.osx.tools.64s.o
-[GodotContentView validAttributesForMarkedText] in display_server_osx.osx.tools.64s.o
MIDIPacketNext(MIDIPacket const*) in libdrivers.osx.tools.64s.a(midi_driver_coremidi.osx.tools.64s.o)
dispatch_get_main_queue() in libmodule_camera.osx.tools.64s.a(camera_osx.osx.tools.64s.o)
"___ubsan_handle_out_of_bounds", referenced from:
handle_crash(int) in crash_handler_osx.osx.tools.64s.o
Variant::clear() in crash_handler_osx.osx.tools.64s.o
void DirAccess::make_default(DirAccess::AccessType) in os_osx.osx.tools.64s.o
remapKey(unsigned int, unsigned int) in display_server_osx.osx.tools.64s.o
translateKey(unsigned int) in display_server_osx.osx.tools.64s.o
DisplayServerOSX::cursor_set_shape(DisplayServer::CursorShape) in display_server_osx.osx.tools.64s.o
DisplayServerOSX::cursor_set_custom_image(Ref const&, DisplayServer::CursorShape, Vector2 const&) in display_server_osx.osx.tools.64s.o
...
"___ubsan_handle_pointer_overflow", referenced from:
handle_crash(int) in crash_handler_osx.osx.tools.64s.o
CowData<char32_t>::_get_refcount() const in crash_handler_osx.osx.tools.64s.o
Variant::clear() in crash_handler_osx.osx.tools.64s.o
CowData::_get_refcount() const in crash_handler_osx.osx.tools.64s.o
CowData<char32_t>::_get_size() const in crash_handler_osx.osx.tools.64s.o
void DirAccess::make_default(DirAccess::AccessType) in os_osx.osx.tools.64s.o
CowData<char32_t>::_get_refcount() const in os_osx.osx.tools.64s.o
...
"___ubsan_handle_shift_out_of_bounds", referenced from:
next_power_of_2(unsigned int) in os_osx.osx.tools.64s.o
remapKey(unsigned int, unsigned int) in display_server_osx.osx.tools.64s.o
sendScrollEvent(int, int, double, int) in display_server_osx.osx.tools.64s.o
DisplayServerOSX::create_sub_window(DisplayServer::WindowMode, unsigned int, Rect2i const&) in display_server_osx.osx.tools.64s.o
DisplayServerOSX::cursor_set_custom_image(Ref const&, DisplayServer::CursorShape, Vector2 const&) in display_server_osx.osx.tools.64s.o
DisplayServerOSX::DisplayServerOSX(String const&, DisplayServer::WindowMode, unsigned int, Vector2i const&, Error&) in display_server_osx.osx.tools.64s.o
next_power_of_2(unsigned int) in display_server_osx.osx.tools.64s.o
...
"___ubsan_handle_sub_overflow", referenced from:
handle_crash(int) in crash_handler_osx.osx.tools.64s.o
String::length() const in crash_handler_osx.osx.tools.64s.o
List<String, DefaultAllocator>::_Data::erase(List<String, DefaultAllocator>::Element const*) in crash_handler_osx.osx.tools.64s.o
Vector<Logger*>::push_back(Logger*) in os_osx.osx.tools.64s.o
String::length() const in os_osx.osx.tools.64s.o
Char16String::length() const in display_server_osx.osx.tools.64s.o
String::length() const in display_server_osx.osx.tools.64s.o
...
"___ubsan_handle_type_mismatch_v1", referenced from:
CrashHandler::CrashHandler() in crash_handler_osx.osx.tools.64s.o
CrashHandler::CrashHandler() in crash_handler_osx.osx.tools.64s.o
CrashHandler::~CrashHandler() in crash_handler_osx.osx.tools.64s.o
CrashHandler::disable() in crash_handler_osx.osx.tools.64s.o
CrashHandler::~CrashHandler() in crash_handler_osx.osx.tools.64s.o
CrashHandler::initialize() in crash_handler_osx.osx.tools.64s.o
handle_crash(int) in crash_handler_osx.osx.tools.64s.o
...
"___ubsan_handle_vla_bound_not_positive", referenced from:
_opus_decode_frame in libmodule_opus.osx.tools.64s.a(opus_decoder.osx.tools.64s.o)
_opus_decode in libmodule_opus.osx.tools.64s.a(opus_decoder.osx.tools.64s.o)
_celt_decode_with_ec in libmodule_opus.osx.tools.64s.a(celt_decoder.osx.tools.64s.o)
_celt_decode_lost in libmodule_opus.osx.tools.64s.a(celt_decoder.osx.tools.64s.o)
_deemphasis in libmodule_opus.osx.tools.64s.a(celt_decoder.osx.tools.64s.o)
_celt_synthesis in libmodule_opus.osx.tools.64s.a(celt_decoder.osx.tools.64s.o)
_silk_Decode in libmodule_opus.osx.tools.64s.a(dec_API.osx.tools.64s.o)
...
"___ubsan_vptr_type_cache", referenced from:
handle_crash(int) in crash_handler_osx.osx.tools.64s.o
OS_OSX::initialize_core() in os_osx.osx.tools.64s.o
OS_OSX::initialize_joypads() in os_osx.osx.tools.64s.o
OS_OSX::initialize() in os_osx.osx.tools.64s.o
OS_OSX::finalize() in os_osx.osx.tools.64s.o
OS_OSX::set_main_loop(MainLoop*) in os_osx.osx.tools.64s.o
OS_OSX::delete_main_loop() in os_osx.osx.tools.64s.o
...
ld: symbol(s) not found for architecture x86_64
clang-11: error: linker command failed with exit code 1 (use -v to see invocation)
scons: *** [bin/godot.osx.tools.64s] Error 1
scons: building terminated because of errors.

Copy link
Contributor

@RevoluPowered RevoluPowered left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixes the problem, if we can support LeakSanitizer in the future on OSX it would be nice, but not something we can fix tonight I don't think.

env.Append(CCFLAGS=["-fsanitize=bounds-strict"])
env.Append(
CCFLAGS=[
"-fsanitize=nullability-return,nullability-arg,function,nullability-assign,implicit-signed-integer-truncation,implicit-unsigned-integer-truncation"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you remove implicit-unsigned-integer-truncation and implicit-signed-integer-truncation from Linux, Mac and Server build?
I was almost sure that I removed that flags, but looks that they still exists here.
They cause error spam just like you show at the top:

thirdparty/misc/pcg.cpp:12:27: runtime error: implicit conversion from type 'unsigned long long' of value 111260437319 (64-bit, unsigned) to type 'uint32_t' (aka 'unsigned int') changed the value to 3886254919 (32-bit, unsigned)
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior thirdparty/misc/pcg.cpp:12:27 in

core/variant/variant.cpp:2852:11: runtime error: implicit conversion from type 'int64_t' (aka 'long long') of value -1 (64-bit, signed) to type 'uint32_t' (aka 'unsigned int') changed the value to 4294967295 (32-bit, unsigned)
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior core/variant/variant.cpp:2852:11 in

thirdparty/nanosvg/nanosvg.h:2190:49: runtime error: division by zero
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior thirdparty/nanosvg/nanosvg.h:2190:49 in

With this flags it is very hard to find real errors in code

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you remove implicit-unsigned-integer-truncation and implicit-signed-integer-truncation from Linux, Mac and Server build?

Done.

@bruvzg bruvzg requested review from a team as code owners April 16, 2021 05:21
@akien-mga akien-mga merged commit dcc82a3 into godotengine:master Apr 16, 2021
@akien-mga
Copy link
Member

Thanks!

@bruvzg bruvzg deleted the macos_sanitizer branch April 16, 2021 07:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants