absl::flat_hash_set fails UBSan on clang 7

[DarkDimius]

absl::flat_hash_set<std::string> set = {
    "s1", "s2", "s3", "s4", "s5", "s6",
    "s7", "s8", "s9", "s10", "s11", "s12",
    "s13", "s14", "s15"};

compiled with http://releases.llvm.org/7.0.0/clang+llvm-7.0.0-x86_64-apple-darwin.tar.xz

build:sanitize --copt=-fsanitize=address --copt=-fsanitize-address-use-after-scope
build:sanitize --cxxopt=-fsanitize=address --cxxopt=-fsanitize-address-use-after-scope
build:sanitize --linkopt=-fsanitize=address --linkopt=-fsanitize-address-use-after-scope
build:sanitize --copt=-fsanitize=undefined --copt=-fno-sanitize-recover=undefined
build:sanitize --cxxopt=-fsanitize=undefined --copt=-fno-sanitize-recover=undefined
build:sanitize --linkopt=-fsanitize=undefined --copt=-fno-sanitize-recover=undefined

fails with the following backtrace

* thread #1, queue = 'com.apple.main-thread', stop reason = Misaligned pointer use
  * frame #0: 0x000000010abd0360 libclang_rt.asan_osx_dynamic.dylib`__ubsan_on_report
    frame #1: 0x000000010abcb1cc libclang_rt.asan_osx_dynamic.dylib`__ubsan::Diag::~Diag() + 140
    frame #2: 0x000000010abccb60 libclang_rt.asan_osx_dynamic.dylib`handleTypeMismatchImpl(__ubsan::TypeMismatchData*, unsigned long, __ubsan::ReportOptions) + 1216
    frame #3: 0x000000010abccca4 libclang_rt.asan_osx_dynamic.dylib`__ubsan_handle_type_mismatch_v1_abort + 68
    frame #4: 0x000000010018ec20 failure_demo`absl::little_endian::Load64(p=0x0000000107036502) at endian.h:198
    frame #5: 0x000000010018de8f v`absl::hash_internal::CityHashState::Read9To16(p="parse-tree", len=10) at hash.h:757
    frame #6: 0x000000010018d861 failure_demo`absl::hash_internal::CityHashState::CombineContiguousImpl(state=4417671936, first="parse-tree", len=10, (null)=integral_constant<int, 8> @ 0x00007ffeefbfa890) at hash.h:841
    frame #7: 0x000000010018d63c failure_demo`absl::hash_internal::CityHashState::combine_contiguous(hash_state=(state_ = 4417671936), first="parse-tree", size=10) at hash.h:703
    frame #8: 0x000000010018f06b failure_demo`std::__1::enable_if<is_uniquely_represented<char>::value, absl::hash_internal::CityHashState>::type absl::hash_internal::hash_range_or_bytes<absl::hash_internal::CityHashState, char>(hash_state=(state_ = 4417671936), data="parse-tree", size=10) at hash.h:517
    frame #9: 0x000000010018cdd2 failure_demo`absl::hash_internal::CityHashState absl::hash_internal::HashStateBase<absl::hash_internal::CityHashState>::combine_contiguous<char>(state=(state_ = 4417671936), data="parse-tree", size=10) at hash.h:889
    frame #10: 0x000000010018c567 failure_demo`absl::hash_internal::CityHashState absl::hash_internal::AbslHashValue<absl::hash_internal::CityHashState>(hash_state=(state_ = 4417671936), str=(__data = "parse-tree", __size = 10)) at hash.h:354
    frame #11: 0x000000010018bff0 failure_demo`std::__1::enable_if<(HashSelect<std::__1::basic_string_view<char, std::__1::char_traits<char> > >::value) == ((absl::hash_internal::InvokeHashTag)1), absl::hash_internal::CityHashState>::type absl::hash_internal::InvokeHash<absl::hash_internal::CityHashState, std::__1::basic_string_view<char, std::__1::char_traits<char> > >(state=(state_ = 4417671936), value=0x00007ffeefbfb460) at hash.h:655
    frame #12: 0x000000010018b960 failure_demo`absl::hash_internal::CityHashState absl::hash_internal::HashStateBase<absl::hash_internal::CityHashState>::combine<std::__1::basic_string_view<char, std::__1::char_traits<char> > >(state=(state_ = 4417671936), value=0x00007ffeefbfb460) at hash.h:881
    frame #13: 0x000000010018b62f failure_demo`unsigned long absl::hash_internal::CityHashState::hash<std::__1::basic_string_view<char, std::__1::char_traits<char> >, 0>(value=0x00007ffeefbfb460) at hash.h:723
    frame #14: 0x000000010018b49f failure_demo`absl::hash_internal::HashImpl<std::__1::basic_string_view<char, std::__1::char_traits<char> > >::operator(this=0x00007ffeefbfb480, value=0x00007ffeefbfb460)(std::__1::basic_string_view<char, std::__1::char_traits<char> > const&) const at hash.h:871
    frame #15: 0x00000001001899e9 failure_demo`absl::container_internal::StringHash::operator(this=0x0000000108c18f00, v=(__data = "parse-tree", __size = 10))(std::__1::basic_string_view<char, std::__1::char_traits<char> >) const at hash_function_defaults.h:72

last executed C++ code is

frame #4: 0x000000010018ec20 failure_demo`absl::little_endian::Load64(p=0x0000000107036502) at endian.h:198
   197 	inline uint64_t Load64(const void *p) {
-> 198 	  return ToHost64(ABSL_INTERNAL_UNALIGNED_LOAD64(p));
   199 	}

[DarkDimius]

adding -DADDRESS_SANITIZER to build fixes it. Should it be auto-detected via __has_feature(address_sanitizer)?

[JonathanDCohen]

I'll inspect this locally. Looks like we may just need the define but I want to check with the swisstable authors to make sure this isn't more serious

[JonathanDCohen]

Just checking, this is Apple Clang? Could you let us know the platform? Looks like we're doing an unaligned load which may be UB on certain platforms.

[EricWF]

Whats the actual failure, not just the backtrace?

…

On Thu., Dec. 6, 2018, 4:16 p.m. Jonathan Cohen ***@***.*** wrote: Just checking, this is Apple Clang? Could you let us know the platform? Looks like we're doing an unaligned load which may be UB on certain platforms. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#228 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABHza_JsRQs5lvWBDBhrWRdMiTMHfB9Cks5u2YkfgaJpZM4Y4-_q> .

[JonathanDCohen]

Closing due to lack of activity. Please feel free to re-open :)

[DarkDimius]

Just checking, this is Apple Clang?

this is clang published by http://releases.llvm.org/

[DarkDimius]

@JonathanDCohen , feel free to reopen, I've defined that constant in our build and this fixed the problem for us.

[EricWF]

I'm working on this. It should be fixed and on github by the end of the week.