Clang address sanitizer detects container-overflow on simple web::json::value::parse

[tonyelewis]

Running a test that just contains this line:

::web::json::value::parse( R"([ { "k1" : "v" }, { "k2" : "v" }, { "k3" : "v" }, { "k4" : "v" } ])" );

...through Clang's address sanitizer, I get:

==9344==ERROR: AddressSanitizer: container-overflow on address 0x6030000029f8 at pc 0x00000059e020 bp 0x7fffeea53300 sp 0x7fffeea532f8
READ of size 8 at 0x6030000029f8 thread T0
    #0 0x59e01f in std::__1::unique_ptr<web::json::details::_Value, std::__1::default_delete<web::json::details::_Value> >::reset(web::json::details::_Value*) /opt/clang+llvm-7.0.0-x86_64-linux-gnu-ubuntu-16.04/bin/../include/c++/v1/memory:2631:28
    #1 0x59e01f in std::__1::unique_ptr<web::json::details::_Value, std::__1::default_delete<web::json::details::_Value> >::~unique_ptr() /opt/clang+llvm-7.0.0-x86_64-linux-gnu-ubuntu-16.04/bin/../include/c++/v1/memory:2588
    #2 0x59e01f in web::json::value::~value() ~/source/cpprestsdk-clang/include/cpprest/json.h:71
    #3 0xdd7d00 in std::__1::allocator<web::json::value>::destroy(web::json::value*) /opt/clang+llvm-7.0.0-x86_64-linux-gnu-ubuntu-16.04/bin/../include/c++/v1/memory:1866:64
    #4 0xdd7d00 in void std::__1::allocator_traits<std::__1::allocator<web::json::value> >::__destroy<web::json::value>(std::__1::integral_constant<bool, true>, std::__1::allocator<web::json::value>&, web::json::value*) /opt/clang+llvm-7.0.0-x86_64-linux-gnu-ubuntu-16.04/bin/../include/c++/v1/memory:1728
    #5 0xdd7d00 in void std::__1::allocator_traits<std::__1::allocator<web::json::value> >::destroy<web::json::value>(std::__1::allocator<web::json::value>&, web::json::value*) /opt/clang+llvm-7.0.0-x86_64-linux-gnu-ubuntu-16.04/bin/../include/c++/v1/memory:1596
    #6 0xdd7d00 in std::__1::__vector_base<web::json::value, std::__1::allocator<web::json::value> >::__destruct_at_end(web::json::value*) /opt/clang+llvm-7.0.0-x86_64-linux-gnu-ubuntu-16.04/bin/../include/c++/v1/vector:421
    #7 0xdd7d00 in std::__1::__vector_base<web::json::value, std::__1::allocator<web::json::value> >::clear() /opt/clang+llvm-7.0.0-x86_64-linux-gnu-ubuntu-16.04/bin/../include/c++/v1/vector:364
    #8 0xdd7d00 in std::__1::__vector_base<web::json::value, std::__1::allocator<web::json::value> >::~__vector_base() /opt/clang+llvm-7.0.0-x86_64-linux-gnu-ubuntu-16.04/bin/../include/c++/v1/vector:458
    #9 0xdd7a84 in std::__1::vector<web::json::value, std::__1::allocator<web::json::value> >::~vector() /opt/clang+llvm-7.0.0-x86_64-linux-gnu-ubuntu-16.04/bin/../include/c++/v1/iterator:1427:74
    #10 0x7efd1d648a24 in web::json::array::~array() (~/source/cpprestsdk-clang/lib/libcpprest.so.2.10+0x57ea24)
    #11 0x7efd1d646780 in web::json::details::_Array::~_Array() (~/source/cpprestsdk-clang/lib/libcpprest.so.2.10+0x57c780)
    #12 0x7efd1d6467b8 in web::json::details::_Array::~_Array() (~/source/cpprestsdk-clang/lib/libcpprest.so.2.10+0x57c7b8)
    #13 0x59e131 in std::__1::default_delete<web::json::details::_Value>::operator()(web::json::details::_Value*) const /opt/clang+llvm-7.0.0-x86_64-linux-gnu-ubuntu-16.04/bin/../include/c++/v1/memory:2321:5
    #14 0x59e131 in std::__1::unique_ptr<web::json::details::_Value, std::__1::default_delete<web::json::details::_Value> >::reset(web::json::details::_Value*) /opt/clang+llvm-7.0.0-x86_64-linux-gnu-ubuntu-16.04/bin/../include/c++/v1/memory:2634
    #15 0x59e131 in std::__1::unique_ptr<web::json::details::_Value, std::__1::default_delete<web::json::details::_Value> >::~unique_ptr() /opt/clang+llvm-7.0.0-x86_64-linux-gnu-ubuntu-16.04/bin/../include/c++/v1/memory:2588
    #16 0x59e131 in web::json::value::~value() ~/source/cpprestsdk-clang/include/cpprest/json.h:71
    #17 0x58ae2a in _DOCTEST_ANON_FUNC_4() my-working-directory/clang-address-sanitizer/../desktop/test_desktop/desktop_test.cpp:19:5
    #18 0xaa9e09 in doctest::Context::run() my-working-directory/clang-address-sanitizer/../3rdparty/doctest.h:5367:21
    #19 0xab0b88 in main my-working-directory/clang-address-sanitizer/../utility/test_main/test_main.cpp:30:29
    #20 0x7efd14eccb96 in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x21b96)
    #21 0x4a50a9 in _start (my-working-directory/clang-address-sanitizer/desktop/test-desktop+0x4a50a9)

0x6030000029f8 is located 24 bytes inside of 32-byte region [0x6030000029e0,0x603000002a00)
allocated by thread T0 here:
    #0 0x57c152 in operator new(unsigned long) /tmp/final/llvm.src/projects/compiler-rt/lib/asan/asan_new_delete.cc:106:3
    #1 0xdd9857 in std::__1::__libcpp_allocate(unsigned long, unsigned long) /opt/clang+llvm-7.0.0-x86_64-linux-gnu-ubuntu-16.04/bin/../include/c++/v1/new:252:10
    #2 0xdd9857 in std::__1::allocator<web::json::value>::allocate(unsigned long, void const*) /opt/clang+llvm-7.0.0-x86_64-linux-gnu-ubuntu-16.04/bin/../include/c++/v1/memory:1799
    #3 0xdd9857 in std::__1::allocator_traits<std::__1::allocator<web::json::value> >::allocate(std::__1::allocator<web::json::value>&, unsigned long) /opt/clang+llvm-7.0.0-x86_64-linux-gnu-ubuntu-16.04/bin/../include/c++/v1/memory:1548
    #4 0xdd9857 in std::__1::__split_buffer<web::json::value, std::__1::allocator<web::json::value>&>::__split_buffer(unsigned long, unsigned long, std::__1::allocator<web::json::value>&) /opt/clang+llvm-7.0.0-x86_64-linux-gnu-ubuntu-16.04/bin/../include/c++/v1/__split_buffer:311
    #5 0x7efd1d65d48c in void std::__1::vector<web::json::value, std::__1::allocator<web::json::value> >::__emplace_back_slow_path<web::json::value>(web::json::value&&) (~/source/cpprestsdk-clang/lib/libcpprest.so.2.10+0x59348c)
    #6 0x7efd1d65b326 in web::json::details::JSON_Parser<char>::_ParseArray(web::json::details::JSON_Parser<char>::Token&) (~/source/cpprestsdk-clang/lib/libcpprest.so.2.10+0x591326)
    #7 0x7efd1d656f4b in web::json::details::JSON_Parser<char>::_ParseValue(web::json::details::JSON_Parser<char>::Token&) (~/source/cpprestsdk-clang/lib/libcpprest.so.2.10+0x58cf4b)
    #8 0x7efd1d652670 in web::json::details::JSON_Parser<char>::ParseValue(web::json::details::JSON_Parser<char>::Token&) (~/source/cpprestsdk-clang/lib/libcpprest.so.2.10+0x588670)
    #9 0x7efd1d65113a in web::json::value::parse(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) (~/source/cpprestsdk-clang/lib/libcpprest.so.2.10+0x58713a)
    #10 0x58ae1c in _DOCTEST_ANON_FUNC_4() my-working-directory/clang-address-sanitizer/../desktop/test_desktop/desktop_test.cpp:19:5
    #11 0xaa9e09 in doctest::Context::run() my-working-directory/clang-address-sanitizer/../3rdparty/doctest.h:5367:21
    #12 0xab0b88 in main my-working-directory/clang-address-sanitizer/../utility/test_main/test_main.cpp:30:29
    #13 0x7efd14eccb96 in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x21b96)

HINT: if you don't care about these errors you may set ASAN_OPTIONS=detect_container_overflow=0.
If you suspect a false positive see also: https://github.com/google/sanitizers/wiki/AddressSanitizerContainerOverflow.
SUMMARY: AddressSanitizer: container-overflow /opt/clang+llvm-7.0.0-x86_64-linux-gnu-ubuntu-16.04/bin/../include/c++/v1/memory:2631:28 in std::__1::unique_ptr<web::json::details::_Value, std::__1::default_delete<web::json::details::_Value> >::reset(web::json::details::_Value*)
Shadow bytes around the buggy address:
  0x0c067fff84e0: fa fa 00 00 00 00 fa fa 00 00 00 00 fa fa 00 00
  0x0c067fff84f0: 00 00 fa fa fd fd fd fd fa fa fd fd fd fd fa fa
  0x0c067fff8500: fd fd fd fd fa fa fd fd fd fd fa fa fd fd fd fd
  0x0c067fff8510: fa fa fd fd fd fd fa fa 00 00 00 fa fa fa 00 00
  0x0c067fff8520: 00 fc fa fa 00 00 00 00 fa fa 00 00 00 00 fa fa
=>0x0c067fff8530: 00 00 00 00 fa fa 00 00 00 00 fa fa 00 00 00[fc]
  0x0c067fff8540: fa fa 00 00 00 00 fa fa fa fa fa fa fa fa fa fa
  0x0c067fff8550: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x0c067fff8560: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x0c067fff8570: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x0c067fff8580: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
Shadow byte legend (one shadow byte represents 8 application bytes):
  Addressable:           00
  Partially addressable: 01 02 03 04 05 06 07 
  Heap left redzone:       fa
  Freed heap region:       fd
  Stack left redzone:      f1
  Stack mid redzone:       f2
  Stack right redzone:     f3
  Stack after return:      f5
  Stack use after scope:   f8
  Global redzone:          f9
  Global init order:       f6
  Poisoned by user:        f7
  Container overflow:      fc
  Array cookie:            ac
  Intra object redzone:    bb
  ASan internal:           fe
  Left alloca redzone:     ca
  Right alloca redzone:    cb
  Shadow gap:              cc

I've confirmed that this problem still occurs with a recent commit (7368961).

I'm using:

clang version 7.0.0 (tags/RELEASE_700/final)
Target: x86_64-unknown-linux-gnu
Thread model: posix
InstalledDir: /usr/bin

Ubuntu 18.04.1 LTS
Linux 4.15.0-39-generic #42-Ubuntu SMP Tue Oct 23 15:48:01 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux

[BillyONeal]

Hi there, I just tried this and asan didn't complain. Perhaps there's something different about my setup than yours, here's what I did:

Created a new vcpkg triplets file in ~/cpprestsdk/vcpkg/triplets/x64-linux-asan.cmake with the following content:

set(VCPKG_TARGET_ARCHITECTURE x64)
set(VCPKG_CRT_LINKAGE dynamic)
set(VCPKG_LIBRARY_LINKAGE static)

set(VCPKG_CMAKE_SYSTEM_NAME Linux)

set(CMAKE_C_COMPILER clang)
set(CMAKE_C_FLAGS "-fsanitize=address -g")
set(CMAKE_CXX_COMPILER clang)
set(CMAKE_CXX_FLAGS "-fsanitize=address -g")

From a clean vcpkg, bootstrapped and then ran this:

./vcpkg/vcpkg install zlib openssl boost-system boost-date-time boost-regex websocketpp boost-thread boost-filesystem boost-random boost-chrono boost-interprocess brotli --triplet x64-linux-asan

Created a directory build.asan inside which I ran the following:

cmake -G Ninja -DCMAKE_BUILD_TYPE=Debug -DCMAKE_TOOLCHAIN_FILE=../vcpkg/scripts/buildsystems/vcpkg.cmake -DCMAKE_C_COMPILER=clang -DCMAKE_CXX_COMPILER=clang++ -DVCPKG_TARGET_TRIPLET=x64-linux-asan -DCMAKE_C_FLAGS="-g -fsanitize=address" -DCMAKE_CXX_FLAGS="-g -fsanitize=address" ..

Added a test in Release/tests/functional/json/construction_tests.cpp:

TEST(github_asan_989)
{
    ::web::json::value::parse( R"([ { "k1" : "v" }, { "k2" : "v" }, { "k3" : "v" }, { "k4" : "v" } ])" );
}

Ran:

~/cpprestsdk/build.asan/Release/Binaries$ ./test_runner *json*test*.so

which reported no asan failures.

To make sure asan was working I added a

auto example = new int;
example[1] = 42;

and asan indeed exploded.

Have I missed something?

[ras0219-msft]

@BillyONeal unfortunately, you'll need to use the VCPKG_CXX_FLAGS variables[1] in order to affect the built libraries.

We also don't have any triplet variables to control the compiler; you'll either need to replace the toolchain file (more reliable) or set the CC and CXX environment variables while running vcpkg install (less reliable).

[1] https://github.com/Microsoft/vcpkg/blob/master/docs/users/triplets.md#vcpkg_cxx_flags

[BillyONeal]

@ras0219-msft In this case we're supposing that the bug lies actually in cpprestsdk which would mean that shouldn't actually make a difference. Last time I did this cpprestsdk exploded horribly; I think one of the recent PRs fixed it...

[tonyelewis]

Thanks for the responses. I'm not able to look at this in detail right at the moment.

But in case it's relevant... I'm not using vcpkg at all. I'm building cpprestsdk from sources using commands like:

cmake -GNinja -B${build_dir} -H${release_dir} -DBOOST_ROOT=${boost_root} -DWERROR:BOOL=OFF -DCMAKE_INSTALL_RPATH_USE_LINK_PATH:BOOL=ON -DCMAKE_INSTALL_PREFIX=${install_dir} ${cmake_flags}
ninja -j 2 -C ${build_dir}
ninja -j 2 -C ${build_dir} install

...where, for Clang, the ${cmake_flags} will be set with -DCMAKE_C_COMPILER=/usr/bin/clang -DCMAKE_CXX_COMPILER=/usr/bin/clang++ -DCMAKE_CXX_FLAGS=" -stdlib=libc++ ${CMAKE_CXX_FLAGS} "

Then in the CMakeLists.txt of my project, I'm using find_package( cpprestsdk REQUIRED ) and depending on the target cpprestsdk::cpprest and I'm specifying the cpprestsdk install directory to CMake with the -DCMAKE_PREFIX_PATH.

[BillyONeal]

-stdlib=libc++

I suspect this may be the difference given the "container overflow" message. Will keep you posted.

[BillyONeal]

Hmm when I try this I get link failures because the boost libraries were built targeting libstdc++ rather than libc++. If I try to get libc++ versions with vcpkg that fails because openssl's build doesn't like clang.

Are you saying you built cpprestsdk without any asan support at all targeting libstdc++ and then tried to link it with some project you built against libc++? That would certainly cause things to explode because different sides of the .so boundary wouldn't agree on the layout of vector and friends.

[tonyelewis]

Apologies for the very slow response. It turned out that some other bits of code in my project were necessary to trigger the problem; it took quite a bit of effort to boil it down to this:

#include <vector>

#include <cpprest/json.h>

void f() {
    std::vector<::web::json::value> x;
    x.push_back( ::web::json::value{} );
}

int main() {
    ::web::json::value::parse( R"([ { "a" : "a" }, { "b" : "b" }, { "c" : "c" }, { "d" : "d" } ])" );
}

I think you're right that libc++ is relevant. I'm using a boost and cpprestsdk built against libc++. Here are commands to reproduce on Ubuntu 18.04...

# Install libssl dev files
sudo apt-get install libssl-dev

# Choose a temporary location and make the directories
export ISSUE_898_ROOTDIR=/tmp/cpprestsdk-issue-989
mkdir -p ${ISSUE_898_ROOTDIR}/{cpprestsdk-clang-build,cpprestsdk-clang-install,boost_1_68_0.clang-install,repro,repro-build}

# Build clang/libc++ Boost
wget "https://dl.bintray.com/boostorg/release/1.68.0/source/boost_1_68_0.tar.gz" -O ${ISSUE_898_ROOTDIR}/boost_1_68_0.tar.gz
tar -zxvf ${ISSUE_898_ROOTDIR}/boost_1_68_0.tar.gz --directory ${ISSUE_898_ROOTDIR}
cd ${ISSUE_898_ROOTDIR}/boost_1_68_0/
./bootstrap.sh --prefix=${ISSUE_898_ROOTDIR}/boost_1_68_0.clang-install
./b2 -j8  --with-atomic --with-chrono --with-date_time --with-filesystem --with-random --with-regex --with-system --with-thread toolset=clang cxxflags="-stdlib=libc++ -std=c++14" linkflags="-stdlib=libc++ -std=c++14" --layout=tagged variant=release dll-path=${ISSUE_898_ROOTDIR}/boost_1_68_0.clang-install/lib
./b2 -j8  --with-atomic --with-chrono --with-date_time --with-filesystem --with-random --with-regex --with-system --with-thread toolset=clang cxxflags="-stdlib=libc++ -std=c++14" linkflags="-stdlib=libc++ -std=c++14" --layout=tagged variant=release dll-path=${ISSUE_898_ROOTDIR}/boost_1_68_0.clang-install/lib install

# Build clang/libc++ cpprest
git clone --recurse-submodules https://github.com/microsoft/cpprestsdk.git ${ISSUE_898_ROOTDIR}/cpprestsdk-master
cmake -GNinja -H${ISSUE_898_ROOTDIR}/cpprestsdk-master -B${ISSUE_898_ROOTDIR}/cpprestsdk-clang-build -DBOOST_ROOT=${ISSUE_898_ROOTDIR}/boost_1_68_0.clang-install -DCMAKE_INSTALL_RPATH_USE_LINK_PATH:BOOL=ON -DCMAKE_INSTALL_PREFIX=${ISSUE_898_ROOTDIR}/cpprestsdk-clang-install -DCMAKE_C_COMPILER=/usr/bin/clang -DCMAKE_CXX_COMPILER=/usr/bin/clang++ -DCMAKE_CXX_FLAGS=" -stdlib=libc++ ${CMAKE_CXX_FLAGS} "
ninja -C ${ISSUE_898_ROOTDIR}/cpprestsdk-clang-build
ninja -C ${ISSUE_898_ROOTDIR}/cpprestsdk-clang-build install

# Write cpp file, compile it with address sanitizer and execute the result
echo '#include <vector>\n#include <cpprest/json.h>\nvoid f() {\n    std::vector<::web::json::value> x;\n    x.push_back( ::web::json::value{} );\n}\nint main() {\n    ::web::json::value::parse( R"([ { "a" : "a" }, { "b" : "b" }, { "c" : "c" }, { "d" : "d" } ])" );\n}\n' > ${ISSUE_898_ROOTDIR}/repro-build/cpprest-issue898.cpp
clang++ -std=c++14 -isystem ${ISSUE_898_ROOTDIR}/cpprestsdk-clang-install/include -stdlib=libc++ -fsanitize=address -fsanitize=undefined -fno-omit-frame-pointer -g ${ISSUE_898_ROOTDIR}/repro-build/cpprest-issue898.cpp -g -o ${ISSUE_898_ROOTDIR}/repro-build/cpprest-issue898.clang_bin -Wl,-rpath,${ISSUE_898_ROOTDIR}/cpprestsdk-clang-install/lib ${ISSUE_898_ROOTDIR}/cpprestsdk-clang-install/lib/libcpprest.so.2.10 -lpthread /usr/lib/x86_64-linux-gnu/libssl.so /usr/lib/x86_64-linux-gnu/libcrypto.so
${ISSUE_898_ROOTDIR}/repro-build/cpprest-issue898.clang_bin

[BillyONeal]

@tonyelewis No need to apologize -- thanks for reducing it to a repro we can test!

I can confirm that it repros for me now. Will keep you posted -- I trying to get cmake to generate debug info now :)

[BillyONeal]

Now if I could only get this thing to give me a stack trace lol...

billy@BION-P1:~/cpprestsdk-issue-989$ ./repro-build/cpprest-issue898.clang_bin                                          =================================================================
==485==ERROR: AddressSanitizer: container-overflow on address 0x6030000002c8 at pc 0x0000005196fc bp 0x7fff4b081e50 sp 0x7fff4b081e48
READ of size 8 at 0x6030000002c8 thread T0                                                                                  #0 0x5196fb  (/home/billy/cpprestsdk-issue-989/repro-build/cpprest-issue898.clang_bin+0x5196fb)
    #1 0x51a381  (/home/billy/cpprestsdk-issue-989/repro-build/cpprest-issue898.clang_bin+0x51a381)
    #2 0x519d14  (/home/billy/cpprestsdk-issue-989/repro-build/cpprest-issue898.clang_bin+0x519d14)
    #3 0x7f6a409ca034  (/home/billy/cpprestsdk-issue-989/cpprestsdk-clang-install/lib/libcpprest.so.2.10+0x614034)          #4 0x7f6a409c7d90  (/home/billy/cpprestsdk-issue-989/cpprestsdk-clang-install/lib/libcpprest.so.2.10+0x611d90)          #5 0x7f6a409c7dc8  (/home/billy/cpprestsdk-issue-989/cpprestsdk-clang-install/lib/libcpprest.so.2.10+0x611dc8)          #6 0x519cad  (/home/billy/cpprestsdk-issue-989/repro-build/cpprest-issue898.clang_bin+0x519cad)
    #7 0x519287  (/home/billy/cpprestsdk-issue-989/repro-build/cpprest-issue898.clang_bin+0x519287)
    #8 0x7f6a3e7bab96  (/lib/x86_64-linux-gnu/libc.so.6+0x21b96)
    #9 0x41aed9  (/home/billy/cpprestsdk-issue-989/repro-build/cpprest-issue898.clang_bin+0x41aed9)

0x6030000002c8 is located 24 bytes inside of 32-byte region [0x6030000002b0,0x6030000002d0)                             allocated by thread T0 here:                                                                                                #0 0x513340  (/home/billy/cpprestsdk-issue-989/repro-build/cpprest-issue898.clang_bin+0x513340)
    #1 0x51f7e6  (/home/billy/cpprestsdk-issue-989/repro-build/cpprest-issue898.clang_bin+0x51f7e6)
    #2 0x7f6a409e0bcc  (/home/billy/cpprestsdk-issue-989/cpprestsdk-clang-install/lib/libcpprest.so.2.10+0x62abcc)          #3 0x7f6a409deb26  (/home/billy/cpprestsdk-issue-989/cpprestsdk-clang-install/lib/libcpprest.so.2.10+0x628b26)          #4 0x7f6a409da80b  (/home/billy/cpprestsdk-issue-989/cpprestsdk-clang-install/lib/libcpprest.so.2.10+0x62480b)          #5 0x7f6a409d47a0  (/home/billy/cpprestsdk-issue-989/cpprestsdk-clang-install/lib/libcpprest.so.2.10+0x61e7a0)          #6 0x7f6a409d2b9a  (/home/billy/cpprestsdk-issue-989/cpprestsdk-clang-install/lib/libcpprest.so.2.10+0x61cb9a)          #7 0x519279  (/home/billy/cpprestsdk-issue-989/repro-build/cpprest-issue898.clang_bin+0x519279)
    #8 0x7f6a3e7bab96  (/lib/x86_64-linux-gnu/libc.so.6+0x21b96)

HINT: if you don't care about these errors you may set ASAN_OPTIONS=detect_container_overflow=0.
If you suspect a false positive see also: https://github.com/google/sanitizers/wiki/AddressSanitizerContainerOverflow.  SUMMARY: AddressSanitizer: container-overflow (/home/billy/cpprestsdk-issue-989/repro-build/cpprest-issue898.clang_bin+0x5196fb)
Shadow bytes around the buggy address:
  0x0c067fff8000: fa fa 00 00 00 fa fa fa 00 00 00 fa fa fa 00 00                                                         0x0c067fff8010: 00 00 fa fa 00 00 00 00 fa fa 00 00 00 00 fa fa                                                         0x0c067fff8020: 00 00 00 00 fa fa 00 00 00 00 fa fa 00 00 00 00                                                         0x0c067fff8030: fa fa 00 00 00 00 fa fa 00 00 00 00 fa fa 00 00                                                         0x0c067fff8040: 00 00 fa fa 00 00 00 00 fa fa 00 00 00 00 fa fa                                                       =>0x0c067fff8050: 00 00 00 00 fa fa 00 00 00[fc]fa fa 00 00 00 00                                                         0x0c067fff8060: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa                                                         0x0c067fff8070: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa                                                         0x0c067fff8080: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa                                                         0x0c067fff8090: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa                                                         0x0c067fff80a0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa                                                       Shadow byte legend (one shadow byte represents 8 application bytes):
  Addressable:           00
  Partially addressable: 01 02 03 04 05 06 07
  Heap left redzone:       fa                                                                                             Freed heap region:       fd                                                                                             Stack left redzone:      f1                                                                                             Stack mid redzone:       f2                                                                                             Stack right redzone:     f3                                                                                             Stack after return:      f5                                                                                             Stack use after scope:   f8                                                                                             Global redzone:          f9                                                                                             Global init order:       f6                                                                                             Poisoned by user:        f7                                                                                             Container overflow:      fc                                                                                             Array cookie:            ac                                                                                             Intra object redzone:    bb                                                                                             ASan internal:           fe                                                                                             Left alloca redzone:     ca                                                                                             Right alloca redzone:    cb                                                                                           ==485==ABORTING
billy@BION-P1:~/cpprestsdk-issue-989$

[BillyONeal]

I think this is a false positive -- the sanitizer is reporting an invalid container read, but we're in the container's destructor. All that code comes from the standard library. Your repro does not build cpprestsdk with sanitizers enabled and https://github.com/google/sanitizers/wiki/AddressSanitizerContainerOverflow indicates that such a configuration can create these kinds of false positives because code in the unsanitized libraries doesn't update asan's view of the end of the container appropriately.

I'm trying to build at least cpprestsdk with sanitizers on and see if that works...

[BillyONeal]

I confirm that if I add -fsanitize=undefined -fsanitize=address to both boost and cpprestsdk in your example repro, no failures are detected.

[tonyelewis]

Ah. Thanks very much for your very quick response and for figuring it out. Apologies that it was ultimately all just a false positive and that I didn't read the AddressSanitizerContainerOverflow doc sufficiently.

[BillyONeal]

No problem! I didn't read it either -- the "WTF" didn't happen until I finally got the symbolizer working and saw all the code was owned by the standard library.