cross compiling for QNX segfaults

[briansoe66]

When cross compile master on QNX, I get a segmentation fault from container_node_sizes_impl.hpp

CMake command with toolchain:
qnx_toolchain.zip

cmake .. -DCMAKE_TOOLCHAIN_FILE=~/Downloads/qnx_toolchain.cmake -DCMAKE_INSTALL_PREFIX=/home/brian/qnx700/target/qnx7/x86/usr/local -DCMAKE_BUILD_TYPE=Release -DCMAKE_VERBOSE_MAKEFILE=ON -DFOONATHAN_MEMORY_BUILD_EXAMPLES=OFF -DFOONATHAN_MEMORY_BUILD_TESTS=OFF -DFOONATHAN_MEMORY_BUILD_TOOLS=ON

Output:
[ 13%] Generating container_node_sizes_impl.hpp cd /home/brian/Downloads/foonathan_memory/build/src && ../tool/nodesize_dbg --code --alignof "FOONATHAN_ALIGNOF(T)" /home/brian/Downloads/foonathan_memory/build/src/container_node_sizes_impl.hpp Segmentation fault (core dumped) src/CMakeFiles/foonathan_memory.dir/build.make:64: recipe for target 'src/container_node_sizes_impl.hpp' failed make[2]: *** [src/container_node_sizes_impl.hpp] Error 139 make[2]: Leaving directory '/home/brian/Downloads/foonathan_memory/build' CMakeFiles/Makefile2:88: recipe for target 'src/CMakeFiles/foonathan_memory.dir/all' failed make[1]: *** [src/CMakeFiles/foonathan_memory.dir/all] Error 2 make[1]: Leaving directory '/home/brian/Downloads/foonathan_memory/build' Makefile:132: recipe for target 'all' failed make: *** [all] Error 2
build_output.txt

[MiguelCompany]

Looking at your toolchain file, it seems that you are building for an x86 system. If you are cross compiling from an x64 system, you have two options, I think.

1. Allowing your x64 system to run 32bit applications. Some information here
2. Using an emulator, by adding set(CMAKE_CROSSCOMPILING_EMULATOR /path/to/emulator) to your toolchain

[foonathan]

This doesn't seem to be an issue in memory itself.

[cjdesno]

in case anybody else stumbles across this issue in future with the same problem:

let's look at the build as a 3-step process.

1. it first builds tool/nodesize_dbg,
2. then runs nodesize_dbg to determine some properties of the system and generate the include/foonathan/memory/detail/container_node_sizes_impl.hpp headerfile,
3. and finally compiles the rest of the library now that all headers are present.

when cross-compiling, step 2 will fail, as it is attempting to run the nodesize_dbg binary (built for the target system) on the host system rather than the target.

the simplest workaround for this is the following:

1. run the build (it will fail, as seen at the top of this thread)
2. when the build fails, copy the nodesize_dbg binary from the host's build directory to the target, then run it with the appropriate arguments to generate the necessary header file (you can find the correct invocation in the build log output or the cmake files)
3. copy the generated headerfile back from the target to the correct location in the project directory on the host
4. comment out the cmake line that runs nodesize_dbg
5. rerun the build

(i'm sure there's a cleaner way to do it, but this approach works well enough for a one-off build)

(p.s.: if you instead changed up the cmake files to build nodesize_dbg for the host, allowing it to run successfully, the information it would gather from the host system is likely significantly different from the target environment. this would cause the build to succeed but likely lead to runtime errors in any applications linked against the library, so we don't want to do this.)

[razr]

Hello @foonathan,
I have the same issue with the VxWorks cross-compilation as well :)
I think it could be handled a little bit more gently, e.g. if CMAKE_CROSSCOMPILING_EMULATOR is not defined - don't run it at all, it will fail anyway, and also probably check if container_node_sizes_impl.hpp is already present, don't run this check as @cjdesno has suggested?
I'm not sure whether it makes sense to provide you e.g. with an autogenerated version for VxWorks.
If you are ready to accept it, I'd be happy to do so.
Kind Regards,
Andrei

[cjdesno]

@razr oh, to be clear, i have no suggestion for a better way the project build could handle this case; my comment was just for a user workaround to this problem. for example, one issue with changing the project code to not generate the header if it already exists, is that when the header-generation code is updated, an incremental build would fail to properly regenerate a new header. so i think it would be difficult to streamline it more than it already is, so long as this header generation stage remains part of the build.

[foonathan]

I used to disable the entire node size machinery when cross compiling precisely to catch such issues, but that makes it impossible to use it in cross compiling at all. I'm open for PRs that improve the behavior, as I don't know anything about cross compiling with CMake.

[razr]

Hello @foonathan,
I have provided a fix, it works on my Ubuntu 18.04 while native and cross-compilation.
However, I see this issue later on:

[ 45%] Building CXX object example/CMakeFiles/foonathan_memory_example_tracking.dir/tracking.cpp.obj
/home/akholodn/github/razr/memory/example/tracking.cpp:53:70: error: no member named 'set_node_size' in namespace 'foonathan::memory'
                                       memory::memory_pool<>(memory::set_node_size<int>::value,
                                                             ~~~~~~~~^
/home/akholodn/github/razr/memory/example/tracking.cpp:53:87: error: expected '(' for function-style cast or type construction
                                       memory::memory_pool<>(memory::set_node_size<int>::value,
                                                                                   ~~~^
/home/akholodn/github/razr/memory/example/tracking.cpp:53:90: error: no member named 'value' in the global namespace
                                       memory::memory_pool<>(memory::set_node_size<int>::value,
                                                                                       ~~^

[foonathan]

It seems it didn't generate the node size header from the tool correctly. But as your PR works on CI, it might be a local issue. Have you tried clearing your build folder and running everything again?

[razr]

Native compile on my Ubuntu 18.04 machine fails somewhere here.
I'm on your latest changes.

akholodn@razilix:~/github/razr/memory/build$ git log
commit de1a887d2049b7dc92efd38deae9fad230b06528 (HEAD -> master, upstream/master, origin/master, origin/HEAD)
Author: Jonathan Müller <git@foonathan.net>
Date:   Fri May 29 18:34:13 2020 +0200

    Fix tracked_allocator for allocators without block allocators

    Fixes #79.

akholodn@razilix:~/github/razr/memory/build$ cmake ..
-- The C compiler identification is GNU 8.4.0
-- The CXX compiler identification is GNU 8.4.0
-- Check for working C compiler: /usr/bin/cc
-- Check for working C compiler: /usr/bin/cc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Check for working CXX compiler: /usr/bin/c++
-- Check for working CXX compiler: /usr/bin/c++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Configuring done
-- Generating done
-- Build files have been written to: /home/akholodn/github/razr/memory/build
akholodn@razilix:~/github/razr/memory/build$ make
Scanning dependencies of target foonathan_memory_node_size_debugger
[  1%] Building CXX object tool/CMakeFiles/foonathan_memory_node_size_debugger.dir/node_size_debugger.cpp.o
[  3%] Linking CXX executable nodesize_dbg
[  3%] Built target foonathan_memory_node_size_debugger
[  5%] Generating container_node_sizes_impl.hpp
Scanning dependencies of target foonathan_memory
[  7%] Building CXX object src/CMakeFiles/foonathan_memory.dir/detail/align.cpp.o
[  8%] Building CXX object src/CMakeFiles/foonathan_memory.dir/detail/debug_helpers.cpp.o
[ 10%] Building CXX object src/CMakeFiles/foonathan_memory.dir/detail/assert.cpp.o
[ 12%] Building CXX object src/CMakeFiles/foonathan_memory.dir/detail/free_list.cpp.o
[ 14%] Building CXX object src/CMakeFiles/foonathan_memory.dir/detail/free_list_array.cpp.o
[ 16%] Building CXX object src/CMakeFiles/foonathan_memory.dir/detail/small_free_list.cpp.o
[ 17%] Building CXX object src/CMakeFiles/foonathan_memory.dir/debugging.cpp.o
[ 19%] Building CXX object src/CMakeFiles/foonathan_memory.dir/error.cpp.o
[ 21%] Building CXX object src/CMakeFiles/foonathan_memory.dir/heap_allocator.cpp.o
[ 23%] Building CXX object src/CMakeFiles/foonathan_memory.dir/iteration_allocator.cpp.o
[ 25%] Building CXX object src/CMakeFiles/foonathan_memory.dir/malloc_allocator.cpp.o
[ 26%] Building CXX object src/CMakeFiles/foonathan_memory.dir/memory_arena.cpp.o
[ 28%] Building CXX object src/CMakeFiles/foonathan_memory.dir/memory_pool.cpp.o
[ 30%] Building CXX object src/CMakeFiles/foonathan_memory.dir/memory_pool_collection.cpp.o
[ 32%] Building CXX object src/CMakeFiles/foonathan_memory.dir/memory_stack.cpp.o
[ 33%] Building CXX object src/CMakeFiles/foonathan_memory.dir/new_allocator.cpp.o
[ 35%] Building CXX object src/CMakeFiles/foonathan_memory.dir/static_allocator.cpp.o
[ 37%] Building CXX object src/CMakeFiles/foonathan_memory.dir/temporary_allocator.cpp.o
[ 39%] Building CXX object src/CMakeFiles/foonathan_memory.dir/virtual_memory.cpp.o
[ 41%] Linking CXX static library libfoonathan_memory-0.6.2.a
[ 41%] Built target foonathan_memory
Scanning dependencies of target foonathan_memory_example_joint_allocation
[ 42%] Building CXX object example/CMakeFiles/foonathan_memory_example_joint_allocation.dir/joint_allocation.cpp.o
[ 44%] Linking CXX executable foonathan_memory_example_joint_allocation
[ 44%] Built target foonathan_memory_example_joint_allocation
Scanning dependencies of target foonathan_memory_example_allocator_storage
[ 46%] Building CXX object example/CMakeFiles/foonathan_memory_example_allocator_storage.dir/allocator_storage.cpp.o
[ 48%] Linking CXX executable foonathan_memory_example_allocator_storage
[ 48%] Built target foonathan_memory_example_allocator_storage
Scanning dependencies of target foonathan_memory_example_taking_allocators
[ 50%] Building CXX object example/CMakeFiles/foonathan_memory_example_taking_allocators.dir/taking_allocators.cpp.o
[ 51%] Linking CXX executable foonathan_memory_example_taking_allocators
[ 51%] Built target foonathan_memory_example_taking_allocators
Scanning dependencies of target foonathan_memory_example_tracking
[ 53%] Building CXX object example/CMakeFiles/foonathan_memory_example_tracking.dir/tracking.cpp.o
[ 55%] Linking CXX executable foonathan_memory_example_tracking
[ 55%] Built target foonathan_memory_example_tracking
Scanning dependencies of target foonathan_memory_example_using_allocators
[ 57%] Building CXX object example/CMakeFiles/foonathan_memory_example_using_allocators.dir/using_allocators.cpp.o
[ 58%] Linking CXX executable foonathan_memory_example_using_allocators
[ 58%] Built target foonathan_memory_example_using_allocators
Scanning dependencies of target foonathan_memory_test
[ 60%] Building CXX object test/CMakeFiles/foonathan_memory_test.dir/test.cpp.o
[ 62%] Building CXX object test/CMakeFiles/foonathan_memory_test.dir/detail/align.cpp.o
[ 64%] Building CXX object test/CMakeFiles/foonathan_memory_test.dir/detail/debug_helpers.cpp.o
[ 66%] Building CXX object test/CMakeFiles/foonathan_memory_test.dir/detail/free_list.cpp.o
[ 67%] Building CXX object test/CMakeFiles/foonathan_memory_test.dir/detail/free_list_array.cpp.o
[ 69%] Building CXX object test/CMakeFiles/foonathan_memory_test.dir/detail/ilog2.cpp.o
[ 71%] Building CXX object test/CMakeFiles/foonathan_memory_test.dir/detail/memory_stack.cpp.o
[ 73%] Building CXX object test/CMakeFiles/foonathan_memory_test.dir/aligned_allocator.cpp.o
[ 75%] Building CXX object test/CMakeFiles/foonathan_memory_test.dir/allocator_traits.cpp.o
[ 76%] Building CXX object test/CMakeFiles/foonathan_memory_test.dir/default_allocator.cpp.o
[ 78%] Building CXX object test/CMakeFiles/foonathan_memory_test.dir/fallback_allocator.cpp.o
[ 80%] Building CXX object test/CMakeFiles/foonathan_memory_test.dir/iteration_allocator.cpp.o
[ 82%] Building CXX object test/CMakeFiles/foonathan_memory_test.dir/joint_allocator.cpp.o
[ 83%] Building CXX object test/CMakeFiles/foonathan_memory_test.dir/memory_arena.cpp.o
[ 85%] Building CXX object test/CMakeFiles/foonathan_memory_test.dir/memory_pool.cpp.o
[ 87%] Building CXX object test/CMakeFiles/foonathan_memory_test.dir/memory_pool_collection.cpp.o
[ 89%] Building CXX object test/CMakeFiles/foonathan_memory_test.dir/memory_resource_adapter.cpp.o
In file included from /home/akholodn/github/razr/memory/include/foonathan/memory/memory_resource_adapter.hpp:23,
                 from /home/akholodn/github/razr/memory/test/memory_resource_adapter.cpp:5:
/usr/include/c++/8/experimental/memory_resource:233:11: error: expected primary-expression before ‘auto’
  decltype(auto)
           ^~~~
/usr/include/c++/8/experimental/memory_resource:233:11: error: expected ‘)’ before ‘auto’
  decltype(auto)
          ~^~~~
           )
/usr/include/c++/8/experimental/memory_resource:233:11: error: expected primary-expression before ‘auto’
  decltype(auto)
           ^~~~
/usr/include/c++/8/experimental/memory_resource:233:11: error: expected primary-expression before ‘auto’
/usr/include/c++/8/experimental/memory_resource:233:11: error: expected primary-expression before ‘auto’
/usr/include/c++/8/experimental/memory_resource:233:11: error: expected primary-expression before ‘auto’
/usr/include/c++/8/experimental/memory_resource:233:2: error: expected unqualified-id before ‘decltype’
  decltype(auto)
  ^~~~~~~~
/usr/include/c++/8/experimental/memory_resource:239:11: error: expected primary-expression before ‘auto’
  decltype(auto)
           ^~~~
/usr/include/c++/8/experimental/memory_resource:239:11: error: expected ‘)’ before ‘auto’
  decltype(auto)
          ~^~~~
           )
/usr/include/c++/8/experimental/memory_resource:239:11: error: expected primary-expression before ‘auto’
  decltype(auto)
           ^~~~
/usr/include/c++/8/experimental/memory_resource:239:11: error: expected primary-expression before ‘auto’
/usr/include/c++/8/experimental/memory_resource:239:11: error: expected primary-expression before ‘auto’
/usr/include/c++/8/experimental/memory_resource:239:11: error: expected primary-expression before ‘auto’
/usr/include/c++/8/experimental/memory_resource:239:2: error: expected unqualified-id before ‘decltype’
  decltype(auto)
  ^~~~~~~~
test/CMakeFiles/foonathan_memory_test.dir/build.make:446: recipe for target 'test/CMakeFiles/foonathan_memory_test.dir/memory_resource_adapter.cpp.o' failed
make[2]: *** [test/CMakeFiles/foonathan_memory_test.dir/memory_resource_adapter.cpp.o] Error 1
CMakeFiles/Makefile2:353: recipe for target 'test/CMakeFiles/foonathan_memory_test.dir/all' failed
make[1]: *** [test/CMakeFiles/foonathan_memory_test.dir/all] Error 2
Makefile:140: recipe for target 'all' failed
make: *** [all] Error 2

[razr]

And the cross-compilation fails here. This is most probably because of FOONATHAN_MEMORY_NO_NODE_SIZE is defined but not added to the memory/example/tracking.cpp? #ifdef FOONATHAN_MEMORY_NO_NODE_SIZE do_something different?

akholodn@razilix:~/github/razr/memory/vxworks-build$ cat ./src/container_node_sizes_impl.hpp
#define FOONATHAN_MEMORY_NO_NODE_SIZE

akholodn@razilix:~/github/razr/memory/vxworks-build$ cmake  .. -DCMAKE_TOOLCHAIN_FILE=$HOME/stash/users/akholodn/cmake-test/buildspecs/cmake/toolchain.cmake
-- The C compiler identification is Clang 9.0.1
-- The CXX compiler identification is Clang 9.0.1
-- Check for working C compiler: /home/akholodn/Downloads/wrsdk-vxworks7-qemu/toolkit/host_tools/x86_64-linux/bin/wr-cc
-- Check for working C compiler: /home/akholodn/Downloads/wrsdk-vxworks7-qemu/toolkit/host_tools/x86_64-linux/bin/wr-cc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Check for working CXX compiler: /home/akholodn/Downloads/wrsdk-vxworks7-qemu/toolkit/host_tools/x86_64-linux/bin/wr-c++
-- Check for working CXX compiler: /home/akholodn/Downloads/wrsdk-vxworks7-qemu/toolkit/host_tools/x86_64-linux/bin/wr-c++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
CMake Warning at src/CMakeLists.txt:93 (message):
  cross-compiling, but emulator is not defined, cannot generate
  container_node_sizes_impl.hpp, node size information will be unavailable


-- Configuring done
-- Generating done
-- Build files have been written to: /home/akholodn/github/razr/memory/vxworks-build
akholodn@razilix:~/github/razr/memory/vxworks-build$
akholodn@razilix:~/github/razr/memory/vxworks-build$
akholodn@razilix:~/github/razr/memory/vxworks-build$ cat ./src/container_node_sizes_impl.hpp
#define FOONATHAN_MEMORY_NO_NODE_SIZE
akholodn@razilix:~/github/razr/memory/vxworks-build$
akholodn@razilix:~/github/razr/memory/vxworks-build$ make
Scanning dependencies of target foonathan_memory
[  1%] Building CXX object src/CMakeFiles/foonathan_memory.dir/detail/align.cpp.o
[  3%] Building CXX object src/CMakeFiles/foonathan_memory.dir/detail/debug_helpers.cpp.o
[  5%] Building CXX object src/CMakeFiles/foonathan_memory.dir/detail/assert.cpp.o
[  7%] Building CXX object src/CMakeFiles/foonathan_memory.dir/detail/free_list.cpp.o
[  9%] Building CXX object src/CMakeFiles/foonathan_memory.dir/detail/free_list_array.cpp.o
[ 10%] Building CXX object src/CMakeFiles/foonathan_memory.dir/detail/small_free_list.cpp.o
[ 12%] Building CXX object src/CMakeFiles/foonathan_memory.dir/debugging.cpp.o
[ 14%] Building CXX object src/CMakeFiles/foonathan_memory.dir/error.cpp.o
[ 16%] Building CXX object src/CMakeFiles/foonathan_memory.dir/heap_allocator.cpp.o
[ 18%] Building CXX object src/CMakeFiles/foonathan_memory.dir/iteration_allocator.cpp.o
[ 20%] Building CXX object src/CMakeFiles/foonathan_memory.dir/malloc_allocator.cpp.o
[ 21%] Building CXX object src/CMakeFiles/foonathan_memory.dir/memory_arena.cpp.o
[ 23%] Building CXX object src/CMakeFiles/foonathan_memory.dir/memory_pool.cpp.o
[ 25%] Building CXX object src/CMakeFiles/foonathan_memory.dir/memory_pool_collection.cpp.o
[ 27%] Building CXX object src/CMakeFiles/foonathan_memory.dir/memory_stack.cpp.o
[ 29%] Building CXX object src/CMakeFiles/foonathan_memory.dir/new_allocator.cpp.o
[ 30%] Building CXX object src/CMakeFiles/foonathan_memory.dir/static_allocator.cpp.o
[ 32%] Building CXX object src/CMakeFiles/foonathan_memory.dir/temporary_allocator.cpp.o
[ 34%] Building CXX object src/CMakeFiles/foonathan_memory.dir/virtual_memory.cpp.o
[ 36%] Linking CXX static library libfoonathan_memory-0.6.2.a
[ 36%] Built target foonathan_memory
Scanning dependencies of target foonathan_memory_example_taking_allocators
[ 38%] Building CXX object example/CMakeFiles/foonathan_memory_example_taking_allocators.dir/taking_allocators.cpp.o
[ 40%] Linking CXX executable foonathan_memory_example_taking_allocators
[ 40%] Built target foonathan_memory_example_taking_allocators
Scanning dependencies of target foonathan_memory_example_allocator_storage
[ 41%] Building CXX object example/CMakeFiles/foonathan_memory_example_allocator_storage.dir/allocator_storage.cpp.o
[ 43%] Linking CXX executable foonathan_memory_example_allocator_storage
[ 43%] Built target foonathan_memory_example_allocator_storage
Scanning dependencies of target foonathan_memory_example_tracking
[ 45%] Building CXX object example/CMakeFiles/foonathan_memory_example_tracking.dir/tracking.cpp.o
/home/akholodn/github/razr/memory/example/tracking.cpp:68:74: error: no member named 'set_node_size' in namespace 'foonathan::memory'
                                           memory::memory_pool<>(memory::set_node_size<int>::value,
                                                                 ~~~~~~~~^
/home/akholodn/github/razr/memory/example/tracking.cpp:68:91: error: expected '(' for function-style cast or type construction
                                           memory::memory_pool<>(memory::set_node_size<int>::value,
                                                                                       ~~~^
/home/akholodn/github/razr/memory/example/tracking.cpp:68:94: error: no member named 'value' in the global namespace
                                           memory::memory_pool<>(memory::set_node_size<int>::value,
                                                                                           ~~^
3 errors generated.
example/CMakeFiles/foonathan_memory_example_tracking.dir/build.make:62: recipe for target 'example/CMakeFiles/foonathan_memory_example_tracking.dir/tracking.cpp.o' failed
make[2]: *** [example/CMakeFiles/foonathan_memory_example_tracking.dir/tracking.cpp.o] Error 1
CMakeFiles/Makefile2:222: recipe for target 'example/CMakeFiles/foonathan_memory_example_tracking.dir/all' failed
make[1]: *** [example/CMakeFiles/foonathan_memory_example_tracking.dir/all] Error 2
Makefile:140: recipe for target 'all' failed
make: *** [all] Error 2

[foonathan]

The first error is unrelated, but annoying: your standard library provides the memory_resource header in C++11 mode, but it doesn't compile when included unless C++17 is activated. I should probably rethink the feature check there.

The second error is as expected. I will disable examples when the node sizes are not available.