Build fails when tests enabled

[hackernoodle]

It works fine when tests are disabled. I am building in a fedora rawhide chroot using mock from a spec file i made. Here is the end of the output:

[100%] Linking CXX executable flac_compressor_test
/usr/bin/cmake -E cmake_link_script CMakeFiles/flac_compressor_test.dir/link.txt --verbose=1
/usr/lib64/ccache/g++ -O2 -flto=auto -ffat-lto-objects -fexceptions -g -grecord-gcc-switches -pipe -Wall -Werror=format-security -Wp,-U_FORTIFY_SOURCE,-D_FORTIFY_SOURCE=3 -Wp,-D_GLIBCXX_ASSERTIONS -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -fstack-protector-strong -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1  -m64   -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -fno-omit-frame-pointer -mno-omit-leaf-frame-pointer -DNDEBUG -Wl,-z,relro -Wl,--as-needed  -Wl,-z,pack-relative-relocs -Wl,-z,now -specs=/usr/lib/rpm/redhat/redhat-hardened-ld-errors -specs=/usr/lib/rpm/redhat/redhat-hardened-ld -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1  -Wl,--build-id=sha1 -specs=/usr/lib/rpm/redhat/redhat-package-notes CMakeFiles/flac_compressor_test.dir/test/flac_compressor_test.cpp.o -o flac_compressor_test  libtest_helpers.a -lgmock -lgtest -lgtest_main libmkdwarfs_main.a libdwarfsck_main.a libdwarfsbench_main.a libdwarfsextract_main.a libdwarfs_main.a -Wl,--push-state,--whole-archive libdwarfs_compression.a -Wl,--pop-state libdwarfs.a libdwarfs_tool.a -Wl,--push-state,--whole-archive folly/folly/experimental/exception_tracer/libfolly_exception_tracer_base.a libdwarfs_categorizer.a -Wl,--pop-state libcompression_thrift.a libdwarfs.a libdwarfs_compression_metadata.a libmetadata_thrift.a libhistory_thrift.a libfeatures_thrift.a libthrift_light.a libfsst.a /usr/lib64/libboost_chrono.so.1.83.0 /usr/lib64/libboost_iostreams.so.1.83.0 /usr/lib64/libFLAC++.so /usr/lib64/libFLAC.so /usr/lib64/libbrotlidec.so /usr/lib64/libbrotlienc.so /usr/lib64/libarchive.so zstd/build/cmake/lib/libzstd.a /usr/lib64/libxxhash.so /usr/lib64/libjemalloc.so folly/libfolly.a /usr/lib64/libboost_program_options.so.1.83.0 /usr/lib64/libfmt.so.10.2.1 /usr/lib64/libboost_context.so.1.83.0 /usr/lib64/libboost_filesystem.so.1.83.0 /usr/lib64/libboost_atomic.so.1.83.0 /usr/lib64/libboost_regex.so.1.83.0 /usr/lib64/libboost_system.so.1.83.0 /usr/lib64/libboost_thread.so.1.83.0 /usr/lib64/libdouble-conversion.so /usr/lib64/libgflags.so.2.2.2 /usr/lib64/libglog.so /usr/lib64/libevent.so /usr/lib64/libz.so /usr/lib64/libssl.so /usr/lib64/libcrypto.so /usr/lib64/liblz4.so /usr/lib64/liblzma.so /usr/lib64/libdwarf.so /usr/lib64/libiberty.a -ldl /usr/lib64/libunwind.so /usr/lib64/libfuse3.so /usr/lib64/libpthread.a 
lto-wrapper: warning: Extra option to ‘-Xassembler’: --noexecstack, dropping all ‘-Xassembler’ and ‘-Wa’ options.
gmake[2]: Leaving directory '/builddir/build/BUILD/dwarfs-0.8.0/redhat-linux-build'
[100%] Built target dwarfs_test
{standard input}: Assembler messages:
{standard input}:49444: Warning: setting incorrect section attributes for .rodata.thrift.data
{standard input}: Assembler messages:
{standard input}:41568: Warning: setting incorrect section attributes for .rodata.thrift.data
/usr/bin/cmake -D TEST_TARGET=worker_group_test -D TEST_EXECUTABLE=/builddir/build/BUILD/dwarfs-0.8.0/redhat-linux-build/worker_group_test -D TEST_EXECUTOR= -D TEST_WORKING_DIR=/builddir/build/BUILD/dwarfs-0.8.0/redhat-linux-build -D TEST_EXTRA_ARGS= -D TEST_PROPERTIES= -D TEST_PREFIX= -D TEST_SUFFIX= -D TEST_FILTER= -D NO_PRETTY_TYPES=FALSE -D NO_PRETTY_VALUES=FALSE -D TEST_LIST=worker_group_test_TESTS -D CTEST_FILE=/builddir/build/BUILD/dwarfs-0.8.0/redhat-linux-build/worker_group_test[1]_tests.cmake -D TEST_DISCOVERY_TIMEOUT=120 -D TEST_XML_OUTPUT_DIR= -P /usr/share/cmake/Modules/GoogleTestAddTests.cmake
gmake[2]: Leaving directory '/builddir/build/BUILD/dwarfs-0.8.0/redhat-linux-build'
[100%] Built target worker_group_test
{standard input}: Assembler messages:
{standard input}:14462: Warning: setting incorrect section attributes for .rodata.thrift.data
/usr/bin/cmake -D TEST_TARGET=flac_compressor_test -D TEST_EXECUTABLE=/builddir/build/BUILD/dwarfs-0.8.0/redhat-linux-build/flac_compressor_test -D TEST_EXECUTOR= -D TEST_WORKING_DIR=/builddir/build/BUILD/dwarfs-0.8.0/redhat-linux-build -D TEST_EXTRA_ARGS= -D TEST_PROPERTIES= -D TEST_PREFIX= -D TEST_SUFFIX= -D TEST_FILTER= -D NO_PRETTY_TYPES=FALSE -D NO_PRETTY_VALUES=FALSE -D TEST_LIST=flac_compressor_test_TESTS -D CTEST_FILE=/builddir/build/BUILD/dwarfs-0.8.0/redhat-linux-build/flac_compressor_test[1]_tests.cmake -D TEST_DISCOVERY_TIMEOUT=120 -D TEST_XML_OUTPUT_DIR= -P /usr/share/cmake/Modules/GoogleTestAddTests.cmake
gmake[2]: Leaving directory '/builddir/build/BUILD/dwarfs-0.8.0/redhat-linux-build'
[100%] Built target flac_compressor_test
{standard input}: Assembler messages:
{standard input}:24776: Warning: setting incorrect section attributes for .rodata.thrift.data
/usr/bin/ld: /tmp/cc3rhCnc.ltrans0.ltrans.o: in function `utils_uft8_truncate_Test::TestBody()':
/usr/include/gtest/gtest-printers.h:668:(.text+0x40f5): undefined reference to `testing::internal::PrintU8StringTo(std::__cxx11::basic_string<char8_t, std::char_traits<char8_t>, std::allocator<char8_t> > const&, std::ostream*)'
/usr/bin/ld: /tmp/cc3rhCnc.ltrans1.ltrans.o: in function `std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > testing::PrintToString<char8_t const*>(char8_t const* const&)':
/usr/include/gtest/gtest-printers.h:668:(.text+0x7b1a): undefined reference to `testing::internal::PrintU8StringTo(std::__cxx11::basic_string<char8_t, std::char_traits<char8_t>, std::allocator<char8_t> > const&, std::ostream*)'
/usr/bin/ld: /tmp/cc3rhCnc.ltrans2.ltrans.o: in function `testing::AssertionResult testing::internal::CmpHelperEQ<char8_t [24], std::__cxx11::basic_string<char8_t, std::char_traits<char8_t>, std::allocator<char8_t> > >(char const*, char const*, char8_t const (&) [24], std::__cxx11::basic_string<char8_t, std::char_traits<char8_t>, std::allocator<char8_t> > const&)':
/usr/include/gtest/gtest-printers.h:668:(.text+0x3030): undefined reference to `testing::internal::PrintU8StringTo(std::__cxx11::basic_string<char8_t, std::char_traits<char8_t>, std::allocator<char8_t> > const&, std::ostream*)'
collect2: error: ld returned 1 exit status
gmake[2]: Leaving directory '/builddir/build/BUILD/dwarfs-0.8.0/redhat-linux-build'
gmake[2]: *** [CMakeFiles/utils_test.dir/build.make:152: utils_test] Error 1
gmake[1]: *** [CMakeFiles/Makefile2:2060: CMakeFiles/utils_test.dir/all] Error 2
gmake[1]: *** Waiting for unfinished jobs....
/usr/bin/cmake -D TEST_TARGET=tools_test -D TEST_EXECUTABLE=/builddir/build/BUILD/dwarfs-0.8.0/redhat-linux-build/tools_test -D TEST_EXECUTOR= -D TEST_WORKING_DIR=/builddir/build/BUILD/dwarfs-0.8.0/redhat-linux-build -D TEST_EXTRA_ARGS= -D TEST_PROPERTIES= -D TEST_PREFIX= -D TEST_SUFFIX= -D TEST_FILTER= -D NO_PRETTY_TYPES=FALSE -D NO_PRETTY_VALUES=FALSE -D TEST_LIST=tools_test_TESTS -D CTEST_FILE=/builddir/build/BUILD/dwarfs-0.8.0/redhat-linux-build/tools_test[1]_tests.cmake -D TEST_DISCOVERY_TIMEOUT=120 -D TEST_XML_OUTPUT_DIR= -P /usr/share/cmake/Modules/GoogleTestAddTests.cmake
gmake[2]: Leaving directory '/builddir/build/BUILD/dwarfs-0.8.0/redhat-linux-build'
[100%] Built target tools_test
{standard input}: Assembler messages:
{standard input}:42655: Warning: setting incorrect section attributes for .rodata.thrift.data
/usr/bin/cmake -D TEST_TARGET=tool_main_test -D TEST_EXECUTABLE=/builddir/build/BUILD/dwarfs-0.8.0/redhat-linux-build/tool_main_test -D TEST_EXECUTOR= -D TEST_WORKING_DIR=/builddir/build/BUILD/dwarfs-0.8.0/redhat-linux-build -D TEST_EXTRA_ARGS= -D TEST_PROPERTIES= -D TEST_PREFIX= -D TEST_SUFFIX= -D TEST_FILTER= -D NO_PRETTY_TYPES=FALSE -D NO_PRETTY_VALUES=FALSE -D TEST_LIST=tool_main_test_TESTS -D CTEST_FILE=/builddir/build/BUILD/dwarfs-0.8.0/redhat-linux-build/tool_main_test[1]_tests.cmake -D TEST_DISCOVERY_TIMEOUT=120 -D TEST_XML_OUTPUT_DIR= -P /usr/share/cmake/Modules/GoogleTestAddTests.cmake
gmake[2]: Leaving directory '/builddir/build/BUILD/dwarfs-0.8.0/redhat-linux-build'
[100%] Built target tool_main_test
gmake[1]: Leaving directory '/builddir/build/BUILD/dwarfs-0.8.0/redhat-linux-build'
gmake: *** [Makefile:169: all] Error 2

error: Bad exit status from /var/tmp/rpm-tmp.5T9D47 (%build)

RPM build errors:
    Bad exit status from /var/tmp/rpm-tmp.5T9D47 (%build)
Finish: rpmbuild dwarfs-0.8.0-1.src.rpm
Finish: build phase for dwarfs-0.8.0-1.src.rpm
ERROR: Exception(/home/noodle/rpmbuild/SRPMS/dwarfs-0.8.0-1.src.rpm) Config(fedora-rawhide-x86_64) 27 minutes 44 seconds
INFO: Results and/or logs in: /var/lib/mock/fedora-rawhide-x86_64/result
ERROR: Command failed: 
 # bash --login -c '/usr/bin/rpmbuild -bb --noclean --target x86_64 --nodeps /builddir/build/SPECS/dwarfs.spec'

[hackernoodle]

the specfile:

Name:           dwarfs
Version:        0.8.0
Release:        1%{?dist}
Summary:        A fast high compression read-only file system

License:        GPLv3
URL:            https://github.com/mhx/%{name}
Source0:        https://github.com/mhx/%{name}/releases/download/v%{version}/%{name}-%{version}.tar.xz

Requires: boost-chrono
Requires: boost-iostreams
Requires: boost-program-options
Requires: bzip2-libs
Requires: double-conversion
Requires: fmt
Requires: gflags
Requires: glog
Requires: jemalloc
Requires: libacl
Requires: libarchive
Requires: libattr
Requires: libbrotli
Requires: libstdc++
Requires: libunwind
Requires: libxml2
Requires: libzstd
Requires: lz4-libs
Requires: openssl-libs
Requires: xxhash-libs
Requires: xz-libs
Requires: zlib

BuildRequires: binutils-devel
BuildRequires: bison
BuildRequires: boost-chrono
BuildRequires: boost-context
BuildRequires: boost-devel
BuildRequires: boost-filesystem
BuildRequires: boost-iostreams
BuildRequires: boost-program-options
BuildRequires: boost-regex
BuildRequires: boost-system
BuildRequires: boost-thread
BuildRequires: brotli-devel
BuildRequires: ccache
BuildRequires: clang
BuildRequires: cmake
BuildRequires: date-devel
BuildRequires: double-conversion-devel
BuildRequires: elfutils-devel
BuildRequires: file-devel
BuildRequires: flac-devel
BuildRequires: flex
BuildRequires: fmt-devel
BuildRequires: fuse3
BuildRequires: fuse3-devel
BuildRequires: gcc
BuildRequires: gcc-c++
BuildRequires: git
BuildRequires: glog-devel
BuildRequires: gmock-devel
BuildRequires: google-benchmark-devel
BuildRequires: gtest-devel 
BuildRequires: jemalloc-devel
BuildRequires: libacl-devel
BuildRequires: libarchive-devel
BuildRequires: libdwarf-devel
BuildRequires: libevent-devel
BuildRequires: libunwind-devel
BuildRequires: lz4-devel
BuildRequires: make
BuildRequires: ninja-build
BuildRequires: openssl-devel
BuildRequires: pkgconf
BuildRequires: rubygem-ronn-ng
BuildRequires: utf8cpp-devel
BuildRequires: xxhash-devel
BuildRequires: xz-devel

%description
DwarFS is a read-only file system with a focus on achieving very high compression ratios in particular for very redundant data.

%prep
%autosetup

%build
%cmake -DBUILD_SHARED_LIBS=OFF \
-DCMAKE_POSITION_INDEPENDENT_CODE=ON \
-DWITH_TESTS=ON \
-DPREFER_SYSTEM_GTEST=ON \
-DPREFER_SYSTEM_LIBFMT=ON \
-DPREFER_SYSTEM_XXHASH=ON \
-DPREFER_SYSTEM_ZSTD=ON \
-DWITH_MAN_OPTION=OFF
%cmake_build

%check
%ctest

%install
%cmake_install

%files
%license LICENSE
%{_bindir}/dwarfsbench
%{_bindir}/dwarfsck
%{_bindir}/dwarfsextract
%{_bindir}/mkdwarfs
%{_sbindir}/dwarfs
%{_sbindir}/mount.dwarfs
%doc %{_mandir}/man1/dwarfs.1.gz
%doc %{_mandir}/man1/dwarfsck.1.gz
%doc %{_mandir}/man1/dwarfsextract.1.gz
%doc %{_mandir}/man1/mkdwarfs.1.gz
%doc %{_mandir}/man5/dwarfs-format.5.gz

%changelog
* Fri Jan 12 2024 silentnoodle - 0.7.4-1
- rewritten spec

[mhx]

You're not using PREFER_SYSTEM_GTEST by any chance? Looks like it has a problem finding the right symbols in the gtest library.

[hackernoodle]

I am using PREFER_SYSTEM_GTEST yes

[mhx]

Did you try without? Maybe the system version is too old or otherwise broken.

[hackernoodle]

It works without. The version of system gtest is 1.14.0 and it worked for dwarfs 0.7.5 and 0.7.4

[mhx]

I've got an idea. Here's the code from googletest that the linker complains about:

#ifdef __cpp_lib_char8_t
void PrintU8StringTo(const ::std::u8string& s, ostream* os) {
  PrintCharsAsStringTo(s.data(), s.size(), os);
}
#endif

I reckon the system-installed version of googletest is built using an older C++ standard and the compiler isn't defining __cpp_lib_char8_t.

This worked with 0.7.5 / 0.7.4 simply because there's a lot more tests in 0.8.0. :)

[mhx]

BTW, Google itself recommends that projects include gtest rather than using a system-installed version. Since this is only a build-time dependency, it doesn't matter at all for the build artifacts, which don't even link against gtest.

You can get into all sorts of issues when linking code produced with different -std= flags if you're not extremely careful.

[hackernoodle]

It's not a hard requirement but it would be preferable to be able to run the build offline, do you have any recommendations?

[mhx]

I actually do, because I occasionally work offline as well and have added a thing to the CMakeLists.txt for that:

if(DEFINED ENV{DWARFS_LOCAL_REPO_PATH})
  set(LIBFMT_GIT_REPO $ENV{DWARFS_LOCAL_REPO_PATH}/fmt)
  set(GOOGLETEST_GIT_REPO $ENV{DWARFS_LOCAL_REPO_PATH}/googletest)
else()
  set(LIBFMT_GIT_REPO https://github.com/fmtlib/fmt.git)
  set(GOOGLETEST_GIT_REPO https://github.com/google/googletest.git)
endif()

So you could set DWARFS_LOCAL_REPO_PATH to a local directory with clones of the fmt and googletest repos. It works fine for my development needs, but I'd be reluctant to add it to a packaging definition.

I could ship a copy of gtest in the dwarfs tarball, but then again, it's rather big already.

I reckon the system-installed version of googletest is built using an older C++ standard and the compiler isn't defining __cpp_lib_char8_t.

I just had a look at the gtest CMakeLists.txt, and it doesn't explicitly define a C++ standard, which means it'll use the compiler's default, which is currently C++17 (or even older, depending on the compiler version). C++17 indeed doesn't define __cpp_lib_char8_t, so that why it doesn't work with the system-installed gtest.

As a last resort, you could patch utils_test out of the CMakeLists.txt, I believe that's the only test dealing with UTF8-strings.

[hackernoodle]

I could add the googletest tarball as an additional source, but would fmt also need to not be the system installed version for DWARFS_LOCAL_REPO_PATH to ve usable?

[mhx]

If you set PREFER_SYSTEM_LIBFMT, you don't need fmt to be present. Keep in mind that you'd need to tarball the git repo of gtest for this to work.

[hackernoodle]

might be that a simple sed -i '/utils_test/d' CMakeLists.txt makes the most sense in the context of the RPM spec, not sure but it's what I'll go with for now.

[mhx]

might be that a simple sed -i '/utils_test/d' CMakeLists.txt makes the most sense in the context of the RPM spec, not sure but it's what I'll go with for now.

Definitely makes sense. These tests are primarily to make sure the code doesn't break when it's being worked on. If you're still running the remainder of the tests that's going to be good enough.

[mhx]

I'll close this as there isn't really much that can trivially be done here. The best option is really not to use the system installed gtest.