OSS build broken as of May, 2024 -> PRs are all blocked

[therealgymmy]

https://github.com/facebook/CacheLib/actions/runs/8937959842/job/24551185009

Thrift dependency fails to build. We cannot import PRs when our OSS builds are broken. We're actively looking into this.

/__w/CacheLib/CacheLib/cachelib/external/fbthrift/thrift/lib/cpp2/server/Cpp2Connection.cpp
In file included from /__w/CacheLib/CacheLib/cachelib/external/fbthrift/./thrift/lib/cpp2/server/Cpp2Worker.h:38,
                 from /__w/CacheLib/CacheLib/cachelib/external/fbthrift/./thrift/lib/cpp2/server/Cpp2Connection.h:32,
                 from /__w/CacheLib/CacheLib/cachelib/external/fbthrift/thrift/lib/cpp2/server/Cpp2Connection.cpp:17:
/__w/CacheLib/CacheLib/cachelib/external/fbthrift/./thrift/lib/cpp2/server/ThriftServer.h:99:10: fatal error: wangle/ssl/TLSInMemoryTicketProcessor.h: No such file or directory
 #include <wangle/ssl/TLSInMemoryTicketProcessor.h>
          ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

[therealgymmy]

Previous issue is now resolved however we ran into the following issue this past weekend. :/

usr/bin/cmake -E cmake_link_script CMakeFiles/FizzTool.dir/link.txt --verbose=YES
/usr/bin/c++ -O2 -g -DNDEBUG -rdynamic CMakeFiles/FizzTool.dir/tool/Main.cpp.o CMakeFiles/FizzTool.dir/tool/FizzClientCommand.cpp.o CMakeFiles/FizzTool.dir/tool/FizzClientLoadGenCommand.cpp.o CMakeFiles/FizzTool.dir/tool/FizzCommandCommon.cpp.o CMakeFiles/FizzTool.dir/tool/FizzGenerateDelegatedCredentialCommand.cpp.o CMakeFiles/FizzTool.dir/tool/FizzServerBenchmarkCommand.cpp.o CMakeFiles/FizzTool.dir/tool/FizzServerCommand.cpp.o -o bin/fizz  -Wl,-rpath,/__w/CacheLib/CacheLib/build-fizz/lib:/__w/CacheLib/CacheLib/opt/cachelib/lib:/__w/CacheLib/CacheLib/opt/cachelib/lib64: lib/libfizz.so.1.0.0 /usr/lib64/libsodium.so /__w/CacheLib/CacheLib/opt/cachelib/lib/libfolly.so.0.58.0-dev /__w/CacheLib/CacheLib/opt/cachelib/lib64/libfmt.so.10.2.1 -lboost_context -lboost_filesystem -lboost_program_options -lboost_regex -lboost_system -lboost_thread -lboost_chrono -lboost_date_time -lboost_atomic -ldouble-conversion /__w/CacheLib/CacheLib/opt/cachelib/lib64/libglog.so -levent -lz -lbz2 -llzma -llz4 -lsnappy -ldwarf -Wl,-Bstatic -liberty -Wl,-Bdynamic -laio -lsodium -ldl -lunwind -lssl -lcrypto /usr/lib64/libz.so /__w/CacheLib/CacheLib/opt/cachelib/lib64/libzstd.so /__w/CacheLib/CacheLib/opt/cachelib/lib/libgflags.so.2.2.2 
/usr/bin/ld: CMakeFiles/FizzTool.dir/tool/FizzGenerateDelegatedCredentialCommand.cpp.o: in function `fizz::tool::fizzGenerateDelegatedCredentialCommand(std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > > const&)':
/__w/CacheLib/CacheLib/cachelib/external/fizz/fizz/tool/FizzGenerateDelegatedCredentialCommand.cpp:186: undefined reference to `fizz::extensions::generateDelegatedCredentialPEM(fizz::extensions::DelegatedCredential, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >)'
collect2: error: ld returned 1 exit status
make[2]: Leaving directory '/__w/CacheLib/CacheLib/build-fizz'

[therealgymmy]

mvfst broke as well. Also fixed internally. Pending the next submodule sync.

[therealgymmy]

welp. folly is now broken. This is kinda ludicrous :-) one thing breaks after another.

https://github.com/facebook/CacheLib/actions/runs/9065222623/job/24905320734

/__w/CacheLib/CacheLib/cachelib/external/folly/folly/io/async/AsyncSocket.cpp:1733:15: note: suggested alternative: 'BOOST_PP_IIF'
     iovec vec[BOOST_PP_IF(FOLLY_HAVE_VLA_01, count, kSmallIoVecSize)];
               ^~~~~~~~~~~
               BOOST_PP_IIF
/__w/CacheLib/CacheLib/cachelib/external/folly/folly/io/async/AsyncSocket.cpp:1736:30: error: 'vec' was not declared in this scope
     writeChainImpl(callback, vec, count, std::move(buf), flags);
                              ^~~
/__w/CacheLib/CacheLib/cachelib/external/folly/folly/io/async/AsyncSocket.cpp: In member function 'virtual folly::AsyncSocket::WriteResult folly::AsyncSocket::sendSocketMessage(const iovec*, size_t, folly::WriteFlags, folly::AsyncSocket::WriteRequestTag)':
/__w/CacheLib/CacheLib/cachelib/external/folly/folly/io/async/AsyncSocket.cpp:3774:18: error: 'BOOST_PP_IF' was not declared in this scope
     iovec tmpVec[BOOST_PP_IF(FOLLY_HAVE_VLA_01, count, kSmallIoVecSize)];
                  ^~~~~~~~~~~
/__w/CacheLib/CacheLib/cachelib/external/folly/folly/io/async/AsyncSocket.cpp:3774:18: note: suggested alternative: 'BOOST_PP_IIF'
     iovec tmpVec[BOOST_PP_IF(FOLLY_HAVE_VLA_01, count, kSmallIoVecSize)];
                  ^~~~~~~~~~~
                  BOOST_PP_IIF
/__w/CacheLib/CacheLib/cachelib/external/folly/folly/io/async/AsyncSocket.cpp:3783:9: error: 'tmpVec' was not declared in this scope
         tmpVec,
         ^~~~~~
/__w/CacheLib/CacheLib/cachelib/external/folly/folly/io/async/AsyncSocket.cpp:3783:9: note: suggested alternative: 'tmpnam'
         tmpVec,
         ^~~~~~
         tmpnam
[ 55%] Building CXX object CMakeFiles/folly_base.dir/folly/io/async/AsyncTimeout.cpp.o
/usr/bin/c++ -DFMT_SHARED -DFOLLY_XLOG_STRIP_PREFIXES=\"/__w/CacheLib/CacheLib/cachelib/external/folly:/__w/CacheLib/CacheLib/build-folly\" -DGFLAGS_IS_A_DLL=0 -D_GNU_SOURCE -D_REENTRANT -I/__w/CacheLib/CacheLib/cachelib/external/folly -I/__w/CacheLib/CacheLib/build-folly -I/__w/CacheLib/CacheLib/opt/cachelib/include -I/usr/include/libdwarf -O2 -g -DNDEBUG -std=gnu++17 -fPIC -g -finput-charset=UTF-8 -fsigned-char -Wall -Wno-deprecated -Wno-deprecated-declarations -Wno-sign-compare -Wno-unused -Wuninitialized -Wunused-label -Wunused-result -Wshadow-compatible-local -Wno-noexcept-type -faligned-new -fopenmp -MD -MT CMakeFiles/folly_base.dir/folly/io/async/AsyncTimeout.cpp.o -MF CMakeFiles/folly_base.dir/folly/io/async/AsyncTimeout.cpp.o.d -o CMakeFiles/folly_base.dir/folly/io/async/AsyncTimeout.cpp.o -c /__w/CacheLib/CacheLib/cachelib/external/folly/folly/io/async/AsyncTimeout.cpp
make[2]: *** [CMakeFiles/folly_base.dir/build.make:2624: CMakeFiles/folly_base.dir/folly/io/async/AsyncSocket.cpp.o] Error 1
make[2]: *** Waiting for unfinished jobs....
make[2]: Leaving directory '/__w/CacheLib/CacheLib/build-folly'
make[1]: *** [CMakeFiles/Makefile2:145: CMakeFiles/folly_base.dir/all] Error 2
make[1]: Leaving directory '/__w/CacheLib/CacheLib/build-folly'
make: *** [Makefile:136: all] Error 2
build-package.sh: error: make failed

[therealgymmy]

Our build is partially back. However, centos 8.1, 8.5, rocky 8.6 and ubuntu 20 are broken on folly.

In member function ‘virtual void folly::AsyncSocket::writeChain(folly::AsyncWriter::WriteCallback*, std::unique_ptr<folly::IOBuf>&&, folly::WriteFlags)’:
/home/CacheLib/cachelib/external/folly/folly/io/async/AsyncSocket.cpp:1733:15: error: ‘BOOST_PP_IF’ was not declared in this scope
     iovec vec[BOOST_PP_IF(FOLLY_HAVE_VLA_01, count, kSmallIoVecSize)];
               ^~~~~~~~~~~
/home/CacheLib/cachelib/external/folly/folly/io/async/AsyncSocket.cpp:1733:15: note: suggested alternative: ‘BOOST_PP_IIF’
     iovec vec[BOOST_PP_IF(FOLLY_HAVE_VLA_01, count, kSmallIoVecSize)];
               ^~~~~~~~~~~
               BOOST_PP_IIF
/home/CacheLib/cachelib/external/folly/folly/io/async/AsyncSocket.cpp:1736:30: error: ‘vec’ was not declared in this scope
     writeChainImpl(callback, vec, count, std::move(buf), flags);

[therealgymmy]

mvfst is broken. All builds are broken once again.

/__w/CacheLib/CacheLib/cachelib/external/mvfst/quic/codec/QuicPacketBuilder.cpp: In member function 'virtual void quic::InplaceQuicPacketBuilder::appendPaddingFrame()':
/__w/CacheLib/CacheLib/cachelib/external/mvfst/quic/codec/QuicPacketBuilder.cpp:676:29: note: '<anonymous>' declared here
  676 |     packet_.frames.push_back(PaddingFrame());
      |     ~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~
In file included from /__w/CacheLib/CacheLib/cachelib/external/mvfst/quic/codec/QuicPacketBuilder.h:13,
                 from /__w/CacheLib/CacheLib/cachelib/external/mvfst/quic/codec/QuicPacketBuilder.cpp:8:
In constructor 'quic::KnobFrame::KnobFrame(quic::KnobFrame&&)',
    inlined from 'quic::QuicSimpleFrame::QuicSimpleFrame(quic::QuicSimpleFrame&&)' at /__w/CacheLib/CacheLib/cachelib/external/mvfst/quic/codec/Types.h:859:1,
    inlined from 'quic::QuicWriteFrame::QuicWriteFrame(quic::QuicWriteFrame&&)' at /__w/CacheLib/CacheLib/cachelib/external/mvfst/quic/codec/Types.h:901:1,
    inlined from 'void __gnu_cxx::new_allocator<_Tp>::construct(_Up*, _Args&& ...) [with _Up = quic::QuicWriteFrame; _Args = {quic::QuicWriteFrame}; _Tp = quic::QuicWriteFrame]' at /usr/include/c++/11/ext/new_allocator.h:162:4,
    inlined from 'static void std::allocator_traits<std::allocator<_CharT> >::construct(std::allocator_traits<std::allocator<_CharT> >::allocator_type&, _Up*, _Args&& ...) [with _Up = quic::QuicWriteFrame; _Args = {quic::QuicWriteFrame}; _Tp = quic::QuicWriteFrame]' at /usr/include/c++/11/bits/alloc_traits.h:516:17,
    inlined from 'std::vector<_Tp, _Alloc>::reference std::vector<_Tp, _Alloc>::emplace_back(_Args&& ...) [with _Args = {quic::QuicWriteFrame}; _Tp = quic::QuicWriteFrame; _Alloc = std::allocator<quic::QuicWriteFrame>]' at /usr/include/c++/11/bits/vector.tcc:115:30,
    inlined from 'void std::vector<_Tp, _Alloc>::push_back(std::vector<_Tp, _Alloc>::value_type&&) [with _Tp = quic::QuicWriteFrame; _Alloc = std::allocator<quic::QuicWriteFrame>]' at /usr/include/c++/11/bits/stl_vector.h:1204:21,
    inlined from 'virtual void quic::InplaceQuicPacketBuilder::appendPaddingFrame()' at /__w/CacheLib/CacheLib/cachelib/external/mvfst/quic/codec/QuicPacketBuilder.cpp:676:29:
/__w/CacheLib/CacheLib/cachelib/external/mvfst/quic/codec/Types.h:109:19: warning: '*(quic::KnobFrame*)((char*)&<unnamed> + offsetof(quic::value_type, quic::QuicWriteFrame::<unnamed>)).quic::KnobFrame::len' may be used uninitialized [-Wmaybe-uninitialized]
  109 |         len(other.len),
      |             ~~~~~~^~~
/__w/CacheLib/CacheLib/cachelib/external/mvfst/quic/codec/QuicPacketBuilder.cpp: In member function 'virtual void quic::InplaceQuicPacketBuilder::appendPaddingFrame()':
/__w/CacheLib/CacheLib/cachelib/external/mvfst/quic/codec/QuicPacketBuilder.cpp:676:29: note: '<anonymous>' declared here
  676 |     packet_.frames.push_back(PaddingFrame());
      |     ~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~

[therealgymmy]

Now it is fbthrift once again :-) what a month. We'll do what we can to continue chase down the fixes one at a time. Long-term, we're discussing to get rid of our thrift dependency (and its own dependencies) as well.

[haowu14]

Dependencies are fixed.

On Ubuntu 22 there's a cachelib compilation failure. Attempting a fix.

In file included from /home/runner/work/CacheLib/CacheLib/cachelib/allocator/tests/ChainedHashTest.cpp:18:
/home/runner/work/CacheLib/CacheLib/cachelib/../cachelib/allocator/tests/AccessTypeTest.h: In member function ‘void facebook::cachelib::tests::AccessTypeTest<AccessType>::testIteratorMayContainNull()’:
/home/runner/work/CacheLib/CacheLib/cachelib/../cachelib/allocator/tests/AccessTypeTest.h:596:49: error: need ‘typename’ before ‘facebook::cachelib::tests::AccessTypeTest<AccessType>::Node::Handle’ because ‘facebook::cachelib::tests::AccessTypeTest<AccessType>::Node’ is a dependent scope
  596 |               [&keyToReturnNull](Node* node) -> Node::Handle {
      |                                                 ^~~~
      |                                                 typename 
/home/runner/work/CacheLib/CacheLib/cachelib/../cachelib/allocator/tests/AccessTypeTest.h: In instantiation of ‘void facebook::cachelib::tests::AccessTypeTest<AccessType>::testIteratorMayContainNull() [with AccessType = facebook::cachelib::ChainedHashTable]’:
/home/runner/work/CacheLib/CacheLib/cachelib/allocator/tests/ChainedHashTest.cpp:57:29:   required from here
/home/runner/work/CacheLib/CacheLib/cachelib/../cachelib/allocator/tests/AccessTypeTest.h:595:13: error: no matching function for call to ‘facebook::cach

[triplewy]

I'm interested in using cachelib but am a bit confused on its networking dependencies. My understanding was that cachelib is simply a storage engine without any networking capabilities. Is there any progress on removal of Thrift from its dependencies?