[Bug] index compaction core

[qidaye]

Search before asking

• I had searched in the issues and found no similar issues.

Version

master

What's Wrong?

It will core dump when index compaction is turned on.

*** Query id: 0-0 ***
*** Aborted at 1695207500 (unix time) try "date -d @1695207500" if you are using GNU date ***
*** Current BE git commitID: 6a33e4639a ***
*** SIGSEGV address not mapped to object (@0x7f7cd8786940) received by PID 1566142 (TID 1568490 OR 0x7f77d23c5700) from PID 18446744073046354240; stack trace: ***
 0# doris::signal::(anonymous namespace)::FailureSignalHandler(int, siginfo_t*, void*) at /mnt/disk2/luen/develop/workspace/incubator-doris/be/src/common/signal_handler.h:413
 1# 0x00007F79BFFD7B50 in /lib64/libc.so.6
 2# lucene::index::IndexWriter::mergeTerms(bool) at /mnt/disk2/luen/develop/workspace/incubator-doris/be/src/clucene/src/core/CLucene/index/IndexWriter.cpp:1691
 3# lucene::index::IndexWriter::indexCompaction(std::vector<lucene::store::Directory*, std::allocator<lucene::store::Directory*> >&, std::vector<lucene::store::Directory*, std::allocator<lucene::store::Directory*> >, std::vector<std::vector<std::pair<unsigned int, unsigned int>, std::allocator<std::pair<unsigned int, unsigned int> > >, std::allocator<std::vector<std::pair<unsigned int, unsigned int>, std::allocator<std::pair<unsigned int, unsigned int> > > > >, std::vector<unsigned int, std::allocator<unsigned int> >) at /mnt/disk2/luen/develop/workspace/incubator-doris/be/src/clucene/src/core/CLucene/index/IndexWriter.cpp:1352
 4# doris::segment_v2::compact_column(int, int, int, std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > >, std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > >, std::shared_ptr<doris::io::FileSystem> const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::vector<std::vector<std::pair<unsigned int, unsigned int>, std::allocator<std::pair<unsigned int, unsigned int> > >, std::allocator<std::vector<std::pair<unsigned int, unsigned int>, std::allocator<std::pair<unsigned int, unsigned int> > > > >, std::vector<unsigned int, std::allocator<unsigned int> >) at /mnt/disk2/luen/develop/workspace/incubator-doris/be/src/olap/rowset/segment_v2/inverted_index_compaction.cpp:59
 5# doris::Compaction::do_compaction_impl(long)::$_1::operator()(int) const at /mnt/disk2/luen/develop/workspace/incubator-doris/be/src/olap/compaction.cpp:453
 6# doris::Compaction::do_compaction_impl(long)::$_1 std::for_each<std::_Rb_tree_const_iterator<int>, doris::Compaction::do_compaction_impl(long)::$_1>(std::_Rb_tree_const_iterator<int>, std::_Rb_tree_const_iterator<int>, doris::Compaction::do_compaction_impl(long)::$_1) at /mnt/disk2/luen/develop/tools/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/stl_algo.h:3819
 7# doris::Compaction::do_compaction_impl(long) at /mnt/disk2/luen/develop/workspace/incubator-doris/be/src/olap/compaction.cpp:448
 8# doris::Compaction::do_compaction(long) at /mnt/disk2/luen/develop/workspace/incubator-doris/be/src/olap/compaction.cpp:123
 9# doris::CumulativeCompaction::execute_compact_impl() at /mnt/disk2/luen/develop/workspace/incubator-doris/be/src/olap/cumulative_compaction.cpp:87
10# doris::Compaction::execute_compact() at /mnt/disk2/luen/develop/workspace/incubator-doris/be/src/olap/compaction.cpp:105
11# doris::Tablet::execute_compaction(doris::CompactionType) at /mnt/disk2/luen/develop/workspace/incubator-doris/be/src/olap/tablet.cpp:1857
12# doris::StorageEngine::_submit_compaction_task(std::shared_ptr<doris::Tablet>, doris::CompactionType, bool)::$_0::operator()() const at /mnt/disk2/luen/develop/workspace/incubator-doris/be/src/olap/olap_server.cpp:954
13# void std::__invoke_impl<void, doris::StorageEngine::_submit_compaction_task(std::shared_ptr<doris::Tablet>, doris::CompactionType, bool)::$_0&>(std::__invoke_other, doris::StorageEngine::_submit_compaction_task(std::shared_ptr<doris::Tablet>, doris::CompactionType, bool)::$_0&) at /mnt/disk2/luen/develop/tools/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/invoke.h:61
14# std::enable_if<is_invocable_r_v<void, doris::StorageEngine::_submit_compaction_task(std::shared_ptr<doris::Tablet>, doris::CompactionType, bool)::$_0&>, void>::type std::__invoke_r<void, doris::StorageEngine::_submit_compaction_task(std::shared_ptr<doris::Tablet>, doris::CompactionType, bool)::$_0&>(doris::StorageEngine::_submit_compaction_task(std::shared_ptr<doris::Tablet>, doris::CompactionType, bool)::$_0&) at /mnt/disk2/luen/develop/tools/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/invoke.h:117
15# std::_Function_handler<void (), doris::StorageEngine::_submit_compaction_task(std::shared_ptr<doris::Tablet>, doris::CompactionType, bool)::$_0>::_M_invoke(std::_Any_data const&) at /mnt/disk2/luen/develop/tools/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/std_function.h:291
16# std::function<void ()>::operator()() const at /mnt/disk2/luen/develop/tools/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/std_function.h:560
17# doris::FunctionRunnable::run() at /mnt/disk2/luen/develop/workspace/incubator-doris/be/src/util/threadpool.cpp:48
18# doris::ThreadPool::dispatch_thread() at /mnt/disk2/luen/develop/workspace/incubator-doris/be/src/util/threadpool.cpp:533
19# void std::__invoke_impl<void, void (doris::ThreadPool::*&)(), doris::ThreadPool*&>(std::__invoke_memfun_deref, void (doris::ThreadPool::*&)(), doris::ThreadPool*&) at /mnt/disk2/luen/develop/tools/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/invoke.h:74
20# std::__invoke_result<void (doris::ThreadPool::*&)(), doris::ThreadPool*&>::type std::__invoke<void (doris::ThreadPool::*&)(), doris::ThreadPool*&>(void (doris::ThreadPool::*&)(), doris::ThreadPool*&) at /mnt/disk2/luen/develop/tools/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/invoke.h:96
21# void std::_Bind<void (doris::ThreadPool::*(doris::ThreadPool*))()>::__call<void, , 0ul>(std::tuple<>&&, std::_Index_tuple<0ul>) at /mnt/disk2/luen/develop/tools/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/functional:420
22# void std::_Bind<void (doris::ThreadPool::*(doris::ThreadPool*))()>::operator()<, void>() at /mnt/disk2/luen/develop/tools/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/functional:503
23# void std::__invoke_impl<void, std::_Bind<void (doris::ThreadPool::*(doris::ThreadPool*))()>&>(std::__invoke_other, std::_Bind<void (doris::ThreadPool::*(doris::ThreadPool*))()>&) at /mnt/disk2/luen/develop/tools/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/invoke.h:61
24# std::enable_if<is_invocable_r_v<void, std::_Bind<void (doris::ThreadPool::*(doris::ThreadPool*))()>&>, void>::type std::__invoke_r<void, std::_Bind<void (doris::ThreadPool::*(doris::ThreadPool*))()>&>(std::_Bind<void (doris::ThreadPool::*(doris::ThreadPool*))()>&) at /mnt/disk2/luen/develop/tools/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/invoke.h:117
25# std::_Function_handler<void (), std::_Bind<void (doris::ThreadPool::*(doris::ThreadPool*))()> >::_M_invoke(std::_Any_data const&) at /mnt/disk2/luen/develop/tools/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/std_function.h:291
26# std::function<void ()>::operator()() const at /mnt/disk2/luen/develop/tools/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/std_function.h:560
27# doris::Thread::supervise_thread(void*) at /mnt/disk2/luen/develop/workspace/incubator-doris/be/src/util/thread.cpp:465
28# start_thread in /lib64/libpthread.so.0
29# __clone in /lib64/libc.so.6

What You Expected?

Function normally.

How to Reproduce?

1. Turn on index compaction in be.conf inverted_index_compaction_enable=true
2. ./run-regression-test.sh --run -d inverted_index_p0
3. Get core dump.

Anything Else?

No response

Are you willing to submit PR?

• Yes I am willing to submit a PR!

Code of Conduct

• I agree to follow this project's Code of Conduct