Closed
Description
The bthread::ExecutionQueue
used in StarRocks may get fatal error when the number of tasks in the queue is overloading.
Steps to reproduce the behavior (Required)
- creating table with massive buckets
To make it easier to reproduce, I create a table with 2000 buckets. - making stream load to the table created before
To make it easier to reproduce, I make stream load with 32 parallelism and 10MB batch.
Expected behavior (Required)
The BE runs properly.
Real behavior (Required)
The BE get fatal error.
W0305 14:53:20.082063 19490 async_delta_writer_executor.h:32] Service unavailable: Thread pool is at capacity (16/16 tasks running, 40960/40960 tasks queued)
F0305 14:53:20.082093 19490 execution_queue.cpp:120] Fail to submit task: Resource temporarily unavailable [11]
*** Check failure stack trace: ***
@ 0x3b2f86d google::LogMessage::Fail()
@ 0x3b31b49 google::LogMessage::SendToLog()
@ 0x3b2f3e9 google::LogMessage::Flush()
@ 0x3b2f5f2 google::LogMessage::~LogMessage()
@ 0x3c0c01c bthread::ExecutionQueueBase::start_execute()
@ 0x30cf32f starrocks::vectorized::AsyncDeltaWriter::write()
@ 0x1e2acec starrocks::TabletsChannel::add_chunk()
@ 0x1e27565 starrocks::LoadChannel::add_chunk()
@ 0x1e2405b starrocks::LoadChannelMgr::add_chunk()
@ 0x3c5f4fe brpc::policy::ProcessRpcRequest()
@ 0x3c55f07 brpc::ProcessInputMessage()
@ 0x3c56db3 brpc::InputMessenger::OnNewMessages()
@ 0x3cfde2e brpc::Socket::ProcessEvent()
@ 0x3da2c6f bthread::TaskGroup::task_runner()
@ 0x3d97191 bthread_make_fcontext
StarRocks version (Required)
main, branch-2.1