Skip to content

[Bug] BE core dump while loading #17587

Closed
@zhannngchen

Description

Search before asking

  • I had searched in the issues and found no similar issues.

Version

1.2.2, master

What's Wrong?

BE core dump with the following error and stack

F0307 14:56:02.353518 3773263 column_string.cpp:152] Check failed: filter.size() == offsets.size() (100 vs. 0)
*** Check failure stack trace: ***
    @     0x559b5299e10d  google::LogMessage::Fail()
    @     0x559b529a0649  google::LogMessage::SendToLog()
    @     0x559b5299dc76  google::LogMessage::Flush()
    @     0x559b529a0cb9  google::LogMessageFatal::~LogMessageFatal()
    @     0x559b4dda9db0  doris::vectorized::ColumnString::filter()
    @     0x559b4dd9f6ef  doris::vectorized::ColumnNullable::filter()
    @     0x559b4de952fd  doris::vectorized::Block::filter_block_internal()
    @     0x559b4de98095  doris::vectorized::Block::filter_block()
    @     0x559b4de98132  doris::vectorized::Block::filter_block()
    @     0x559b51f1fdf7  doris::vectorized::VFileScanner::_convert_to_output_block()
    @     0x559b51f2100f  doris::vectorized::VFileScanner::_get_block_impl()
    @     0x559b51eb6a37  doris::vectorized::VScanner::get_block()
    @     0x559b51eafebc  doris::vectorized::ScannerScheduler::_scanner_scan()
    @     0x559b4c0e7b05  doris::ThreadPool::dispatch_thread()
    @     0x559b4c0dc97f  doris::Thread::supervise_thread()
    @     0x7f61dfbfa1ca  start_thread
    @     0x7f61df5a7ef3  __GI___clone
    @              (nil)  (unknown)
*** Query id: b483f35fa9be68f-a48ed76d53dbe0a5 ***
*** Aborted at 1678172162 (unix time) try "date -d @1678172162" if you are using GNU date ***
*** Current BE git commitID: e94170d81f ***
*** SIGABRT unknown detail explain (@0x4530039914f) received by PID 3772751 (TID 3773263 OR 0x7f60d8f6b700) from PID 3772751; stack trace: ***
 0# doris::signal::(anonymous namespace)::FailureSignalHandler(int, siginfo_t*, void*) at /mnt/disk2/zhangchen/doris-dev/be/src/common/signal_handler.h:428
 1# 0x00007F61DFC04CE0 in /lib64/libpthread.so.0
 2# __GI_raise in /lib64/libc.so.6
 3# abort in /lib64/libc.so.6
 4# 0x0000559B529A8AF9 in /mnt/disk2/zhangchen/deploy/be/lib/doris_be
 5# 0x0000559B5299E10D in /mnt/disk2/zhangchen/deploy/be/lib/doris_be
 6# google::LogMessage::SendToLog() in /mnt/disk2/zhangchen/deploy/be/lib/doris_be
 7# google::LogMessage::Flush() in /mnt/disk2/zhangchen/deploy/be/lib/doris_be
 8# google::LogMessageFatal::~LogMessageFatal() in /mnt/disk2/zhangchen/deploy/be/lib/doris_be
 9# doris::vectorized::ColumnString::filter(doris::vectorized::PODArray<unsigned char, 4096ul, Allocator<false, false>, 15ul, 16ul> const&) at /mnt/disk2/zhangchen/doris-dev/be/src/vec/columns/column_string.cpp:159
10# doris::vectorized::ColumnNullable::filter(doris::vectorized::PODArray<unsigned char, 4096ul, Allocator<false, false>, 15ul, 16ul> const&) at /mnt/disk2/zhangchen/doris-dev/be/src/vec/columns/column_nullable.cpp:314
11# doris::vectorized::Block::filter_block_internal(doris::vectorized::Block*, std::vector<unsigned int, std::allocator<unsigned int> > const&, doris::vectorized::PODArray<unsigned char, 4096ul, Allocator<false, false>, 15ul, 16ul> const&) at /mnt/disk2/zhangchen/doris-dev/be/src/vec/core/block.cpp:672
12# doris::vectorized::Block::filter_block(doris::vectorized::Block*, std::vector<unsigned int, std::allocator<unsigned int> > const&, int, int) at /mnt/disk2/zhangchen/doris-dev/be/src/vec/core/block.cpp:744
13# doris::vectorized::Block::filter_block(doris::vectorized::Block*, int, int) at /mnt/disk2/zhangchen/doris-dev/be/src/vec/core/block.cpp:758
14# doris::vectorized::VFileScanner::_convert_to_output_block(doris::vectorized::Block*) at /mnt/disk2/zhangchen/doris-dev/be/src/vec/exec/scan/vfile_scanner.cpp:547
15# doris::vectorized::VFileScanner::_get_block_impl(doris::RuntimeState*, doris::vectorized::Block*, bool*) at /mnt/disk2/zhangchen/doris-dev/be/src/vec/exec/scan/vfile_scanner.cpp:197
16# doris::vectorized::VScanner::get_block(doris::RuntimeState*, doris::vectorized::Block*, bool*) at /mnt/disk2/zhangchen/doris-dev/be/src/vec/exec/scan/vscanner.cpp:69
17# doris::vectorized::ScannerScheduler::_scanner_scan(doris::vectorized::ScannerScheduler*, doris::vectorized::ScannerContext*, doris::vectorized::VScanner*) at /mnt/disk2/zhangchen/doris-dev/be/src/vec/exec/scan/scanner_scheduler.cpp:306
18# doris::ThreadPool::dispatch_thread() at /mnt/disk2/zhangchen/doris-dev/be/src/util/threadpool.cpp:537
19# doris::Thread::supervise_thread(void*) at /mnt/disk2/zhangchen/doris-dev/be/src/util/thread.cpp:454
20# start_thread in /lib64/libpthread.so.0

What You Expected?

should not core

How to Reproduce?

use the following schema

CREATE TABLE test_table (
col1 tinyint(4) NOT NULL COMMENT '',
col2 varchar(64) NOT NULL COMMENT '',
col3 varchar(128) NOT NULL COMMENT '',
col4 varchar(64) NOT NULL COMMENT '',
col5 varchar(128) REPLACE_IF_NOT_NULL NULL COMMENT '',
col6 int(11) REPLACE_IF_NOT_NULL NULL DEFAULT "0" COMMENT '',
col7 int(11) REPLACE_IF_NOT_NULL NULL DEFAULT "0" COMMENT '',
col8 int(11) REPLACE_IF_NOT_NULL NULL DEFAULT "0" COMMENT '',
col9 int(11) REPLACE_IF_NOT_NULL NULL DEFAULT "0" COMMENT '',
col10 datetime REPLACE_IF_NOT_NULL NULL COMMENT '',
col11 datetime REPLACE_IF_NOT_NULL NULL COMMENT '',
col12 int(11) REPLACE_IF_NOT_NULL NULL DEFAULT "0" COMMENT '',
col13 int(11) REPLACE_IF_NOT_NULL NULL DEFAULT "0" COMMENT '',
col14 int(11) REPLACE_IF_NOT_NULL NULL DEFAULT "0" COMMENT '',
col15 int(11) REPLACE_IF_NOT_NULL NULL DEFAULT "0" COMMENT '',
col16 datetime REPLACE_IF_NOT_NULL NULL COMMENT '',
col17 int(11) REPLACE_IF_NOT_NULL NULL DEFAULT "0" COMMENT '',
col18 tinyint(4) REPLACE_IF_NOT_NULL NULL COMMENT '',
col19 datetime REPLACE_IF_NOT_NULL NULL COMMENT '',
col20 varchar(128) REPLACE_IF_NOT_NULL NULL COMMENT '',
col21 int(11) REPLACE_IF_NOT_NULL NULL DEFAULT "2" COMMENT '',
col22 tinyint(4) REPLACE_IF_NOT_NULL NULL DEFAULT "0" COMMENT '',
col23 varchar(128) REPLACE_IF_NOT_NULL NULL COMMENT '',
col24 tinyint(4) REPLACE_IF_NOT_NULL NULL DEFAULT "0" COMMENT '',
col25 int(11) REPLACE_IF_NOT_NULL NULL DEFAULT "0" COMMENT '',
col26 int(11) REPLACE_IF_NOT_NULL NULL DEFAULT "0" COMMENT '',
col27 int(11) REPLACE_IF_NOT_NULL NULL DEFAULT "0" COMMENT '',
col28 int(11) REPLACE_IF_NOT_NULL NULL DEFAULT "0" COMMENT '',
col29 int(11) REPLACE_IF_NOT_NULL NULL DEFAULT "0" COMMENT '',
col30 int(11) REPLACE_IF_NOT_NULL NULL DEFAULT "0" COMMENT '',
col31 int(11) REPLACE_IF_NOT_NULL NULL DEFAULT "0" COMMENT '',
col32 int(11) REPLACE_IF_NOT_NULL NULL DEFAULT "0" COMMENT '',
col33 int(11) REPLACE_IF_NOT_NULL NULL DEFAULT "0" COMMENT ''
) ENGINE=OLAP
AGGREGATE KEY(`col1`, `col2`, `col3`, `col4`)
PARTITION BY LIST(`col1`)
(PARTITION p_00 VALUES IN ("0"),
PARTITION p_01 VALUES IN ("1"),
PARTITION p_02 VALUES IN ("2"),
PARTITION p_03 VALUES IN ("3"),
PARTITION p_04 VALUES IN ("4"),
PARTITION p_05 VALUES IN ("5"),
PARTITION p_06 VALUES IN ("6"),
PARTITION p_07 VALUES IN ("7"),
PARTITION p_08 VALUES IN ("8"),
PARTITION p_09 VALUES IN ("9"),
PARTITION p_10 VALUES IN ("10"),
PARTITION p_11 VALUES IN ("11"),
PARTITION p_12 VALUES IN ("12"),
PARTITION p_13 VALUES IN ("13"),
PARTITION p_14 VALUES IN ("14"),
PARTITION p_15 VALUES IN ("15"),
PARTITION p_16 VALUES IN ("16"),
PARTITION p_17 VALUES IN ("17"),
PARTITION p_18 VALUES IN ("18"),
PARTITION p_19 VALUES IN ("19"),
PARTITION p_20 VALUES IN ("20"),
PARTITION p_21 VALUES IN ("21"),
PARTITION p_22 VALUES IN ("22"),
PARTITION p_23 VALUES IN ("23"),
PARTITION p_24 VALUES IN ("24"),
PARTITION p_25 VALUES IN ("25"),
PARTITION p_26 VALUES IN ("26"),
PARTITION p_27 VALUES IN ("27"),
PARTITION p_28 VALUES IN ("28"),
PARTITION p_29 VALUES IN ("29"),
PARTITION p_30 VALUES IN ("30"),
PARTITION p_31 VALUES IN ("31"),
PARTITION p_32 VALUES IN ("32"),
PARTITION p_33 VALUES IN ("33"),
PARTITION p_34 VALUES IN ("34"),
PARTITION p_35 VALUES IN ("35"),
PARTITION p_36 VALUES IN ("36"),
PARTITION p_37 VALUES IN ("37"),
PARTITION p_38 VALUES IN ("38"),
PARTITION p_39 VALUES IN ("39"),
PARTITION p_40 VALUES IN ("40"),
PARTITION p_41 VALUES IN ("41"),
PARTITION p_42 VALUES IN ("42"),
PARTITION p_43 VALUES IN ("43"),
PARTITION p_44 VALUES IN ("44"),
PARTITION p_45 VALUES IN ("45"),
PARTITION p_46 VALUES IN ("46"),
PARTITION p_47 VALUES IN ("47"),
PARTITION p_48 VALUES IN ("48"),
PARTITION p_49 VALUES IN ("49"),
PARTITION p_50 VALUES IN ("50"),
PARTITION p_51 VALUES IN ("51"),
PARTITION p_52 VALUES IN ("52"),
PARTITION p_53 VALUES IN ("53"),
PARTITION p_54 VALUES IN ("54"),
PARTITION p_55 VALUES IN ("55"),
PARTITION p_56 VALUES IN ("56"),
PARTITION p_57 VALUES IN ("57"),
PARTITION p_58 VALUES IN ("58"),
PARTITION p_59 VALUES IN ("59"))
DISTRIBUTED BY HASH(`col2`) BUCKETS 6
PROPERTIES (
"replication_allocation" = "tag.location.default: 1",
"bloom_filter_columns" = "col3, col4",
"colocate_with" = "app_group",
"in_memory" = "false",
"storage_format" = "V2",
"disable_auto_compaction" = "false"
);

stream load with following command:

curl --location-trusted -u root: -T random_numbers_with_dates.csv -H "column_separator:," -H "columns:col1=coalesce(cast(substr(col2,-2) as int),0000),col2=coalesce(col2,'a'),col3=coalesce(col3,''),col4=coalesce(col4,''),col5=col5,col6=col6,col7=col7,col8=col8,col9=col9,col10=from_unixtime(col10/1000),col11=from_unixtime(col11/1000),col12=col12,col13=col13,col14=col14,col15=col15,col16=from_unixtime(col16/1000),col17=col17,col18=from_unixtime(col18/1000),col19=col19,col20=col20,col21=col21,col22=col22,col23=col23,col24=col24,col25=col25,col26=col26,col27=col27,col28=col28,col29=col29,col30=col30,col31=col31,col32=col32,col33=col33" http://127.0.0.1:48037/api/db1/test_table/_stream_load

Anything Else?

No response

Are you willing to submit PR?

  • Yes I am willing to submit a PR!

Code of Conduct

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions