[Bug] Be crash and failed to restart after executing a `alter table modify column xxx` stmt

[dutyu]

Search before asking

• I had searched in the issues and found no similar issues.

Version

2.0.1-rc04 and 2.0.2

What's Wrong?

Today we did a alter table modify column stmt and BE crashed immediately and failed to restart.

ddl:

alter table db1.tbl1  MODIFY COLUMN `col1` VARCHAR(256) COMMENT "xxx";

2.0.1-rc04 be.out:

start time: Thu Oct 26 18:09:01 CST 2023
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/data/doris/be/lib/java_extensions/preload-extensions/preload-extensions-jar-with-dependencies.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/data/doris/be/lib/java_extensions/java-udf/java-udf-jar-with-dependencies.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/data/doris/be/lib/hadoop_hdfs/common/lib/slf4j-reload4j-1.7.36.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Reload4jLoggerFactory]
Java HotSpot(TM) 64-Bit Server VM warning: You have loaded library /data/doris/be/lib/hadoop_hdfs/native/libhadoop.so.1.0.0 which might have disabled stack guard. The VM will try to fix the stack guard now.
It's highly recommended that you fix the library with 'execstack -c <libfile>', or link it with '-z noexecstack'. 
*** Query id: 0-0 ****** Aborted at 1698314952 (unix time) try "date -d @1698314952" if you are using GNU date ***
*** Current BE git commitID: 30d35c4 ****** SIGSEGV unknown detail explain (@0x0) received by PID 96836 (TID 99142 OR 0x7f9c911c5700) from PID 0; stack trace: ***
 0# doris::signal::(anonymous namespace)::FailureSignalHandler(int, siginfo_t*, void*) at /root/src/doris-2.0/be/src/common/signal_handler.h:413
 1# os::Linux::chained_handler(int, siginfo*, void*) in /data/doris/java8/jre/lib/amd64/server/libjvm.so 2# JVM_handle_linux_signal in /data/doris/java8/jre/lib/amd64/server/libjvm.so
 3# signalHandler(int, siginfo*, void*) in /data/doris/java8/jre/lib/amd64/server/libjvm.so 4# 0x00007F9EA1301400 in /lib64/libc.so.6 5# memcpy at /root/src/doris-2.0/be/src/glibc-compatibility/memcpy/memcpy_x86_64.cpp:219
 6# doris::FieldTypeTraits<(doris::FieldType)13>::direct_copy_may_cut(void*, void const*) at /root/src/doris-2.0/be/src/olap/types.h:1336
 7# doris::segment_v2::ScalarColumnWriter::append_data_in_current_page(unsigned char const*, unsigned long*) at /root/src/doris-2.0/be/src/olap/rowset/segment_v2/column_writer.cpp:544
 8# doris::segment_v2::ScalarColumnWriter::append_data(unsigned char const**, unsigned long) at /root/src/doris-2.0/be/src/olap/rowset/segment_v2/column_writer.cpp:528
 9# doris::segment_v2::ColumnWriter::append_nullable(unsigned char const*, unsigned char const**, unsigned long) at /root/src/doris-2.0/be/src/olap/rowset/segment_v2/column_writer.cpp:403
10# doris::segment_v2::ColumnWriter::append(unsigned char const*, void const*, unsigned long) in /data/doris/be/lib/doris_be
11# doris::segment_v2::SegmentWriter::append_block(doris::vectorized::Block const*, unsigned long, unsigned long) in /data/doris/be/lib/doris_be
12# doris::BetaRowsetWriter::_do_add_block(doris::vectorized::Block const*, std::unique_ptr<doris::segment_v2::SegmentWriter, std::default_delete<doris::segment_v2::SegmentWriter> >*, unsigned long, unsigned long) at /root/src/doris-2.0/be/src/olap/rowset/beta_rowset_writer.cpp:392
13# doris::BetaRowsetWriter::_add_block(doris::vectorized::Block const*, std::unique_ptr<doris::segment_v2::SegmentWriter, std::default_delete<doris::segment_v2::SegmentWriter> >*, doris::FlushContext const*) at /root/src/doris-2.0/be/src/olap/rowset/beta_rowset_writer.cpp:423
14# doris::BetaRowsetWriter::flush_single_memtable(doris::vectorized::Block const*, long*, doris::FlushContext const*) at /root/src/doris-2.0/be/src/olap/rowset/beta_rowset_writer.cpp:472
15# doris::Tablet::calc_segment_delete_bitmap(std::shared_ptr<doris::Rowset>, std::shared_ptr<doris::segment_v2::Segment> const&, std::vector<std::shared_ptr<doris::Rowset>, std::allocator<std::shared_ptr<doris::Rowset> > > const&, std::shared_ptr<doris::DeleteBitmap>, long, doris::RowsetWriter*) at /root/src/doris-2.0/be/src/olap/tablet.cpp:3001
16# std::_Function_handler<void (), doris::CalcDeleteBitmapToken::submit(std::shared_ptr<doris::Tablet>, std::shared_ptr<doris::Rowset>, std::shared_ptr<doris::segment_v2::Segment> const&, std::vector<std::shared_ptr<doris::Rowset>, std::allocator<std::shared_ptr<doris::Rowset> > > const&, long, doris::RowsetWriter*)::$_0>::_M_invoke(std::_Any_data const&) at /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/std_function.h:291
17# doris::ThreadPool::dispatch_thread() in /data/doris/be/lib/doris_be
18# doris::Thread::supervise_thread(void*) at /root/src/doris-2.0/be/src/util/thread.cpp:466
19# start_thread in /lib64/libpthread.so.0
20# clone in /lib64/libc.so.6

2.0.2 be.out:

start time: Thu Oct 26 18:03:45 CST 2023
INFO: java_cmd /data/doris/java8/bin/java
INFO: jdk_version 8
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/data/doris/be/lib/java_extensions/preload-extensions/preload-extensions-jar-with-dependencies.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/data/doris/be/lib/java_extensions/java-udf/java-udf-jar-with-dependencies.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/data/doris/be/lib/hadoop_hdfs/common/lib/slf4j-reload4j-1.7.36.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Reload4jLoggerFactory]
Java HotSpot(TM) 64-Bit Server VM warning: You have loaded library /data/doris/be/lib/hadoop_hdfs/native/libhadoop.so.1.0.0 which might have disabled stack guard. The VM will try to fix the stack guard now.
It's highly recommended that you fix the library with 'execstack -c <libfile>', or link it with '-z noexecstack'.
*** Query id: 0-0 ***
*** tablet id: 0 ***
*** Aborted at 1698314635 (unix time) try "date -d @1698314635" if you are using GNU date ***
*** Current BE git commitID: ae923f7 ***
*** SIGSEGV unknown detail explain (@0x0) received by PID 89146 (TID 91444 OR 0x7f1880dc3700) from PID 0; stack trace: ***
 0# doris::signal::(anonymous namespace)::FailureSignalHandler(int, siginfo_t*, void*) at /root/src/doris-2.0/be/src/common/signal_handler.h:417
 1# os::Linux::chained_handler(int, siginfo*, void*) in /data/doris/java8/jre/lib/amd64/server/libjvm.so
 2# JVM_handle_linux_signal in /data/doris/java8/jre/lib/amd64/server/libjvm.so
 3# signalHandler(int, siginfo*, void*) in /data/doris/java8/jre/lib/amd64/server/libjvm.so
 4# 0x00007F1A9250D400 in /lib64/libc.so.6
 5# memcpy at /root/src/doris-2.0/be/src/glibc-compatibility/memcpy/memcpy_x86_64.cpp:219
 6# doris::FieldTypeTraits<(doris::FieldType)13>::direct_copy_may_cut(void*, void const*) at /root/src/doris-2.0/be/src/olap/types.h:1336
 7# doris::segment_v2::ScalarColumnWriter::append_data_in_current_page(unsigned char const*, unsigned long*) at /root/src/doris-2.0/be/src/olap/rowset/segment_v2/column_writer.cpp:544
 8# doris::segment_v2::ScalarColumnWriter::append_data(unsigned char const**, unsigned long) at /root/src/doris-2.0/be/src/olap/rowset/segment_v2/column_writer.cpp:528
 9# doris::segment_v2::ColumnWriter::append_nullable(unsigned char const*, unsigned char const**, unsigned long) at /root/src/doris-2.0/be/src/olap/rowset/segment_v2/column_writer.cpp:403
10# doris::segment_v2::ColumnWriter::append(unsigned char const*, void const*, unsigned long) in /data/doris/be/lib/doris_be
11# doris::segment_v2::SegmentWriter::append_block(doris::vectorized::Block const*, unsigned long, unsigned long) in /data/doris/be/lib/doris_be
12# doris::BetaRowsetWriter::_do_add_block(doris::vectorized::Block const*, std::unique_ptr<doris::segment_v2::SegmentWriter, std::default_delete<doris::segment_v2::SegmentWriter> >*, unsigned long, unsigned long) at /root/src/doris-2.0/be/src/olap/rowset/beta_rowset_writer.cpp:392
13# doris::BetaRowsetWriter::_add_block(doris::vectorized::Block const*, std::unique_ptr<doris::segment_v2::SegmentWriter, std::default_delete<doris::segment_v2::SegmentWriter> >*, doris::FlushContext const*) at /root/src/doris-2.0/be/src/olap/rowset/beta_rowset_writer.cpp:423
14# doris::BetaRowsetWriter::flush_single_memtable(doris::vectorized::Block const*, long*, doris::FlushContext const*) at /root/src/doris-2.0/be/src/olap/rowset/beta_rowset_writer.cpp:472
15# doris::Tablet::calc_segment_delete_bitmap(std::shared_ptr<doris::Rowset>, std::shared_ptr<doris::segment_v2::Segment> const&, std::vector<std::shared_ptr<doris::Rowset>, std::allocator<std::shared_ptr<doris::Rowset> > > const&, std::shared_ptr<doris::DeleteBitmap>, long, doris::RowsetWriter*) at /root/src/doris-2.0/be/src/olap/tablet.cpp:3042
16# std::_Function_handler<void (), doris::CalcDeleteBitmapToken::submit(std::shared_ptr<doris::Tablet>, std::shared_ptr<doris::Rowset>, std::shared_ptr<doris::segment_v2::Segment> const&, std::vector<std::shared_ptr<doris::Rowset>, std::allocator<std::shared_ptr<doris::Rowset> > > const&, long, doris::RowsetWriter*)::$_0>::_M_invoke(std::_Any_data const&) at /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/std_function.h:291
17# doris::ThreadPool::dispatch_thread() in /data/doris/be/lib/doris_be
18# doris::Thread::supervise_thread(void*) at /root/src/doris-2.0/be/src/util/thread.cpp:470
19# start_thread in /lib64/libpthread.so.0
20# clone in /lib64/libc.so.6

What You Expected?

Be can not start now

How to Reproduce?

No response

Anything Else?

No response

Are you willing to submit PR?

• Yes I am willing to submit a PR!

Code of Conduct

• I agree to follow this project's Code of Conduct

[hf200012]

#24139

[zhannngchen]

Hi @dutyu , can you provide the table schema?

[dutyu]

Hi @dutyu , can you provide the table schema?

I've submitted schema info to @hf200012 (for data security consideration), or you can add my wechat 24663660 so i can send it to you.

[zhannngchen]

#26210