[fix](memory allocate) Fix reinitialization of TabletReader #36314
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
zhiqiang-hhhh
changed the title
|
run buildall |
|
clang-tidy review says "All clean, LGTM! 👍" |
|
PR approved by at least one committer and no changes requested. |
github-actions
bot
added
approved
|
PR approved by anyone and no changes requested. |
dataroaring
pushed a commit
that referenced
this pull request
Jun 21, 2024
`ASSIGN_STATUS_IF_CATCH_EXCEPTION` will convert error code from `MEM_ALLOC_FAILED` to `MEM_LIMIT_EXCEEDED`, this will lead to problem like below: ```text F20240614 16:07:18.529981 3400047 delete_handler.cpp:388] Check failed: !_is_inited reinitialize delete handler. *** Check failure stack trace: *** @ 0x561cdc597926 google::LogMessage::SendToLog() @ 0x561cdc594370 google::LogMessage::Flush() @ 0x561cdc598169 google::LogMessageFatal::~LogMessageFatal() @ 0x561cacf6ba70 doris::DeleteHandler::init() @ 0x561caf6e9298 doris::TabletReader::_init_delete_condition() @ 0x561caf6e257c doris::TabletReader::_init_params() @ 0x561caf6e1a09 doris::TabletReader::init() @ 0x561cd847917f doris::vectorized::BlockReader::init() @ 0x561cdbc466a4 doris::vectorized::NewOlapScanner::open() @ 0x561cc3a1b623 doris::vectorized::ScannerScheduler::_scanner_scan() @ 0x561cc3a1fae1 _ZZZZN5doris10vectorized16ScannerScheduler6submitESt10shared_ptrINS0_14ScannerContextEES2_INS0_8ScanTaskEEENK3$_1clEvENKUlvE_clEvENKUlvE_clEv @ 0x561cc3a1f554 _ZZZN5doris10vectorized16ScannerScheduler6submitESt10shared_ptrINS0_14ScannerContextEES2_INS0_8ScanTaskEEENK3$_1clEvENKUlvE_clEv @ 0x561cc3a1f3c5 _ZSt13__invoke_implIvRZZN5doris10vectorized16ScannerScheduler6submitESt10shared_ptrINS1_14ScannerContextEES3_INS1_8ScanTaskEEENK3$_1clEvEUlvE_JEET_St14__invoke_otherOT0_DpOT1_ @ 0x561cc3a1f365 _ZSt10__invoke_rIvRZZN5doris10vectorized16ScannerScheduler6submitESt10shared_ptrINS1_14ScannerContextEES3_INS1_8ScanTaskEEENK3$_1clEvEUlvE_JEENSt9enable_ifIX16is_invocable_r_vIT_T0_DpT1_EESC_E4typeEOSD_DpOSE_ @ 0x561cc3a1f18d _ZNSt17_Function_handlerIFvvEZZN5doris10vectorized16ScannerScheduler6submitESt10shared_ptrINS2_14ScannerContextEES4_INS2_8ScanTaskEEENK3$_1clEvEUlvE_E9_M_invokeERKSt9_Any_data @ 0x561cac9f8023 std::function<>::operator()() @ 0x561cc3a333b5 _ZZN5doris10vectorized23SimplifiedScanScheduler16submit_scan_taskENS0_18SimplifiedScanTaskEENKUlvE_clEv @ 0x561cc3a33395 _ZSt13__invoke_implIvRZN5doris10vectorized23SimplifiedScanScheduler16submit_scan_taskENS1_18SimplifiedScanTaskEEUlvE_JEET_St14__invoke_otherOT0_DpOT1_ @ 0x561cc3a33335 _ZSt10__invoke_rIvRZN5doris10vectorized23SimplifiedScanScheduler16submit_scan_taskENS1_18SimplifiedScanTaskEEUlvE_JEENSt9enable_ifIX16is_invocable_r_vIT_T0_DpT1_EES7_E4typeEOS8_DpOS9_ @ 0x561cc3a3312d _ZNSt17_Function_handlerIFvvEZN5doris10vectorized23SimplifiedScanScheduler16submit_scan_taskENS2_18SimplifiedScanTaskEEUlvE_E9_M_invokeERKSt9_Any_data @ 0x561cac9f8023 std::function<>::operator()() @ 0x561cb095d4e9 doris::FunctionRunnable::run() @ 0x561cb094913e doris::ThreadPool::dispatch_thread() @ 0x561cb0970564 std::__invoke_impl<>() @ 0x561cb097043d std::__invoke<>() @ 0x561cb09703c5 _ZNSt5_BindIFMN5doris10ThreadPoolEFvvEPS1_EE6__callIvJEJLm0EEEET_OSt5tupleIJDpT0_EESt12_Index_tupleIJXspT1_EEE @ 0x561cb097026e std::_Bind<>::operator()<>() @ 0x561cb0970185 std::__invoke_impl<>() @ 0x561cb0970125 _ZSt10__invoke_rIvRSt5_BindIFMN5doris10ThreadPoolEFvvEPS2_EEJEENSt9enable_ifIX16is_invocable_r_vIT_T0_DpT1_EESA_E4typeEOSB_DpOSC_ @ 0x561cb096fdcd std::_Function_handler<>::_M_invoke() @ 0x561cac9f8023 std::function<>::operator()() @ 0x561cb091666c doris::Thread::supervise_thread() ```
dataroaring
pushed a commit
that referenced
this pull request
Jul 2, 2024
## Proposed changes
```
F20240628 01:49:16.382710 4183685 delete_handler.cpp:388] Check failed: !_is_inited reinitialize delete handler.
*** Check failure stack trace: ***
@ 0x55700470e3c6 google::LogMessage::SendToLog()
@ 0x55700470ae10 google::LogMessage::Flush()
@ 0x55700470ec09 google::LogMessageFatal::~LogMessageFatal()
@ 0x556fccf40e64 doris::DeleteHandler::init()
@ 0x556fcff46678 doris::TabletReader::_init_delete_condition()
@ 0x556fcff3a2dd doris::TabletReader::_init_params()
@ 0x556fcff39432 doris::TabletReader::init()
@ 0x556fffb8c2dd doris::vectorized::BlockReader::init()
@ 0x557002cca96a doris::vectorized::NewOlapScanner::open()
@ 0x556fe892d565 doris::vectorized::ScannerScheduler::_scanner_scan()
@ 0x556fe8931a0f _ZNSt17_Function_handlerIFvvEZZN5doris10vectorized16ScannerScheduler6submitESt10shared_ptrINS2_14ScannerContextEES4_INS2_8ScanTaskEEENK3$_1clEvEUlvE_E9_M_invokeERKSt9_Any_data
@ 0x556fd0ed95dc doris::ThreadPool::dispatch_thread()
@ 0x556fd0eb1288 doris::Thread::supervise_thread()
@ 0x7f95143b5609 start_thread
@ 0x7f9514662133 clone
@ (nil) (unknown)
*** Query id: c389fc2a1ff6473c-a06f032b8970810c ***
*** is nereids: 1 ***
*** tablet id: 0 ***
*** Aborted at 1719510556 (unix time) try "date -d @1719510556" if you are using GNU date ***
*** Current BE git commitID: b13c17d ***
*** SIGABRT unknown detail explain (@0x3fca33) received by PID 4180531 (TID 4183685 OR 0x7f89734a5700) from PID 4180531; stack trace: ***
0# doris::signal::(anonymous namespace)::FailureSignalHandler(int, siginfo_t*, void*) at /home/zcp/repo_center/doris_master/doris/be/src/common/signal_handler.h:421
1# 0x00007F9514586090 in /lib/x86_64-linux-gnu/libc.so.6
2# raise at ../sysdeps/unix/sysv/linux/raise.c:51
3# abort at /build/glibc-SzIz7B/glibc-2.31/stdlib/abort.c:81
4# 0x0000557004718C9D in /mnt/hdd01/ci/compatibility-deploy/be/lib/doris_be
5# 0x000055700470B2DA in /mnt/hdd01/ci/compatibility-deploy/be/lib/doris_be
6# google::LogMessage::SendToLog() in /mnt/hdd01/ci/compatibility-deploy/be/lib/doris_be
7# google::LogMessage::Flush() in /mnt/hdd01/ci/compatibility-deploy/be/lib/doris_be
8# google::LogMessageFatal::~LogMessageFatal() in /mnt/hdd01/ci/compatibility-deploy/be/lib/doris_be
9# doris::DeleteHandler::init(std::shared_ptr<doris::TabletSchema>, std::vector<std::shared_ptr<doris::RowsetMeta>, std::allocator<std::shared_ptr<doris::RowsetMeta> > > const&, long, bool) at /home/zcp/repo_center/doris_master/doris/be/src/olap/delete_handler.cpp:388
10# doris::TabletReader::_init_delete_condition(doris::TabletReader::ReaderParams const&) at /home/zcp/repo_center/doris_master/doris/be/src/olap/tablet_reader.cpp:654
11# doris::TabletReader::_init_params(doris::TabletReader::ReaderParams const&) at /home/zcp/repo_center/doris_master/doris/be/src/olap/tablet_reader.cpp:295
12# doris::TabletReader::init(doris::TabletReader::ReaderParams const&) at /home/zcp/repo_center/doris_master/doris/be/src/olap/tablet_reader.cpp:128
13# doris::vectorized::BlockReader::init(doris::TabletReader::ReaderParams const&) in /mnt/hdd01/ci/compatibility-deploy/be/lib/doris_be
14# doris::vectorized::NewOlapScanner::open(doris::RuntimeState*) at /home/zcp/repo_center/doris_master/doris/be/src/vec/exec/scan/new_olap_scanner.cpp:219
15# doris::vectorized::ScannerScheduler::_scanner_scan(std::shared_ptr<doris::vectorized::ScannerContext>, std::shared_ptr<doris::vectorized::ScanTask>) at /home/zcp/repo_center/doris_master/doris/be/src/vec/exec/scan/scanner_scheduler.cpp:250
16# std::_Function_handler<void (), doris::vectorized::ScannerScheduler::submit(std::shared_ptr<doris::vectorized::ScannerContext>, std::shared_ptr<doris::vectorized::ScanTask>)::$_1::operator()() const::{lambda()#1}>::_M_invoke(std::_Any_data const&) at /var/local/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/std_function.h:291
17# doris::ThreadPool::dispatch_thread() in /mnt/hdd01/ci/compatibility-deploy/be/lib/doris_be
18# doris::Thread::supervise_thread(void*) at /home/zcp/repo_center/doris_master/doris/be/src/util/thread.cpp:499
19# start_thread at /build/glibc-SzIz7B/glibc-2.31/nptl/pthread_create.c:478
20# __clone at ../sysdeps/unix/sysv/linux/x86_64/clone.S:97
```
related PRs: #36090,
#36101,
#36314
dataroaring
pushed a commit
that referenced
this pull request
Jul 17, 2024
## Proposed changes
```
F20240628 01:49:16.382710 4183685 delete_handler.cpp:388] Check failed: !_is_inited reinitialize delete handler.
*** Check failure stack trace: ***
@ 0x55700470e3c6 google::LogMessage::SendToLog()
@ 0x55700470ae10 google::LogMessage::Flush()
@ 0x55700470ec09 google::LogMessageFatal::~LogMessageFatal()
@ 0x556fccf40e64 doris::DeleteHandler::init()
@ 0x556fcff46678 doris::TabletReader::_init_delete_condition()
@ 0x556fcff3a2dd doris::TabletReader::_init_params()
@ 0x556fcff39432 doris::TabletReader::init()
@ 0x556fffb8c2dd doris::vectorized::BlockReader::init()
@ 0x557002cca96a doris::vectorized::NewOlapScanner::open()
@ 0x556fe892d565 doris::vectorized::ScannerScheduler::_scanner_scan()
@ 0x556fe8931a0f _ZNSt17_Function_handlerIFvvEZZN5doris10vectorized16ScannerScheduler6submitESt10shared_ptrINS2_14ScannerContextEES4_INS2_8ScanTaskEEENK3$_1clEvEUlvE_E9_M_invokeERKSt9_Any_data
@ 0x556fd0ed95dc doris::ThreadPool::dispatch_thread()
@ 0x556fd0eb1288 doris::Thread::supervise_thread()
@ 0x7f95143b5609 start_thread
@ 0x7f9514662133 clone
@ (nil) (unknown)
*** Query id: c389fc2a1ff6473c-a06f032b8970810c ***
*** is nereids: 1 ***
*** tablet id: 0 ***
*** Aborted at 1719510556 (unix time) try "date -d @1719510556" if you are using GNU date ***
*** Current BE git commitID: b13c17d ***
*** SIGABRT unknown detail explain (@0x3fca33) received by PID 4180531 (TID 4183685 OR 0x7f89734a5700) from PID 4180531; stack trace: ***
0# doris::signal::(anonymous namespace)::FailureSignalHandler(int, siginfo_t*, void*) at /home/zcp/repo_center/doris_master/doris/be/src/common/signal_handler.h:421
1# 0x00007F9514586090 in /lib/x86_64-linux-gnu/libc.so.6
2# raise at ../sysdeps/unix/sysv/linux/raise.c:51
3# abort at /build/glibc-SzIz7B/glibc-2.31/stdlib/abort.c:81
4# 0x0000557004718C9D in /mnt/hdd01/ci/compatibility-deploy/be/lib/doris_be
5# 0x000055700470B2DA in /mnt/hdd01/ci/compatibility-deploy/be/lib/doris_be
6# google::LogMessage::SendToLog() in /mnt/hdd01/ci/compatibility-deploy/be/lib/doris_be
7# google::LogMessage::Flush() in /mnt/hdd01/ci/compatibility-deploy/be/lib/doris_be
8# google::LogMessageFatal::~LogMessageFatal() in /mnt/hdd01/ci/compatibility-deploy/be/lib/doris_be
9# doris::DeleteHandler::init(std::shared_ptr<doris::TabletSchema>, std::vector<std::shared_ptr<doris::RowsetMeta>, std::allocator<std::shared_ptr<doris::RowsetMeta> > > const&, long, bool) at /home/zcp/repo_center/doris_master/doris/be/src/olap/delete_handler.cpp:388
10# doris::TabletReader::_init_delete_condition(doris::TabletReader::ReaderParams const&) at /home/zcp/repo_center/doris_master/doris/be/src/olap/tablet_reader.cpp:654
11# doris::TabletReader::_init_params(doris::TabletReader::ReaderParams const&) at /home/zcp/repo_center/doris_master/doris/be/src/olap/tablet_reader.cpp:295
12# doris::TabletReader::init(doris::TabletReader::ReaderParams const&) at /home/zcp/repo_center/doris_master/doris/be/src/olap/tablet_reader.cpp:128
13# doris::vectorized::BlockReader::init(doris::TabletReader::ReaderParams const&) in /mnt/hdd01/ci/compatibility-deploy/be/lib/doris_be
14# doris::vectorized::NewOlapScanner::open(doris::RuntimeState*) at /home/zcp/repo_center/doris_master/doris/be/src/vec/exec/scan/new_olap_scanner.cpp:219
15# doris::vectorized::ScannerScheduler::_scanner_scan(std::shared_ptr<doris::vectorized::ScannerContext>, std::shared_ptr<doris::vectorized::ScanTask>) at /home/zcp/repo_center/doris_master/doris/be/src/vec/exec/scan/scanner_scheduler.cpp:250
16# std::_Function_handler<void (), doris::vectorized::ScannerScheduler::submit(std::shared_ptr<doris::vectorized::ScannerContext>, std::shared_ptr<doris::vectorized::ScanTask>)::$_1::operator()() const::{lambda()#1}>::_M_invoke(std::_Any_data const&) at /var/local/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/std_function.h:291
17# doris::ThreadPool::dispatch_thread() in /mnt/hdd01/ci/compatibility-deploy/be/lib/doris_be
18# doris::Thread::supervise_thread(void*) at /home/zcp/repo_center/doris_master/doris/be/src/util/thread.cpp:499
19# start_thread at /build/glibc-SzIz7B/glibc-2.31/nptl/pthread_create.c:478
20# __clone at ../sysdeps/unix/sysv/linux/x86_64/clone.S:97
```
related PRs: #36090,
#36101,
#36314
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
ASSIGN_STATUS_IF_CATCH_EXCEPTIONwill convert error code fromMEM_ALLOC_FAILEDtoMEM_LIMIT_EXCEEDED, this will lead to problem like below: