Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

居然是Java还是ant #4

Closed
luyee opened this issue Aug 11, 2017 · 1 comment
Closed

居然是Java还是ant #4

luyee opened this issue Aug 11, 2017 · 1 comment

Comments

@luyee
Copy link

luyee commented Aug 11, 2017

居然不是maven，也不是gradle!
@luyee luyee closed this as completed Aug 12, 2017
@maruyue
Copy link
Contributor

maruyue commented Aug 14, 2017

为什么选用 ant，而不是 maven 和 graddle？考虑如下：

ant 更简单，maven 和 graddle 使用更复杂
对于很多在内网使用我们产品的同学，使用 maven 和 graddle 去下载外面的 jar 包，尤其遇到一些 jar 包还在国外封禁网站的，更是困难重重。很多公司的内网，是不能随意有外网权限的。
palo 内置了一些 jar 包，不需要大家去下载，大大节省了大家的下载时间，用起来更方便。

我们在选用一些技术时，做了自己的一些考虑，不是带来实际收益，我们一般不会考虑引入。这里还是追求简洁有效。

@wangbo wangbo mentioned this issue Sep 30, 2019
Merged
@wangbo wangbo mentioned this issue Oct 11, 2019
Merged
@wangbo wangbo mentioned this issue Oct 22, 2019
Closed
@zhangy5 zhangy5 mentioned this issue Oct 22, 2019
Closed
@zhangy5 zhangy5 mentioned this issue Nov 6, 2019
Closed
@HangyuanLiu HangyuanLiu mentioned this issue Nov 13, 2019
Closed
@kangpinghuang kangpinghuang mentioned this issue Nov 14, 2019
Closed
@jmk1011 jmk1011 mentioned this issue Dec 23, 2019
Closed
@worker24h worker24h mentioned this issue Feb 17, 2020
Closed
wangbo added a commit to wangbo/incubator-doris that referenced this issue Apr 27, 2020
@wangbo
Merge pull request apache#4 from wangbo/hkp_spark_etl_fe_dev
7f1048c 
1 add hll,hll udaf,replace udaf
@marising marising mentioned this issue Jun 1, 2020
Closed
@morningman morningman mentioned this issue Jun 23, 2020
Open
morningman referenced this issue in morningman/doris Aug 3, 2020
@hf200012
Merge pull request #4 from morningman/doris-ui-doc
c36211b 
Add restful api docs and modify some codes
@ZhangYu0123 ZhangYu0123 mentioned this issue Sep 3, 2020
Closed
@caiconghui caiconghui mentioned this issue Mar 3, 2021
Closed
@xinghuayu007 xinghuayu007 mentioned this issue Apr 16, 2021
Open
@xqinghu xqinghu mentioned this issue May 20, 2021
Closed
@xinyiZzz xinyiZzz mentioned this issue Dec 16, 2021
Closed
3 tasks
@caiconghui caiconghui mentioned this issue Dec 19, 2021
Open
@larryju larryju mentioned this issue Feb 24, 2022
Open
3 tasks
@caiconghui caiconghui mentioned this issue Mar 21, 2022
Closed
3 tasks
This was referenced Mar 30, 2022
Open
Open
@xinyiZzz xinyiZzz mentioned this issue Apr 8, 2022
Closed
3 tasks
@xinyiZzz xinyiZzz mentioned this issue Apr 25, 2022
Open
3 tasks
morningman referenced this issue in morningman/doris May 4, 2022
@morningman
[fix](ut) fix a potential memory leak in BE ut
cbd8383 
```
 ==12681==ERROR: LeakSanitizer: detected memory leaks
 Direct leak of 128 byte(s) in 1 object(s) allocated from:
     #0 0x562b93d0fc57 in operator new(unsigned long) (/root/doris/be/ut_build_ASAN/test/doris_be_test+0x16e02c57)
     #1 0x562b955fd176 in doris::ArrowWorkFlowTest::init_runtime_state() /root/doris/be/test/util/arrow/arrow_work_flow_test.cpp:92
     #2 0x562b955fcf58 in doris::ArrowWorkFlowTest::init() /root/doris/be/test/util/arrow/arrow_work_flow_test.cpp:87
     #3 0x562b9560ba7d in doris::ArrowWorkFlowTest::SetUp() /root/doris/be/test/util/arrow/arrow_work_flow_test.cpp:61
     #4 0x562b9f38263c in void testing::internal::HandleSehExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) (/root/doris/be/ut_build_ASAN/test/
     #5 0x562b9f37ca78 in void testing::internal::HandleExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) (/root/doris/be/ut_build_ASAN/test/
     #6 0x562b9f3540e1 in testing::Test::Run() (/root/doris/be/ut_build_ASAN/test/doris_be_test+0x224470e1)
     #7 0x562b9f354b7f in testing::TestInfo::Run() (/root/doris/be/ut_build_ASAN/test/doris_be_test+0x22447b7f)
     apache#8 0x562b9f35543f in testing::TestSuite::Run() (/root/doris/be/ut_build_ASAN/test/doris_be_test+0x2244843f)
     apache#9 0x562b9f364aff in testing::internal::UnitTestImpl::RunAllTests() (/root/doris/be/ut_build_ASAN/test/doris_be_test+0x22457aff)
     apache#10 0x562b9f3833ed in bool testing::internal::HandleSehExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (Impl::*)(), char const*) (/root/doris/be/ut_build_ASAN/test/doris_be_test+0x224763ed)
     apache#11 0x562b9f37d932 in bool testing::internal::HandleExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), e/ut_build_ASAN/test/doris_be_test+0x22470932)
     apache#12 0x562b9f3632f4 in testing::UnitTest::Run() (/root/doris/be/ut_build_ASAN/test/doris_be_test+0x224562f4)
     apache#13 0x562b95369a7c in RUN_ALL_TESTS() /var/local/thirdparty/installed/include/gtest/gtest.h:2490
     apache#14 0x562b95366048 in main /root/doris/be/test/testutil/run_all_tests.cpp:43
     apache#15 0x7f40dcbb6554 in __libc_start_main (/lib64/libc.so.6+0x22554)
 Direct leak of 104 byte(s) in 1 object(s) allocated from:
     #0 0x562b93d0fc57 in operator new(unsigned long) (/root/doris/be/ut_build_ASAN/test/doris_be_test+0x16e02c57)
     #1 0x562b955fd213 in doris::ArrowWorkFlowTest::init_runtime_state() /root/doris/be/test/util/arrow/arrow_work_flow_test.cpp:93
     #2 0x562b955fcf58 in doris::ArrowWorkFlowTest::init() /root/doris/be/test/util/arrow/arrow_work_flow_test.cpp:87
     #3 0x562b9560ba7d in doris::ArrowWorkFlowTest::SetUp() /root/doris/be/test/util/arrow/arrow_work_flow_test.cpp:61
     #4 0x562b9f38263c in void testing::internal::HandleSehExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) (/root/doris/be/ut_build_ASAN/test/
     #5 0x562b9f37ca78 in void testing::internal::HandleExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) (/root/doris/be/ut_build_ASAN/test/
     #6 0x562b9f3540e1 in testing::Test::Run() (/root/doris/be/ut_build_ASAN/test/doris_be_test+0x224470e1)
     #7 0x562b9f354b7f in testing::TestInfo::Run() (/root/doris/be/ut_build_ASAN/test/doris_be_test+0x22447b7f)
     apache#8 0x562b9f35543f in testing::TestSuite::Run() (/root/doris/be/ut_build_ASAN/test/doris_be_test+0x2244843f)
     apache#9 0x562b9f364aff in testing::internal::UnitTestImpl::RunAllTests() (/root/doris/be/ut_build_ASAN/test/doris_be_test+0x22457aff)
     apache#10 0x562b9f3833ed in bool testing::internal::HandleSehExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (Impl::*)(), char const*) (/root/doris/be/ut_build_ASAN/test/doris_be_test+0x224763ed)
     apache#11 0x562b9f37d932 in bool testing::internal::HandleExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), e/ut_build_ASAN/test/doris_be_test+0x22470932)
     apache#12 0x562b9f3632f4 in testing::UnitTest::Run() (/root/doris/be/ut_build_ASAN/test/doris_be_test+0x224562f4)
     apache#13 0x562b95369a7c in RUN_ALL_TESTS() /var/local/thirdparty/installed/include/gtest/gtest.h:2490
     apache#14 0x562b95366048 in main /root/doris/be/test/testutil/run_all_tests.cpp:43
     apache#15 0x7f40dcbb6554 in __libc_start_main (/lib64/libc.so.6+0x22554)
 SUMMARY: AddressSanitizer: 232 byte(s) leaked in 2 allocation(s).
```
@xinyiZzz xinyiZzz mentioned this issue May 5, 2022
Closed
3 tasks
@Lchangliang Lchangliang mentioned this issue May 13, 2022
Closed
3 tasks
@sollhui sollhui mentioned this issue Dec 16, 2024
Merged
16 tasks
morningman pushed a commit that referenced this issue Dec 31, 2024
@kaka11chen
[Fix](multi-catalog) Fix column mutate() crash replace it by assume_m…
ed61d66 
…utable(). (#46151)

### What problem does this PR solve?

Problem Summary:

```
warning: Unable to find libthread_db matching inferior's thread library, thread debugging will not be available.
Core was generated by `/mnt/doris/be/lib/doris_be'.
Program terminated with signal SIGSEGV, Segmentation fault.
#0  0x0000000000000000 in ?? ()
[Current thread is 1 (LWP 3923404)]
(gdb) bt
#0  0x0000000000000000 in ?? ()
#1  0x000055f44f97dda7 in COW<doris::vectorized::IColumn>::release_ref (this=0x7f6bf7d07cc8) at /home/zcp/repo_center/doris_enterprise/doris/be/src/vec/common/cow.h:99
#2  COW<doris::vectorized::IColumn>::intrusive_ptr<doris::vectorized::IColumn>::~intrusive_ptr (this=0x7f6b792f9670) at /home/zcp/repo_center/doris_enterprise/doris/be/src/vec/common/cow.h:133
#3  doris::vectorized::ScalarColumnReader::_read_nested_column (this=this@entry=0x7f6be31f8900, doris_column=..., type=
    std::shared_ptr<const doris::vectorized::IDataType> (use count 1, weak count 0) = {...}, select_vector=..., batch_size=<optimized out>, batch_size@entry=4064, read_rows=0x7f6b792f9ad8, 
    eof=0x7f6b792f9af0, is_dict_filter=<optimized out>, align_rows=<optimized out>) at /home/zcp/repo_center/doris_enterprise/doris/be/src/vec/exec/format/parquet/vparquet_column_reader.cpp:447
#4  0x000055f44f97e1fc in doris::vectorized::ScalarColumnReader::read_column_data (this=0x7f6be31f8900, doris_column=..., 
    type=std::shared_ptr<const doris::vectorized::IDataType> (use count 1, weak count 0) = {...}, select_vector=..., batch_size=4064, read_rows=0x7f6b792f9ad8, eof=0x7f6b792f9af0, 
    is_dict_filter=<optimized out>) at /home/zcp/repo_center/doris_enterprise/doris/be/src/vec/exec/format/parquet/vparquet_column_reader.cpp:504
#5  0x000055f44f97ecbe in doris::vectorized::ArrayColumnReader::read_column_data (this=0x7f69a922ce00, doris_column=..., type=..., select_vector=..., batch_size=4064, read_rows=0x7f6b792f9ad8, 
    eof=0x7f6b792f9af0, is_dict_filter=<optimized out>) at /home/zcp/repo_center/doris_enterprise/doris/be/src/vec/exec/format/parquet/vparquet_column_reader.cpp:616
#6  0x000055f44f975460 in doris::vectorized::RowGroupReader::_read_column_data (this=this@entry=0x7f6cf83dd180, block=block@entry=0x7f6bbcc66938, columns=..., batch_size=4064, 
    read_rows=read_rows@entry=0x7f6b792f9ee0, batch_eof=0x7f6cf83d71f0, select_vector=...) at /home/zcp/repo_center/doris_enterprise/doris/be/src/vec/exec/format/parquet/vparquet_group_reader.cpp:426
#7  0x000055f44f972682 in doris::vectorized::RowGroupReader::next_batch (this=0x7f6cf83dd180, block=0x7f6bbcc66938, batch_size=140099571418880, read_rows=0x7f6b792f9ee0, batch_eof=0x7f6f06724610)
    at /home/zcp/repo_center/doris_enterprise/doris/be/src/vec/exec/format/parquet/vparquet_group_reader.cpp:321
#8  0x000055f44f9141f8 in doris::vectorized::ParquetReader::get_next_block (this=0x7f6cf83d7000, block=0x7f6bbcc66938, read_rows=0x7f6b792f9ee0, eof=0x7f6bbcc66f88)
    at /home/zcp/repo_center/doris_enterprise/doris/be/src/vec/exec/format/parquet/vparquet_reader.cpp:576
#9  0x000055f450aba6f4 in doris::vectorized::IcebergTableReader::get_next_block (this=0x7f6bb5611180, block=0x7f6bbcc66938, read_rows=0x7f6b792f9ee0, eof=0x7f6bbcc66f88)
    at /home/zcp/repo_center/doris_enterprise/doris/be/src/vec/exec/format/table/iceberg_reader.cpp:138
#10 0x000055f450aaa351 in doris::vectorized::VFileScanner::_get_block_wrapped (this=0x7f6bbcc66800, state=<optimized out>, block=0x7f6bbcc66938, eof=0x7f6b792fa2f7)
    at /home/zcp/repo_center/doris_enterprise/doris/be/src/vec/exec/scan/vfile_scanner.cpp:359
#11 0x000055f450aa9ecc in doris::vectorized::VFileScanner::_get_block_impl (this=0x0, state=0xffffffffffffa770, block=0x8c3de0, eof=0x7f6b79300700)
    at /home/zcp/repo_center/doris_enterprise/doris/be/src/vec/exec/scan/vfile_scanner.cpp:301
#12 0x000055f450b41e9c in doris::vectorized::VScanner::get_block (this=this@entry=0x7f6bbcc66800, state=state@entry=0x7f6f06724000, block=block@entry=0x7f6bbcc66938, eof=eof@entry=0x7f6b792fa2f7)
    at /home/zcp/repo_center/doris_enterprise/doris/be/src/vec/exec/scan/vscanner.cpp:133
#13 0x000055f450b41977 in doris::vectorized::VScanner::get_block_after_projects (this=0x7f6bbcc66800, state=0x7f6f06724000, block=0x7f6cf8394b80, eos=0x7f6b792fa2f7)
    at /home/zcp/repo_center/doris_enterprise/doris/be/src/vec/exec/scan/vscanner.cpp:96
#14 0x000055f450a941ff in doris::vectorized::ScannerScheduler::_scanner_scan (ctx=std::shared_ptr<doris::vectorized::ScannerContext> (use count 10, weak count 1) = {...}, 
    scan_task=std::shared_ptr<doris::vectorized::ScanTask> (use count 2, weak count 0) = {...}) at /home/zcp/repo_center/doris_enterprise/doris/be/src/vec/exec/scan/scanner_scheduler.cpp:289
#15 0x000055f450a94b73 in doris::vectorized::ScannerScheduler::submit(std::shared_ptr<doris::vectorized::ScannerContext>, std::shared_ptr<doris::vectorized::ScanTask>)::$_1::operator()() const::{lambda()#1}::operator()() const::{lambda()#2}::operator()() const (this=<optimized out>) at /home/zcp/repo_center/doris_enterprise/doris/be/src/vec/exec/scan/scanner_scheduler.cpp:180
#16 doris::vectorized::ScannerScheduler::submit(std::shared_ptr<doris::vectorized::ScannerContext>, std::shared_ptr<doris::vectorized::ScanTask>)::$_1::operator()() const::{lambda()#1}::operator()() const (this=0x7f70ccc56ee0) at /home/zcp/repo_center/doris_enterprise/doris/be/src/vec/exec/scan/scanner_scheduler.cpp:179
#17 std::__invoke_impl<void, doris::vectorized::ScannerScheduler::submit(std::shared_ptr<doris::vectorized::ScannerContext>, std::shared_ptr<doris::vectorized::ScanTask>)::$_1::operator()() const::{lambda()#1}&>(std::__invoke_other, doris::vectorized::ScannerScheduler::submit(std::shared_ptr<doris::vectorized::ScannerContext>, std::shared_ptr<doris::vectorized::ScanTask>)::$_1::operator()() const::{lambda()#1}&) (__f=...) at /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/invoke.h:61
#18 std::__invoke_r<void, doris::vectorized::ScannerScheduler::submit(std::shared_ptr<doris::vectorized::ScannerContext>, std::shared_ptr<doris::vectorized::ScanTask>)::$_1::operator()() const::{lambda()#1}&>(doris::vectorized::ScannerScheduler::submit(std::shared_ptr<doris::vectorized::ScannerContext>, std::shared_ptr<doris::vectorized::ScanTask>)::$_1::operator()() const::{lambda()#1}&) (__fn=...)
    at /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/invoke.h:111
#19 std::_Function_handler<void (), doris::vectorized::ScannerScheduler::submit(std::shared_ptr<doris::vectorized::ScannerContext>, std::shared_ptr<doris::vectorized::ScanTask>)::$_1::operator()() const::{lambda()#1}>::_M_invoke(std::_Any_data const&) (__functor=...) at /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/std_function.h:291
#20 0x000055f44ca858c8 in doris::ThreadPool::dispatch_thread (this=0x7f70ba259200) at /home/zcp/repo_center/doris_enterprise/doris/be/src/util/threadpool.cpp:543
#21 0x000055f44ca7ad91 in std::function<void ()>::operator()() const (this=0x7f6bf7d07cc0)
    at /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/std_function.h:560
#22 doris::Thread::supervise_thread (arg=0x7f6fa185a020) at /home/zcp/repo_center/doris_enterprise/doris/be/src/util/thread.cpp:498
#23 0x00007f70ebf78e25 in ?? ()
#24 0x0000000000000000 in ?? ()
```
github-actions bot pushed a commit that referenced this issue Dec 31, 2024
@kaka11chen
[Fix](multi-catalog) Fix column mutate() crash replace it by assume_m…
6a131a8 
…utable(). (#46151)

### What problem does this PR solve?

Problem Summary:

```
warning: Unable to find libthread_db matching inferior's thread library, thread debugging will not be available.
Core was generated by `/mnt/doris/be/lib/doris_be'.
Program terminated with signal SIGSEGV, Segmentation fault.
#0  0x0000000000000000 in ?? ()
[Current thread is 1 (LWP 3923404)]
(gdb) bt
#0  0x0000000000000000 in ?? ()
#1  0x000055f44f97dda7 in COW<doris::vectorized::IColumn>::release_ref (this=0x7f6bf7d07cc8) at /home/zcp/repo_center/doris_enterprise/doris/be/src/vec/common/cow.h:99
#2  COW<doris::vectorized::IColumn>::intrusive_ptr<doris::vectorized::IColumn>::~intrusive_ptr (this=0x7f6b792f9670) at /home/zcp/repo_center/doris_enterprise/doris/be/src/vec/common/cow.h:133
#3  doris::vectorized::ScalarColumnReader::_read_nested_column (this=this@entry=0x7f6be31f8900, doris_column=..., type=
    std::shared_ptr<const doris::vectorized::IDataType> (use count 1, weak count 0) = {...}, select_vector=..., batch_size=<optimized out>, batch_size@entry=4064, read_rows=0x7f6b792f9ad8, 
    eof=0x7f6b792f9af0, is_dict_filter=<optimized out>, align_rows=<optimized out>) at /home/zcp/repo_center/doris_enterprise/doris/be/src/vec/exec/format/parquet/vparquet_column_reader.cpp:447
#4  0x000055f44f97e1fc in doris::vectorized::ScalarColumnReader::read_column_data (this=0x7f6be31f8900, doris_column=..., 
    type=std::shared_ptr<const doris::vectorized::IDataType> (use count 1, weak count 0) = {...}, select_vector=..., batch_size=4064, read_rows=0x7f6b792f9ad8, eof=0x7f6b792f9af0, 
    is_dict_filter=<optimized out>) at /home/zcp/repo_center/doris_enterprise/doris/be/src/vec/exec/format/parquet/vparquet_column_reader.cpp:504
#5  0x000055f44f97ecbe in doris::vectorized::ArrayColumnReader::read_column_data (this=0x7f69a922ce00, doris_column=..., type=..., select_vector=..., batch_size=4064, read_rows=0x7f6b792f9ad8, 
    eof=0x7f6b792f9af0, is_dict_filter=<optimized out>) at /home/zcp/repo_center/doris_enterprise/doris/be/src/vec/exec/format/parquet/vparquet_column_reader.cpp:616
#6  0x000055f44f975460 in doris::vectorized::RowGroupReader::_read_column_data (this=this@entry=0x7f6cf83dd180, block=block@entry=0x7f6bbcc66938, columns=..., batch_size=4064, 
    read_rows=read_rows@entry=0x7f6b792f9ee0, batch_eof=0x7f6cf83d71f0, select_vector=...) at /home/zcp/repo_center/doris_enterprise/doris/be/src/vec/exec/format/parquet/vparquet_group_reader.cpp:426
#7  0x000055f44f972682 in doris::vectorized::RowGroupReader::next_batch (this=0x7f6cf83dd180, block=0x7f6bbcc66938, batch_size=140099571418880, read_rows=0x7f6b792f9ee0, batch_eof=0x7f6f06724610)
    at /home/zcp/repo_center/doris_enterprise/doris/be/src/vec/exec/format/parquet/vparquet_group_reader.cpp:321
#8  0x000055f44f9141f8 in doris::vectorized::ParquetReader::get_next_block (this=0x7f6cf83d7000, block=0x7f6bbcc66938, read_rows=0x7f6b792f9ee0, eof=0x7f6bbcc66f88)
    at /home/zcp/repo_center/doris_enterprise/doris/be/src/vec/exec/format/parquet/vparquet_reader.cpp:576
#9  0x000055f450aba6f4 in doris::vectorized::IcebergTableReader::get_next_block (this=0x7f6bb5611180, block=0x7f6bbcc66938, read_rows=0x7f6b792f9ee0, eof=0x7f6bbcc66f88)
    at /home/zcp/repo_center/doris_enterprise/doris/be/src/vec/exec/format/table/iceberg_reader.cpp:138
#10 0x000055f450aaa351 in doris::vectorized::VFileScanner::_get_block_wrapped (this=0x7f6bbcc66800, state=<optimized out>, block=0x7f6bbcc66938, eof=0x7f6b792fa2f7)
    at /home/zcp/repo_center/doris_enterprise/doris/be/src/vec/exec/scan/vfile_scanner.cpp:359
#11 0x000055f450aa9ecc in doris::vectorized::VFileScanner::_get_block_impl (this=0x0, state=0xffffffffffffa770, block=0x8c3de0, eof=0x7f6b79300700)
    at /home/zcp/repo_center/doris_enterprise/doris/be/src/vec/exec/scan/vfile_scanner.cpp:301
#12 0x000055f450b41e9c in doris::vectorized::VScanner::get_block (this=this@entry=0x7f6bbcc66800, state=state@entry=0x7f6f06724000, block=block@entry=0x7f6bbcc66938, eof=eof@entry=0x7f6b792fa2f7)
    at /home/zcp/repo_center/doris_enterprise/doris/be/src/vec/exec/scan/vscanner.cpp:133
#13 0x000055f450b41977 in doris::vectorized::VScanner::get_block_after_projects (this=0x7f6bbcc66800, state=0x7f6f06724000, block=0x7f6cf8394b80, eos=0x7f6b792fa2f7)
    at /home/zcp/repo_center/doris_enterprise/doris/be/src/vec/exec/scan/vscanner.cpp:96
#14 0x000055f450a941ff in doris::vectorized::ScannerScheduler::_scanner_scan (ctx=std::shared_ptr<doris::vectorized::ScannerContext> (use count 10, weak count 1) = {...}, 
    scan_task=std::shared_ptr<doris::vectorized::ScanTask> (use count 2, weak count 0) = {...}) at /home/zcp/repo_center/doris_enterprise/doris/be/src/vec/exec/scan/scanner_scheduler.cpp:289
#15 0x000055f450a94b73 in doris::vectorized::ScannerScheduler::submit(std::shared_ptr<doris::vectorized::ScannerContext>, std::shared_ptr<doris::vectorized::ScanTask>)::$_1::operator()() const::{lambda()#1}::operator()() const::{lambda()#2}::operator()() const (this=<optimized out>) at /home/zcp/repo_center/doris_enterprise/doris/be/src/vec/exec/scan/scanner_scheduler.cpp:180
#16 doris::vectorized::ScannerScheduler::submit(std::shared_ptr<doris::vectorized::ScannerContext>, std::shared_ptr<doris::vectorized::ScanTask>)::$_1::operator()() const::{lambda()#1}::operator()() const (this=0x7f70ccc56ee0) at /home/zcp/repo_center/doris_enterprise/doris/be/src/vec/exec/scan/scanner_scheduler.cpp:179
#17 std::__invoke_impl<void, doris::vectorized::ScannerScheduler::submit(std::shared_ptr<doris::vectorized::ScannerContext>, std::shared_ptr<doris::vectorized::ScanTask>)::$_1::operator()() const::{lambda()#1}&>(std::__invoke_other, doris::vectorized::ScannerScheduler::submit(std::shared_ptr<doris::vectorized::ScannerContext>, std::shared_ptr<doris::vectorized::ScanTask>)::$_1::operator()() const::{lambda()#1}&) (__f=...) at /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/invoke.h:61
#18 std::__invoke_r<void, doris::vectorized::ScannerScheduler::submit(std::shared_ptr<doris::vectorized::ScannerContext>, std::shared_ptr<doris::vectorized::ScanTask>)::$_1::operator()() const::{lambda()#1}&>(doris::vectorized::ScannerScheduler::submit(std::shared_ptr<doris::vectorized::ScannerContext>, std::shared_ptr<doris::vectorized::ScanTask>)::$_1::operator()() const::{lambda()#1}&) (__fn=...)
    at /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/invoke.h:111
#19 std::_Function_handler<void (), doris::vectorized::ScannerScheduler::submit(std::shared_ptr<doris::vectorized::ScannerContext>, std::shared_ptr<doris::vectorized::ScanTask>)::$_1::operator()() const::{lambda()#1}>::_M_invoke(std::_Any_data const&) (__functor=...) at /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/std_function.h:291
#20 0x000055f44ca858c8 in doris::ThreadPool::dispatch_thread (this=0x7f70ba259200) at /home/zcp/repo_center/doris_enterprise/doris/be/src/util/threadpool.cpp:543
#21 0x000055f44ca7ad91 in std::function<void ()>::operator()() const (this=0x7f6bf7d07cc0)
    at /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/std_function.h:560
#22 doris::Thread::supervise_thread (arg=0x7f6fa185a020) at /home/zcp/repo_center/doris_enterprise/doris/be/src/util/thread.cpp:498
#23 0x00007f70ebf78e25 in ?? ()
#24 0x0000000000000000 in ?? ()
```
github-actions bot pushed a commit that referenced this issue Dec 31, 2024
@kaka11chen
[Fix](multi-catalog) Fix column mutate() crash replace it by assume_m…
d666f99 
…utable(). (#46151)

### What problem does this PR solve?

Problem Summary:

```
warning: Unable to find libthread_db matching inferior's thread library, thread debugging will not be available.
Core was generated by `/mnt/doris/be/lib/doris_be'.
Program terminated with signal SIGSEGV, Segmentation fault.
#0  0x0000000000000000 in ?? ()
[Current thread is 1 (LWP 3923404)]
(gdb) bt
#0  0x0000000000000000 in ?? ()
#1  0x000055f44f97dda7 in COW<doris::vectorized::IColumn>::release_ref (this=0x7f6bf7d07cc8) at /home/zcp/repo_center/doris_enterprise/doris/be/src/vec/common/cow.h:99
#2  COW<doris::vectorized::IColumn>::intrusive_ptr<doris::vectorized::IColumn>::~intrusive_ptr (this=0x7f6b792f9670) at /home/zcp/repo_center/doris_enterprise/doris/be/src/vec/common/cow.h:133
#3  doris::vectorized::ScalarColumnReader::_read_nested_column (this=this@entry=0x7f6be31f8900, doris_column=..., type=
    std::shared_ptr<const doris::vectorized::IDataType> (use count 1, weak count 0) = {...}, select_vector=..., batch_size=<optimized out>, batch_size@entry=4064, read_rows=0x7f6b792f9ad8, 
    eof=0x7f6b792f9af0, is_dict_filter=<optimized out>, align_rows=<optimized out>) at /home/zcp/repo_center/doris_enterprise/doris/be/src/vec/exec/format/parquet/vparquet_column_reader.cpp:447
#4  0x000055f44f97e1fc in doris::vectorized::ScalarColumnReader::read_column_data (this=0x7f6be31f8900, doris_column=..., 
    type=std::shared_ptr<const doris::vectorized::IDataType> (use count 1, weak count 0) = {...}, select_vector=..., batch_size=4064, read_rows=0x7f6b792f9ad8, eof=0x7f6b792f9af0, 
    is_dict_filter=<optimized out>) at /home/zcp/repo_center/doris_enterprise/doris/be/src/vec/exec/format/parquet/vparquet_column_reader.cpp:504
#5  0x000055f44f97ecbe in doris::vectorized::ArrayColumnReader::read_column_data (this=0x7f69a922ce00, doris_column=..., type=..., select_vector=..., batch_size=4064, read_rows=0x7f6b792f9ad8, 
    eof=0x7f6b792f9af0, is_dict_filter=<optimized out>) at /home/zcp/repo_center/doris_enterprise/doris/be/src/vec/exec/format/parquet/vparquet_column_reader.cpp:616
#6  0x000055f44f975460 in doris::vectorized::RowGroupReader::_read_column_data (this=this@entry=0x7f6cf83dd180, block=block@entry=0x7f6bbcc66938, columns=..., batch_size=4064, 
    read_rows=read_rows@entry=0x7f6b792f9ee0, batch_eof=0x7f6cf83d71f0, select_vector=...) at /home/zcp/repo_center/doris_enterprise/doris/be/src/vec/exec/format/parquet/vparquet_group_reader.cpp:426
#7  0x000055f44f972682 in doris::vectorized::RowGroupReader::next_batch (this=0x7f6cf83dd180, block=0x7f6bbcc66938, batch_size=140099571418880, read_rows=0x7f6b792f9ee0, batch_eof=0x7f6f06724610)
    at /home/zcp/repo_center/doris_enterprise/doris/be/src/vec/exec/format/parquet/vparquet_group_reader.cpp:321
#8  0x000055f44f9141f8 in doris::vectorized::ParquetReader::get_next_block (this=0x7f6cf83d7000, block=0x7f6bbcc66938, read_rows=0x7f6b792f9ee0, eof=0x7f6bbcc66f88)
    at /home/zcp/repo_center/doris_enterprise/doris/be/src/vec/exec/format/parquet/vparquet_reader.cpp:576
#9  0x000055f450aba6f4 in doris::vectorized::IcebergTableReader::get_next_block (this=0x7f6bb5611180, block=0x7f6bbcc66938, read_rows=0x7f6b792f9ee0, eof=0x7f6bbcc66f88)
    at /home/zcp/repo_center/doris_enterprise/doris/be/src/vec/exec/format/table/iceberg_reader.cpp:138
#10 0x000055f450aaa351 in doris::vectorized::VFileScanner::_get_block_wrapped (this=0x7f6bbcc66800, state=<optimized out>, block=0x7f6bbcc66938, eof=0x7f6b792fa2f7)
    at /home/zcp/repo_center/doris_enterprise/doris/be/src/vec/exec/scan/vfile_scanner.cpp:359
#11 0x000055f450aa9ecc in doris::vectorized::VFileScanner::_get_block_impl (this=0x0, state=0xffffffffffffa770, block=0x8c3de0, eof=0x7f6b79300700)
    at /home/zcp/repo_center/doris_enterprise/doris/be/src/vec/exec/scan/vfile_scanner.cpp:301
#12 0x000055f450b41e9c in doris::vectorized::VScanner::get_block (this=this@entry=0x7f6bbcc66800, state=state@entry=0x7f6f06724000, block=block@entry=0x7f6bbcc66938, eof=eof@entry=0x7f6b792fa2f7)
    at /home/zcp/repo_center/doris_enterprise/doris/be/src/vec/exec/scan/vscanner.cpp:133
#13 0x000055f450b41977 in doris::vectorized::VScanner::get_block_after_projects (this=0x7f6bbcc66800, state=0x7f6f06724000, block=0x7f6cf8394b80, eos=0x7f6b792fa2f7)
    at /home/zcp/repo_center/doris_enterprise/doris/be/src/vec/exec/scan/vscanner.cpp:96
#14 0x000055f450a941ff in doris::vectorized::ScannerScheduler::_scanner_scan (ctx=std::shared_ptr<doris::vectorized::ScannerContext> (use count 10, weak count 1) = {...}, 
    scan_task=std::shared_ptr<doris::vectorized::ScanTask> (use count 2, weak count 0) = {...}) at /home/zcp/repo_center/doris_enterprise/doris/be/src/vec/exec/scan/scanner_scheduler.cpp:289
#15 0x000055f450a94b73 in doris::vectorized::ScannerScheduler::submit(std::shared_ptr<doris::vectorized::ScannerContext>, std::shared_ptr<doris::vectorized::ScanTask>)::$_1::operator()() const::{lambda()#1}::operator()() const::{lambda()#2}::operator()() const (this=<optimized out>) at /home/zcp/repo_center/doris_enterprise/doris/be/src/vec/exec/scan/scanner_scheduler.cpp:180
#16 doris::vectorized::ScannerScheduler::submit(std::shared_ptr<doris::vectorized::ScannerContext>, std::shared_ptr<doris::vectorized::ScanTask>)::$_1::operator()() const::{lambda()#1}::operator()() const (this=0x7f70ccc56ee0) at /home/zcp/repo_center/doris_enterprise/doris/be/src/vec/exec/scan/scanner_scheduler.cpp:179
#17 std::__invoke_impl<void, doris::vectorized::ScannerScheduler::submit(std::shared_ptr<doris::vectorized::ScannerContext>, std::shared_ptr<doris::vectorized::ScanTask>)::$_1::operator()() const::{lambda()#1}&>(std::__invoke_other, doris::vectorized::ScannerScheduler::submit(std::shared_ptr<doris::vectorized::ScannerContext>, std::shared_ptr<doris::vectorized::ScanTask>)::$_1::operator()() const::{lambda()#1}&) (__f=...) at /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/invoke.h:61
#18 std::__invoke_r<void, doris::vectorized::ScannerScheduler::submit(std::shared_ptr<doris::vectorized::ScannerContext>, std::shared_ptr<doris::vectorized::ScanTask>)::$_1::operator()() const::{lambda()#1}&>(doris::vectorized::ScannerScheduler::submit(std::shared_ptr<doris::vectorized::ScannerContext>, std::shared_ptr<doris::vectorized::ScanTask>)::$_1::operator()() const::{lambda()#1}&) (__fn=...)
    at /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/invoke.h:111
#19 std::_Function_handler<void (), doris::vectorized::ScannerScheduler::submit(std::shared_ptr<doris::vectorized::ScannerContext>, std::shared_ptr<doris::vectorized::ScanTask>)::$_1::operator()() const::{lambda()#1}>::_M_invoke(std::_Any_data const&) (__functor=...) at /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/std_function.h:291
#20 0x000055f44ca858c8 in doris::ThreadPool::dispatch_thread (this=0x7f70ba259200) at /home/zcp/repo_center/doris_enterprise/doris/be/src/util/threadpool.cpp:543
#21 0x000055f44ca7ad91 in std::function<void ()>::operator()() const (this=0x7f6bf7d07cc0)
    at /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/std_function.h:560
#22 doris::Thread::supervise_thread (arg=0x7f6fa185a020) at /home/zcp/repo_center/doris_enterprise/doris/be/src/util/thread.cpp:498
#23 0x00007f70ebf78e25 in ?? ()
#24 0x0000000000000000 in ?? ()
```
BiteTheDDDDt added a commit that referenced this issue Jan 3, 2025
@BiteTheDDDDt
[Bug](scan) do not release tablet_reader on NewOlapScanner::close (#4…
7380005 
…6296)

### What problem does this PR solve?

```cpp
172.20.50.85 be.out: #8 0x56435579ab5e in lucene::store::BufferedIndexInput::readBytes(unsigned char*, int, bool) /home/zcp/repo_center/doris_branch-3.0/doris/be/src/clucene/src/core/CLucene/store/IndexInput.cpp:192:9
    #9 0x564319f7d95d in doris::segment_v2::CSIndexInput::readInternal(unsigned char*, int) /home/zcp/repo_center/doris_branch-3.0/doris/be/src/olap/rowset/segment_v2/inverted_index_compound_reader.cpp:107:11
    #10 0x56435579bb5b in lucene::store::BufferedIndexInput::refill() /home/zcp/repo_center/doris_branch-3.0/doris/be/src/clucene/src/core/CLucene/store/IndexInput.cpp:285:5
    #11 0x56435579b096 in lucene::store::BufferedIndexInput::readBytes(unsigned char*, int, int, bool) /home/zcp/repo_center/doris_branch-3.0/doris/be/src/clucene/src/core/CLucene/store/IndexInput.cpp:217:9
    #12 0x56435579a9f6 in lucene::store::BufferedIndexInput::readBytes(unsigned char*, int) /home/zcp/repo_center/doris_branch-3.0/doris/be/src/clucene/src/core/CLucene/store/IndexInput.cpp:186:9
    #13 0x564319fe8aee in doris::segment_v2::InvertedIndexReader::read_null_bitmap(doris::io::IOContext const*, doris::OlapReaderStatistics*, doris::segment_v2::InvertedIndexQueryCacheHandle*, lucene::store::Directory*) /home/zcp/repo_center/doris_branch-3.0/doris/be/src/olap/rowset/segment_v2/inverted_index_reader.cpp:144:29
    #14 0x564319febab4 in doris::segment_v2::InvertedIndexReader::handle_searcher_cache(doris::segment_v2::InvertedIndexCacheHandle*, doris::io::IOContext const*, doris::OlapReaderStatistics*) /home/zcp/repo_center/doris_branch-3.0/doris/be/src/olap/rowset/segment_v2/inverted_index_reader.cpp:194:27
    #15 0x564319ff4b0b in doris::segment_v2::StringTypeInvertedIndexReader::query(doris::io::IOContext const*, doris::OlapReaderStatistics*, doris::RuntimeState*, std::__cxx11::basic_string, std::allocator> const&, void const*, doris::segment_v2::InvertedIndexQueryType, std::shared_ptr&) /home/zcp/repo_center/doris_branch-3.0/doris/be/src/olap/rowset/segment_v2/inverted_index_reader.cpp:407:5
    #16 0x56431a001f00 in doris::segment_v2::InvertedIndexIterator::read_from_inverted_index(std::__cxx11::basic_string, std::allocator> const&, void const*, doris::segment_v2::InvertedIndexQueryType, unsigned int, std::shared_ptr&, bool) /home/zcp/repo_center/doris_branch-3.0/doris/be/src/olap/rowset/segment_v2/inverted_index_reader.cpp:1137:5
    #17 0x5643191f4a52 in doris::ComparisonPredicateBase<(doris::PrimitiveType)23, (doris::PredicateType)1>::evaluate(std::pair, std::allocator>, std::shared_ptr> const&, doris::segment_v2::InvertedIndexIterator*, unsigned int, roaring::Roaring*) const /home/zcp/repo_center/doris_branch-3.0/doris/be/src/olap/comparison_predicate.h:107:9
    #18 0x56431a249ef7 in doris::segment_v2::SegmentIterator::_apply_inverted_index_on_column_predicate(doris::ColumnPredicate*, std::vector>&, bool*) /home/zcp/repo_center/doris_branch-3.0/doris/be/src/olap/rowset/segment_v2/segment_iterator.cpp:871:28
    #19 0x56431a23ddc0 in doris::segment_v2::SegmentIterator::_apply_inverted_index() /home/zcp/repo_center/doris_branch-3.0/doris/be/src/olap/rowset/segment_v2/segment_iterator.cpp:951:13
    #20 0x56431a22f956 in doris::segment_v2::SegmentIterator::_get_row_ranges_by_column_conditions() /home/zcp/repo_center/doris_branch-3.0/doris/be/src/olap/rowset/segment_v2/segment_iterator.cpp:512:13
    #21 0x56431a22b78f in doris::segment_v2::SegmentIterator::_lazy_init() /home/zcp/repo_center/doris_branch-3.0/doris/be/src/olap/rowset/segment_v2/segment_iterator.cpp:383:5
    #22 0x56431a26c667 in doris::segment_v2::SegmentIterator::_next_batch_internal(doris::vectorized::Block*) /home/zcp/repo_center/doris_branch-3.0/doris/be/src/olap/rowset/segment_v2/segment_iterator.cpp:2008:9
    #23 0x56431a264fb6 in doris::segment_v2::SegmentIterator::next_batch(doris::vectorized::Block*)::$_0::operator()() const /home/zcp/repo_center/doris_branch-3.0/doris/be/src/olap/rowset/segment_v2/segment_iterator.cpp:1917:9
    #24 0x56431a264fb6 in doris::segment_v2::SegmentIterator::next_batch(doris::vectorized::Block*) /home/zcp/repo_center/doris_branch-3.0/doris/be/src/olap/rowset/segment_v2/segment_iterator.cpp:1916:19
    #25 0x56431a08315c in doris::segment_v2::LazyInitSegmentIterator::next_batch(doris::vectorized::Block*) /home/zcp/repo_center/doris_branch-3.0/doris/be/src/olap/rowset/segment_v2/lazy_init_segment_iterator.h:44:33
    #26 0x564319b95855 in doris::BetaRowsetReader::next_block(doris::vectorized::Block*) /home/zcp/repo_center/doris_branch-3.0/doris/be/src/olap/rowset/beta_rowset_reader.cpp:366:29
    #27 0x56434e3a0c10 in doris::vectorized::VCollectIterator::Level0Iterator::_refresh() /home/zcp/repo_center/doris_branch-3.0/doris/be/src/vec/olap/vcollect_iterator.h
    #28 0x56434e38ba5b in doris::vectorized::VCollectIterator::Level0Iterator::refresh_current_row() /home/zcp/repo_center/doris_branch-3.0/doris/be/src/vec/olap/vcollect_iterator.cpp:509:24
    #29 0x56434e38cd76 in doris::vectorized::VCollectIterator::Level0Iterator::ensure_first_row_ref() /home/zcp/repo_center/doris_branch-3.0/doris/be/src/vec/olap/vcollect_iterator.cpp:482:14
    #30 0x56434e399745 in doris::vectorized::VCollectIterator::Level1Iterator::ensure_first_row_ref() /home/zcp/repo_center/doris_branch-3.0/doris/be/src/vec/olap/vcollect_iterator.cpp:696:27
    #31 0x56434e37f7f9 in doris::vectorized::VCollectIterator::build_heap(std::vector, std::allocator>>&) /home/zcp/repo_center/doris_branch-3.0/doris/be/src/vec/olap/vcollect_iterator.cpp:186:9
    #32 0x56434e312fe2 in doris::vectorized::BlockReader::_init_collect_iter(doris::TabletReader::ReaderParams const&) /home/zcp/repo_center/doris_branch-3.0/doris/be/src/vec/olap/block_reader.cpp:140:5
    #33 0x56434e3165df in doris::vectorized::BlockReader::init(doris::TabletReader::ReaderParams const&) /home/zcp/repo_center/doris_branch-3.0/doris/be/src/vec/olap/block_reader.cpp:212:19
    #34 0x5643515aba04 in doris::vectorized::NewOlapScanner::open(doris::RuntimeState*) /home/zcp/repo_center/doris_branch-3.0/doris/be/src/vec/exec/scan/new_olap_scanner.cpp:231:32
    #35 0x564336c138c6 in doris::vectorized::ScannerScheduler::_scanner_scan(std::shared_ptr, std::shared_ptr) /home/zcp/repo_center/doris_branch-3.0/doris/be/src/vec/exec/scan/scanner_scheduler.cpp:247:5
    #36 0x564336c18db0 in doris::vectorized::ScannerScheduler::submit(std::shared_ptr, std::shared_ptr)::$_1::operator()() const::'lambda'()::operator()() const::'lambda0'()::operator()() const /home/zcp/repo_center/doris_branch-3.0/doris/be/src/vec/exec/scan/scanner_scheduler.cpp:180:21
    #37 0x564336c18db0 in doris::vectorized::ScannerScheduler::submit(std::shared_ptr, std::shared_ptr)::$_1::operator()() const::'lambda'()::operator()() const /home/zcp/repo_center/doris_branch-3.0/doris/be/src/vec/exec/scan/scanner_scheduler.cpp:179:31
    #38 0x564336c18db0 in void std::__invoke_impl, std::shared_ptr)::$_1::operator()() const::'lambda'()&>(std::__invoke_other, doris::vectorized::ScannerScheduler::submit(std::shared_ptr, std::shared_ptr)::$_1::operator()() const::'lambda'()&) /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/invoke.h:61:14
    #39 0x564336c18db0 in std::enable_if, std::shared_ptr)::$_1::operator()() const::'lambda'()&>, void>::type std::__invoke_r, std::shared_ptr)::$_1::operator()() const::'lambda'()&>(doris::vectorized::ScannerScheduler::submit(std::shared_ptr, std::shared_ptr)::$_1::operator()() const::'lambda'()&) /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/invoke.h:111:2
    #40 0x564336c18db0 in std::_Function_handler, std::shared_ptr)::$_1::operator()() const::'lambda'()>::_M_invoke(std::_Any_data const&) /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/std_function.h:291:9
    #41 0x56431b9d94eb in doris::ThreadPool::dispatch_thread() /home/zcp/repo_center/doris_branch-3.0/doris/be/src/util/threadpool.cpp:543:24
    #42 0x56431b9b18c7 in doris::Thread::supervise_thread(void*) /home/zcp/repo_center/doris_branch-3.0/doris/be/src/util/thread.cpp:498:5
    #43 0x7f31d3b87ac2 in start_thread nptl/pthread_create.c:442:8
    #44 0x7f31d3c1984f  misc/../sysdeps/unix/sysv/linux/x86_64/clone3.S:81

0x61d000a7f3c8 is located 1352 bytes inside of 2064-byte region [0x61d000a7ee80,0x61d000a7f690)
freed by thread T1698 (Pipe_normal [wo) here:
    #0 0x564317278d9d in operator delete(void*) (/mnt/hdd01/ci/doris-deploy-branch-3.0-cloud/be/lib/doris_be+0x3626bd9d) (BuildId: 9dab40b94b1dc995)
    #1 0x5643515b192c in std::unique_ptr>::reset(doris::TabletReader*) /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/unique_ptr.h:456:7
    #2 0x5643515b192c in doris::vectorized::NewOlapScanner::close(doris::RuntimeState*) /home/zcp/repo_center/doris_branch-3.0/doris/be/src/vec/exec/scan/new_olap_scanner.cpp:534:20
    #3 0x564352457463 in doris::vectorized::ScannerDelegate::~ScannerDelegate() /home/zcp/repo_center/doris_branch-3.0/doris/be/src/vec/exec/scan/vscan_node.h:35:31
    #4 0x5643172a8324 in std::_Sp_counted_base<(__gnu_cxx::_Lock_policy)2>::_M_release() /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/shared_ptr_base.h:168:6
    #5 0x56434e7d7870 in std::__shared_ptr::~__shared_ptr() /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/shared_ptr_base.h:1149:31
    #6 0x56434e7d7870 in void std::destroy_at>(std::shared_ptr*) /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/stl_construct.h:88:15
    #7 0x56434e7d7870 in void std::allocator_traits>>>::destroy>(std::allocator>>&, std::shared_ptr*) /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/alloc_traits.h:533:4
    #8 0x56434e7d7870 in std::__cxx11::_List_base, std::allocator>>::_M_clear() /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/list.tcc:77:4
    #9 0x564351db11d2 in std::__cxx11::_List_base, std::allocator>>::~_List_base() /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/stl_list.h:499:9
    #10 0x564351db11d2 in doris::pipeline::ScanLocalState::close(doris::RuntimeState*) /home/zcp/repo_center/doris_branch-3.0/doris/be/src/pipeline/exec/scan_operator.cpp:1261:5
    #11 0x56434e6357e3 in doris::pipeline::OperatorXBase::close(doris::RuntimeState*) /home/zcp/repo_center/doris_branch-3.0/doris/be/src/pipeline/exec/operator.cpp:245:28
    #12 0x564352ad00c1 in doris::pipeline::PipelineTask::close(doris::Status) /home/zcp/repo_center/doris_branch-3.0/doris/be/src/pipeline/pipeline_task.cpp:489:28
    #13 0x564352b0701f in doris::pipeline::_close_task(doris::pipeline::PipelineTask*, doris::Status) /home/zcp/repo_center/doris_branch-3.0/doris/be/src/pipeline/task_scheduler.cpp:91:27
    #14 0x564352b095f3 in doris::pipeline::TaskScheduler::_do_work(unsigned long) /home/zcp/repo_center/doris_branch-3.0/doris/be/src/pipeline/task_scheduler.cpp:181:17
    #15 0x56431b9d94eb in doris::ThreadPool::dispatch_thread() /home/zcp/repo_center/doris_branch-3.0/doris/be/src/util/threadpool.cpp:543:24
    #16 0x56431b9b18c7 in doris::Thread::supervise_thread(void*) /home/zcp/repo_center/doris_branch-3.0/doris/be/src/util/thread.cpp:498:5
    #17 0x7f31d3b87ac2 in start_thread nptl/pthread_create.c:442:8

```

Problem Summary:

### Release note

None

### Check List (For Author)

- Test <!-- At least one of them must be included. -->
    - [ ] Regression test
    - [ ] Unit Test
    - [ ] Manual test (add detailed scripts or steps below)
    - [ ] No need to test or manual test. Explain why:
- [ ] This is a refactor/code format and no logic has been changed.
        - [ ] Previous test can cover this change.
        - [ ] No code files have been changed.
        - [ ] Other reason <!-- Add your reason?  -->

- Behavior changed:
    - [ ] No.
    - [ ] Yes. <!-- Explain the behavior change -->

- Does this need documentation?
    - [ ] No.
- [ ] Yes. <!-- Add document PR link here. eg:
apache/doris-website#1214 -->

### Check List (For Reviewer who merge this PR)

- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label <!-- Add branch pick label that this PR
should merge into -->
github-actions bot pushed a commit that referenced this issue Jan 3, 2025
@BiteTheDDDDt
[Bug](scan) do not release tablet_reader on NewOlapScanner::close (#4…
6cf8a55 
…6296)

### What problem does this PR solve?

```cpp
172.20.50.85 be.out: #8 0x56435579ab5e in lucene::store::BufferedIndexInput::readBytes(unsigned char*, int, bool) /home/zcp/repo_center/doris_branch-3.0/doris/be/src/clucene/src/core/CLucene/store/IndexInput.cpp:192:9
    #9 0x564319f7d95d in doris::segment_v2::CSIndexInput::readInternal(unsigned char*, int) /home/zcp/repo_center/doris_branch-3.0/doris/be/src/olap/rowset/segment_v2/inverted_index_compound_reader.cpp:107:11
    #10 0x56435579bb5b in lucene::store::BufferedIndexInput::refill() /home/zcp/repo_center/doris_branch-3.0/doris/be/src/clucene/src/core/CLucene/store/IndexInput.cpp:285:5
    #11 0x56435579b096 in lucene::store::BufferedIndexInput::readBytes(unsigned char*, int, int, bool) /home/zcp/repo_center/doris_branch-3.0/doris/be/src/clucene/src/core/CLucene/store/IndexInput.cpp:217:9
    #12 0x56435579a9f6 in lucene::store::BufferedIndexInput::readBytes(unsigned char*, int) /home/zcp/repo_center/doris_branch-3.0/doris/be/src/clucene/src/core/CLucene/store/IndexInput.cpp:186:9
    #13 0x564319fe8aee in doris::segment_v2::InvertedIndexReader::read_null_bitmap(doris::io::IOContext const*, doris::OlapReaderStatistics*, doris::segment_v2::InvertedIndexQueryCacheHandle*, lucene::store::Directory*) /home/zcp/repo_center/doris_branch-3.0/doris/be/src/olap/rowset/segment_v2/inverted_index_reader.cpp:144:29
    #14 0x564319febab4 in doris::segment_v2::InvertedIndexReader::handle_searcher_cache(doris::segment_v2::InvertedIndexCacheHandle*, doris::io::IOContext const*, doris::OlapReaderStatistics*) /home/zcp/repo_center/doris_branch-3.0/doris/be/src/olap/rowset/segment_v2/inverted_index_reader.cpp:194:27
    #15 0x564319ff4b0b in doris::segment_v2::StringTypeInvertedIndexReader::query(doris::io::IOContext const*, doris::OlapReaderStatistics*, doris::RuntimeState*, std::__cxx11::basic_string, std::allocator> const&, void const*, doris::segment_v2::InvertedIndexQueryType, std::shared_ptr&) /home/zcp/repo_center/doris_branch-3.0/doris/be/src/olap/rowset/segment_v2/inverted_index_reader.cpp:407:5
    #16 0x56431a001f00 in doris::segment_v2::InvertedIndexIterator::read_from_inverted_index(std::__cxx11::basic_string, std::allocator> const&, void const*, doris::segment_v2::InvertedIndexQueryType, unsigned int, std::shared_ptr&, bool) /home/zcp/repo_center/doris_branch-3.0/doris/be/src/olap/rowset/segment_v2/inverted_index_reader.cpp:1137:5
    #17 0x5643191f4a52 in doris::ComparisonPredicateBase<(doris::PrimitiveType)23, (doris::PredicateType)1>::evaluate(std::pair, std::allocator>, std::shared_ptr> const&, doris::segment_v2::InvertedIndexIterator*, unsigned int, roaring::Roaring*) const /home/zcp/repo_center/doris_branch-3.0/doris/be/src/olap/comparison_predicate.h:107:9
    #18 0x56431a249ef7 in doris::segment_v2::SegmentIterator::_apply_inverted_index_on_column_predicate(doris::ColumnPredicate*, std::vector>&, bool*) /home/zcp/repo_center/doris_branch-3.0/doris/be/src/olap/rowset/segment_v2/segment_iterator.cpp:871:28
    #19 0x56431a23ddc0 in doris::segment_v2::SegmentIterator::_apply_inverted_index() /home/zcp/repo_center/doris_branch-3.0/doris/be/src/olap/rowset/segment_v2/segment_iterator.cpp:951:13
    #20 0x56431a22f956 in doris::segment_v2::SegmentIterator::_get_row_ranges_by_column_conditions() /home/zcp/repo_center/doris_branch-3.0/doris/be/src/olap/rowset/segment_v2/segment_iterator.cpp:512:13
    #21 0x56431a22b78f in doris::segment_v2::SegmentIterator::_lazy_init() /home/zcp/repo_center/doris_branch-3.0/doris/be/src/olap/rowset/segment_v2/segment_iterator.cpp:383:5
    #22 0x56431a26c667 in doris::segment_v2::SegmentIterator::_next_batch_internal(doris::vectorized::Block*) /home/zcp/repo_center/doris_branch-3.0/doris/be/src/olap/rowset/segment_v2/segment_iterator.cpp:2008:9
    #23 0x56431a264fb6 in doris::segment_v2::SegmentIterator::next_batch(doris::vectorized::Block*)::$_0::operator()() const /home/zcp/repo_center/doris_branch-3.0/doris/be/src/olap/rowset/segment_v2/segment_iterator.cpp:1917:9
    #24 0x56431a264fb6 in doris::segment_v2::SegmentIterator::next_batch(doris::vectorized::Block*) /home/zcp/repo_center/doris_branch-3.0/doris/be/src/olap/rowset/segment_v2/segment_iterator.cpp:1916:19
    #25 0x56431a08315c in doris::segment_v2::LazyInitSegmentIterator::next_batch(doris::vectorized::Block*) /home/zcp/repo_center/doris_branch-3.0/doris/be/src/olap/rowset/segment_v2/lazy_init_segment_iterator.h:44:33
    #26 0x564319b95855 in doris::BetaRowsetReader::next_block(doris::vectorized::Block*) /home/zcp/repo_center/doris_branch-3.0/doris/be/src/olap/rowset/beta_rowset_reader.cpp:366:29
    #27 0x56434e3a0c10 in doris::vectorized::VCollectIterator::Level0Iterator::_refresh() /home/zcp/repo_center/doris_branch-3.0/doris/be/src/vec/olap/vcollect_iterator.h
    #28 0x56434e38ba5b in doris::vectorized::VCollectIterator::Level0Iterator::refresh_current_row() /home/zcp/repo_center/doris_branch-3.0/doris/be/src/vec/olap/vcollect_iterator.cpp:509:24
    #29 0x56434e38cd76 in doris::vectorized::VCollectIterator::Level0Iterator::ensure_first_row_ref() /home/zcp/repo_center/doris_branch-3.0/doris/be/src/vec/olap/vcollect_iterator.cpp:482:14
    #30 0x56434e399745 in doris::vectorized::VCollectIterator::Level1Iterator::ensure_first_row_ref() /home/zcp/repo_center/doris_branch-3.0/doris/be/src/vec/olap/vcollect_iterator.cpp:696:27
    #31 0x56434e37f7f9 in doris::vectorized::VCollectIterator::build_heap(std::vector, std::allocator>>&) /home/zcp/repo_center/doris_branch-3.0/doris/be/src/vec/olap/vcollect_iterator.cpp:186:9
    #32 0x56434e312fe2 in doris::vectorized::BlockReader::_init_collect_iter(doris::TabletReader::ReaderParams const&) /home/zcp/repo_center/doris_branch-3.0/doris/be/src/vec/olap/block_reader.cpp:140:5
    #33 0x56434e3165df in doris::vectorized::BlockReader::init(doris::TabletReader::ReaderParams const&) /home/zcp/repo_center/doris_branch-3.0/doris/be/src/vec/olap/block_reader.cpp:212:19
    #34 0x5643515aba04 in doris::vectorized::NewOlapScanner::open(doris::RuntimeState*) /home/zcp/repo_center/doris_branch-3.0/doris/be/src/vec/exec/scan/new_olap_scanner.cpp:231:32
    #35 0x564336c138c6 in doris::vectorized::ScannerScheduler::_scanner_scan(std::shared_ptr, std::shared_ptr) /home/zcp/repo_center/doris_branch-3.0/doris/be/src/vec/exec/scan/scanner_scheduler.cpp:247:5
    #36 0x564336c18db0 in doris::vectorized::ScannerScheduler::submit(std::shared_ptr, std::shared_ptr)::$_1::operator()() const::'lambda'()::operator()() const::'lambda0'()::operator()() const /home/zcp/repo_center/doris_branch-3.0/doris/be/src/vec/exec/scan/scanner_scheduler.cpp:180:21
    #37 0x564336c18db0 in doris::vectorized::ScannerScheduler::submit(std::shared_ptr, std::shared_ptr)::$_1::operator()() const::'lambda'()::operator()() const /home/zcp/repo_center/doris_branch-3.0/doris/be/src/vec/exec/scan/scanner_scheduler.cpp:179:31
    #38 0x564336c18db0 in void std::__invoke_impl, std::shared_ptr)::$_1::operator()() const::'lambda'()&>(std::__invoke_other, doris::vectorized::ScannerScheduler::submit(std::shared_ptr, std::shared_ptr)::$_1::operator()() const::'lambda'()&) /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/invoke.h:61:14
    #39 0x564336c18db0 in std::enable_if, std::shared_ptr)::$_1::operator()() const::'lambda'()&>, void>::type std::__invoke_r, std::shared_ptr)::$_1::operator()() const::'lambda'()&>(doris::vectorized::ScannerScheduler::submit(std::shared_ptr, std::shared_ptr)::$_1::operator()() const::'lambda'()&) /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/invoke.h:111:2
    #40 0x564336c18db0 in std::_Function_handler, std::shared_ptr)::$_1::operator()() const::'lambda'()>::_M_invoke(std::_Any_data const&) /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/std_function.h:291:9
    #41 0x56431b9d94eb in doris::ThreadPool::dispatch_thread() /home/zcp/repo_center/doris_branch-3.0/doris/be/src/util/threadpool.cpp:543:24
    #42 0x56431b9b18c7 in doris::Thread::supervise_thread(void*) /home/zcp/repo_center/doris_branch-3.0/doris/be/src/util/thread.cpp:498:5
    #43 0x7f31d3b87ac2 in start_thread nptl/pthread_create.c:442:8
    #44 0x7f31d3c1984f  misc/../sysdeps/unix/sysv/linux/x86_64/clone3.S:81

0x61d000a7f3c8 is located 1352 bytes inside of 2064-byte region [0x61d000a7ee80,0x61d000a7f690)
freed by thread T1698 (Pipe_normal [wo) here:
    #0 0x564317278d9d in operator delete(void*) (/mnt/hdd01/ci/doris-deploy-branch-3.0-cloud/be/lib/doris_be+0x3626bd9d) (BuildId: 9dab40b94b1dc995)
    #1 0x5643515b192c in std::unique_ptr>::reset(doris::TabletReader*) /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/unique_ptr.h:456:7
    #2 0x5643515b192c in doris::vectorized::NewOlapScanner::close(doris::RuntimeState*) /home/zcp/repo_center/doris_branch-3.0/doris/be/src/vec/exec/scan/new_olap_scanner.cpp:534:20
    #3 0x564352457463 in doris::vectorized::ScannerDelegate::~ScannerDelegate() /home/zcp/repo_center/doris_branch-3.0/doris/be/src/vec/exec/scan/vscan_node.h:35:31
    #4 0x5643172a8324 in std::_Sp_counted_base<(__gnu_cxx::_Lock_policy)2>::_M_release() /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/shared_ptr_base.h:168:6
    #5 0x56434e7d7870 in std::__shared_ptr::~__shared_ptr() /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/shared_ptr_base.h:1149:31
    #6 0x56434e7d7870 in void std::destroy_at>(std::shared_ptr*) /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/stl_construct.h:88:15
    #7 0x56434e7d7870 in void std::allocator_traits>>>::destroy>(std::allocator>>&, std::shared_ptr*) /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/alloc_traits.h:533:4
    #8 0x56434e7d7870 in std::__cxx11::_List_base, std::allocator>>::_M_clear() /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/list.tcc:77:4
    #9 0x564351db11d2 in std::__cxx11::_List_base, std::allocator>>::~_List_base() /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/stl_list.h:499:9
    #10 0x564351db11d2 in doris::pipeline::ScanLocalState::close(doris::RuntimeState*) /home/zcp/repo_center/doris_branch-3.0/doris/be/src/pipeline/exec/scan_operator.cpp:1261:5
    #11 0x56434e6357e3 in doris::pipeline::OperatorXBase::close(doris::RuntimeState*) /home/zcp/repo_center/doris_branch-3.0/doris/be/src/pipeline/exec/operator.cpp:245:28
    #12 0x564352ad00c1 in doris::pipeline::PipelineTask::close(doris::Status) /home/zcp/repo_center/doris_branch-3.0/doris/be/src/pipeline/pipeline_task.cpp:489:28
    #13 0x564352b0701f in doris::pipeline::_close_task(doris::pipeline::PipelineTask*, doris::Status) /home/zcp/repo_center/doris_branch-3.0/doris/be/src/pipeline/task_scheduler.cpp:91:27
    #14 0x564352b095f3 in doris::pipeline::TaskScheduler::_do_work(unsigned long) /home/zcp/repo_center/doris_branch-3.0/doris/be/src/pipeline/task_scheduler.cpp:181:17
    #15 0x56431b9d94eb in doris::ThreadPool::dispatch_thread() /home/zcp/repo_center/doris_branch-3.0/doris/be/src/util/threadpool.cpp:543:24
    #16 0x56431b9b18c7 in doris::Thread::supervise_thread(void*) /home/zcp/repo_center/doris_branch-3.0/doris/be/src/util/thread.cpp:498:5
    #17 0x7f31d3b87ac2 in start_thread nptl/pthread_create.c:442:8

```

Problem Summary:

### Release note

None

### Check List (For Author)

- Test <!-- At least one of them must be included. -->
    - [ ] Regression test
    - [ ] Unit Test
    - [ ] Manual test (add detailed scripts or steps below)
    - [ ] No need to test or manual test. Explain why:
- [ ] This is a refactor/code format and no logic has been changed.
        - [ ] Previous test can cover this change.
        - [ ] No code files have been changed.
        - [ ] Other reason <!-- Add your reason?  -->

- Behavior changed:
    - [ ] No.
    - [ ] Yes. <!-- Explain the behavior change -->

- Does this need documentation?
    - [ ] No.
- [ ] Yes. <!-- Add document PR link here. eg:
apache/doris-website#1214 -->

### Check List (For Reviewer who merge this PR)

- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label <!-- Add branch pick label that this PR
should merge into -->
github-actions bot pushed a commit that referenced this issue Jan 3, 2025
@BiteTheDDDDt
[Bug](scan) do not release tablet_reader on NewOlapScanner::close (#4…
bf84196 
…6296)

### What problem does this PR solve?

```cpp
172.20.50.85 be.out: #8 0x56435579ab5e in lucene::store::BufferedIndexInput::readBytes(unsigned char*, int, bool) /home/zcp/repo_center/doris_branch-3.0/doris/be/src/clucene/src/core/CLucene/store/IndexInput.cpp:192:9
    #9 0x564319f7d95d in doris::segment_v2::CSIndexInput::readInternal(unsigned char*, int) /home/zcp/repo_center/doris_branch-3.0/doris/be/src/olap/rowset/segment_v2/inverted_index_compound_reader.cpp:107:11
    #10 0x56435579bb5b in lucene::store::BufferedIndexInput::refill() /home/zcp/repo_center/doris_branch-3.0/doris/be/src/clucene/src/core/CLucene/store/IndexInput.cpp:285:5
    #11 0x56435579b096 in lucene::store::BufferedIndexInput::readBytes(unsigned char*, int, int, bool) /home/zcp/repo_center/doris_branch-3.0/doris/be/src/clucene/src/core/CLucene/store/IndexInput.cpp:217:9
    #12 0x56435579a9f6 in lucene::store::BufferedIndexInput::readBytes(unsigned char*, int) /home/zcp/repo_center/doris_branch-3.0/doris/be/src/clucene/src/core/CLucene/store/IndexInput.cpp:186:9
    #13 0x564319fe8aee in doris::segment_v2::InvertedIndexReader::read_null_bitmap(doris::io::IOContext const*, doris::OlapReaderStatistics*, doris::segment_v2::InvertedIndexQueryCacheHandle*, lucene::store::Directory*) /home/zcp/repo_center/doris_branch-3.0/doris/be/src/olap/rowset/segment_v2/inverted_index_reader.cpp:144:29
    #14 0x564319febab4 in doris::segment_v2::InvertedIndexReader::handle_searcher_cache(doris::segment_v2::InvertedIndexCacheHandle*, doris::io::IOContext const*, doris::OlapReaderStatistics*) /home/zcp/repo_center/doris_branch-3.0/doris/be/src/olap/rowset/segment_v2/inverted_index_reader.cpp:194:27
    #15 0x564319ff4b0b in doris::segment_v2::StringTypeInvertedIndexReader::query(doris::io::IOContext const*, doris::OlapReaderStatistics*, doris::RuntimeState*, std::__cxx11::basic_string, std::allocator> const&, void const*, doris::segment_v2::InvertedIndexQueryType, std::shared_ptr&) /home/zcp/repo_center/doris_branch-3.0/doris/be/src/olap/rowset/segment_v2/inverted_index_reader.cpp:407:5
    #16 0x56431a001f00 in doris::segment_v2::InvertedIndexIterator::read_from_inverted_index(std::__cxx11::basic_string, std::allocator> const&, void const*, doris::segment_v2::InvertedIndexQueryType, unsigned int, std::shared_ptr&, bool) /home/zcp/repo_center/doris_branch-3.0/doris/be/src/olap/rowset/segment_v2/inverted_index_reader.cpp:1137:5
    #17 0x5643191f4a52 in doris::ComparisonPredicateBase<(doris::PrimitiveType)23, (doris::PredicateType)1>::evaluate(std::pair, std::allocator>, std::shared_ptr> const&, doris::segment_v2::InvertedIndexIterator*, unsigned int, roaring::Roaring*) const /home/zcp/repo_center/doris_branch-3.0/doris/be/src/olap/comparison_predicate.h:107:9
    #18 0x56431a249ef7 in doris::segment_v2::SegmentIterator::_apply_inverted_index_on_column_predicate(doris::ColumnPredicate*, std::vector>&, bool*) /home/zcp/repo_center/doris_branch-3.0/doris/be/src/olap/rowset/segment_v2/segment_iterator.cpp:871:28
    #19 0x56431a23ddc0 in doris::segment_v2::SegmentIterator::_apply_inverted_index() /home/zcp/repo_center/doris_branch-3.0/doris/be/src/olap/rowset/segment_v2/segment_iterator.cpp:951:13
    #20 0x56431a22f956 in doris::segment_v2::SegmentIterator::_get_row_ranges_by_column_conditions() /home/zcp/repo_center/doris_branch-3.0/doris/be/src/olap/rowset/segment_v2/segment_iterator.cpp:512:13
    #21 0x56431a22b78f in doris::segment_v2::SegmentIterator::_lazy_init() /home/zcp/repo_center/doris_branch-3.0/doris/be/src/olap/rowset/segment_v2/segment_iterator.cpp:383:5
    #22 0x56431a26c667 in doris::segment_v2::SegmentIterator::_next_batch_internal(doris::vectorized::Block*) /home/zcp/repo_center/doris_branch-3.0/doris/be/src/olap/rowset/segment_v2/segment_iterator.cpp:2008:9
    #23 0x56431a264fb6 in doris::segment_v2::SegmentIterator::next_batch(doris::vectorized::Block*)::$_0::operator()() const /home/zcp/repo_center/doris_branch-3.0/doris/be/src/olap/rowset/segment_v2/segment_iterator.cpp:1917:9
    #24 0x56431a264fb6 in doris::segment_v2::SegmentIterator::next_batch(doris::vectorized::Block*) /home/zcp/repo_center/doris_branch-3.0/doris/be/src/olap/rowset/segment_v2/segment_iterator.cpp:1916:19
    #25 0x56431a08315c in doris::segment_v2::LazyInitSegmentIterator::next_batch(doris::vectorized::Block*) /home/zcp/repo_center/doris_branch-3.0/doris/be/src/olap/rowset/segment_v2/lazy_init_segment_iterator.h:44:33
    #26 0x564319b95855 in doris::BetaRowsetReader::next_block(doris::vectorized::Block*) /home/zcp/repo_center/doris_branch-3.0/doris/be/src/olap/rowset/beta_rowset_reader.cpp:366:29
    #27 0x56434e3a0c10 in doris::vectorized::VCollectIterator::Level0Iterator::_refresh() /home/zcp/repo_center/doris_branch-3.0/doris/be/src/vec/olap/vcollect_iterator.h
    #28 0x56434e38ba5b in doris::vectorized::VCollectIterator::Level0Iterator::refresh_current_row() /home/zcp/repo_center/doris_branch-3.0/doris/be/src/vec/olap/vcollect_iterator.cpp:509:24
    #29 0x56434e38cd76 in doris::vectorized::VCollectIterator::Level0Iterator::ensure_first_row_ref() /home/zcp/repo_center/doris_branch-3.0/doris/be/src/vec/olap/vcollect_iterator.cpp:482:14
    #30 0x56434e399745 in doris::vectorized::VCollectIterator::Level1Iterator::ensure_first_row_ref() /home/zcp/repo_center/doris_branch-3.0/doris/be/src/vec/olap/vcollect_iterator.cpp:696:27
    #31 0x56434e37f7f9 in doris::vectorized::VCollectIterator::build_heap(std::vector, std::allocator>>&) /home/zcp/repo_center/doris_branch-3.0/doris/be/src/vec/olap/vcollect_iterator.cpp:186:9
    #32 0x56434e312fe2 in doris::vectorized::BlockReader::_init_collect_iter(doris::TabletReader::ReaderParams const&) /home/zcp/repo_center/doris_branch-3.0/doris/be/src/vec/olap/block_reader.cpp:140:5
    #33 0x56434e3165df in doris::vectorized::BlockReader::init(doris::TabletReader::ReaderParams const&) /home/zcp/repo_center/doris_branch-3.0/doris/be/src/vec/olap/block_reader.cpp:212:19
    #34 0x5643515aba04 in doris::vectorized::NewOlapScanner::open(doris::RuntimeState*) /home/zcp/repo_center/doris_branch-3.0/doris/be/src/vec/exec/scan/new_olap_scanner.cpp:231:32
    #35 0x564336c138c6 in doris::vectorized::ScannerScheduler::_scanner_scan(std::shared_ptr, std::shared_ptr) /home/zcp/repo_center/doris_branch-3.0/doris/be/src/vec/exec/scan/scanner_scheduler.cpp:247:5
    #36 0x564336c18db0 in doris::vectorized::ScannerScheduler::submit(std::shared_ptr, std::shared_ptr)::$_1::operator()() const::'lambda'()::operator()() const::'lambda0'()::operator()() const /home/zcp/repo_center/doris_branch-3.0/doris/be/src/vec/exec/scan/scanner_scheduler.cpp:180:21
    #37 0x564336c18db0 in doris::vectorized::ScannerScheduler::submit(std::shared_ptr, std::shared_ptr)::$_1::operator()() const::'lambda'()::operator()() const /home/zcp/repo_center/doris_branch-3.0/doris/be/src/vec/exec/scan/scanner_scheduler.cpp:179:31
    #38 0x564336c18db0 in void std::__invoke_impl, std::shared_ptr)::$_1::operator()() const::'lambda'()&>(std::__invoke_other, doris::vectorized::ScannerScheduler::submit(std::shared_ptr, std::shared_ptr)::$_1::operator()() const::'lambda'()&) /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/invoke.h:61:14
    #39 0x564336c18db0 in std::enable_if, std::shared_ptr)::$_1::operator()() const::'lambda'()&>, void>::type std::__invoke_r, std::shared_ptr)::$_1::operator()() const::'lambda'()&>(doris::vectorized::ScannerScheduler::submit(std::shared_ptr, std::shared_ptr)::$_1::operator()() const::'lambda'()&) /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/invoke.h:111:2
    #40 0x564336c18db0 in std::_Function_handler, std::shared_ptr)::$_1::operator()() const::'lambda'()>::_M_invoke(std::_Any_data const&) /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/std_function.h:291:9
    #41 0x56431b9d94eb in doris::ThreadPool::dispatch_thread() /home/zcp/repo_center/doris_branch-3.0/doris/be/src/util/threadpool.cpp:543:24
    #42 0x56431b9b18c7 in doris::Thread::supervise_thread(void*) /home/zcp/repo_center/doris_branch-3.0/doris/be/src/util/thread.cpp:498:5
    #43 0x7f31d3b87ac2 in start_thread nptl/pthread_create.c:442:8
    #44 0x7f31d3c1984f  misc/../sysdeps/unix/sysv/linux/x86_64/clone3.S:81

0x61d000a7f3c8 is located 1352 bytes inside of 2064-byte region [0x61d000a7ee80,0x61d000a7f690)
freed by thread T1698 (Pipe_normal [wo) here:
    #0 0x564317278d9d in operator delete(void*) (/mnt/hdd01/ci/doris-deploy-branch-3.0-cloud/be/lib/doris_be+0x3626bd9d) (BuildId: 9dab40b94b1dc995)
    #1 0x5643515b192c in std::unique_ptr>::reset(doris::TabletReader*) /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/unique_ptr.h:456:7
    #2 0x5643515b192c in doris::vectorized::NewOlapScanner::close(doris::RuntimeState*) /home/zcp/repo_center/doris_branch-3.0/doris/be/src/vec/exec/scan/new_olap_scanner.cpp:534:20
    #3 0x564352457463 in doris::vectorized::ScannerDelegate::~ScannerDelegate() /home/zcp/repo_center/doris_branch-3.0/doris/be/src/vec/exec/scan/vscan_node.h:35:31
    #4 0x5643172a8324 in std::_Sp_counted_base<(__gnu_cxx::_Lock_policy)2>::_M_release() /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/shared_ptr_base.h:168:6
    #5 0x56434e7d7870 in std::__shared_ptr::~__shared_ptr() /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/shared_ptr_base.h:1149:31
    #6 0x56434e7d7870 in void std::destroy_at>(std::shared_ptr*) /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/stl_construct.h:88:15
    #7 0x56434e7d7870 in void std::allocator_traits>>>::destroy>(std::allocator>>&, std::shared_ptr*) /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/alloc_traits.h:533:4
    #8 0x56434e7d7870 in std::__cxx11::_List_base, std::allocator>>::_M_clear() /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/list.tcc:77:4
    #9 0x564351db11d2 in std::__cxx11::_List_base, std::allocator>>::~_List_base() /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/stl_list.h:499:9
    #10 0x564351db11d2 in doris::pipeline::ScanLocalState::close(doris::RuntimeState*) /home/zcp/repo_center/doris_branch-3.0/doris/be/src/pipeline/exec/scan_operator.cpp:1261:5
    #11 0x56434e6357e3 in doris::pipeline::OperatorXBase::close(doris::RuntimeState*) /home/zcp/repo_center/doris_branch-3.0/doris/be/src/pipeline/exec/operator.cpp:245:28
    #12 0x564352ad00c1 in doris::pipeline::PipelineTask::close(doris::Status) /home/zcp/repo_center/doris_branch-3.0/doris/be/src/pipeline/pipeline_task.cpp:489:28
    #13 0x564352b0701f in doris::pipeline::_close_task(doris::pipeline::PipelineTask*, doris::Status) /home/zcp/repo_center/doris_branch-3.0/doris/be/src/pipeline/task_scheduler.cpp:91:27
    #14 0x564352b095f3 in doris::pipeline::TaskScheduler::_do_work(unsigned long) /home/zcp/repo_center/doris_branch-3.0/doris/be/src/pipeline/task_scheduler.cpp:181:17
    #15 0x56431b9d94eb in doris::ThreadPool::dispatch_thread() /home/zcp/repo_center/doris_branch-3.0/doris/be/src/util/threadpool.cpp:543:24
    #16 0x56431b9b18c7 in doris::Thread::supervise_thread(void*) /home/zcp/repo_center/doris_branch-3.0/doris/be/src/util/thread.cpp:498:5
    #17 0x7f31d3b87ac2 in start_thread nptl/pthread_create.c:442:8

```

Problem Summary:

### Release note

None

### Check List (For Author)

- Test <!-- At least one of them must be included. -->
    - [ ] Regression test
    - [ ] Unit Test
    - [ ] Manual test (add detailed scripts or steps below)
    - [ ] No need to test or manual test. Explain why:
- [ ] This is a refactor/code format and no logic has been changed.
        - [ ] Previous test can cover this change.
        - [ ] No code files have been changed.
        - [ ] Other reason <!-- Add your reason?  -->

- Behavior changed:
    - [ ] No.
    - [ ] Yes. <!-- Explain the behavior change -->

- Does this need documentation?
    - [ ] No.
- [ ] Yes. <!-- Add document PR link here. eg:
apache/doris-website#1214 -->

### Check List (For Reviewer who merge this PR)

- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label <!-- Add branch pick label that this PR
should merge into -->
@yiguolei yiguolei mentioned this issue Jan 15, 2025
Merged
16 tasks
yiguolei added a commit that referenced this issue Jan 15, 2025
@yiguolei
[bugfix](memtable) arena is freed early and will cause use after free (…
81534dd 
…#46997)

### What problem does this PR solve?

Related PR: #40912

Problem Summary:

Do not reset _arena in MemTable::to_block(), because it is still used in
~MemTable() when releasing agg places

Fix the following use-after-free

Use:

==3628099==ERROR: AddressSanitizer: heap-use-after-free on address
0x52100381be60 at pc 0x5648f30893f8 bp 0x7f8842433310 sp 0x7f8842433308
READ of size 8 at 0x52100381be60 thread T4767 (wg_flush_broker)
#0 0x5648f30893f7 in
phmap::priv::raw_hash_set<phmap::priv::FlatHashSetPolicy<unsigned long>,
phmap::Hash<unsigned long>, phmap::EqualTo<unsigned long>,
std::allocator<unsigned long>>::destroy_slots()
doris/thirdparty/installed/include/parallel_hashmap/phmap.h:1992:14
#1 0x5648f30936f6 in
phmap::priv::raw_hash_set<phmap::priv::FlatHashSetPolicy<unsigned long>,
phmap::Hash<unsigned long>, phmap::EqualTo<unsigned long>,
std::allocator<unsigned long>>::~raw_hash_set()
doris/thirdparty/installed/include/parallel_hashmap/phmap.h:1236:23
#2 0x5648f3089276 in phmap::flat_hash_set<unsigned long,
phmap::Hash<unsigned long>, phmap::EqualTo<unsigned long>,
std::allocator<unsigned long>>::~flat_hash_set()
doris/thirdparty/installed/include/parallel_hashmap/phmap.h:4577:7
#3 0x5648f308922a in doris::BitmapValue::~BitmapValue()
doris/be/src/util/bitmap_value.h:824:7
#4 0x56490d319fa6 in
doris::vectorized::AggregateFunctionBitmapData<doris::vectorized::AggregateFunctionBitmapUnionOp>::~AggregateFunctionBitmapData()
doris/be/src/vec/aggregate_functions/aggregate_function_bitmap.h:127:8
#5 0x56490d49636a in
doris::vectorized::IAggregateFunctionDataHelper<doris::vectorized::AggregateFunctionBitmapData<doris::vectorized::AggregateFunctionBitmapUnionOp>,
doris::vectorized::AggregateFunctionBitmapOp<doris::vectorized::AggregateFunctionBitmapUnionOp>>::destroy(char*)
const doris/be/src/vec/aggregate_functions/aggregate_function.h:563:92
#6 0x5648f68376e9 in doris::MemTable::~MemTable()
doris/be/src/olap/memtable.cpp:159:27
Free:

0x52100381be60 is located 352 bytes inside of 4096-byte region
[0x52100381bd00,0x52100381cd00)
freed by thread T4767 (wg_flush_broker) here:
#0 0x5648f2f3ee46 in free (doris/output/be/lib/doris_be+0x57418e46)
(BuildId: 298b9c91a1ec8fe0)
#1 0x5648f3080dfc in DefaultMemoryAllocator::free(void*)
doris/be/src/vec/common/allocator.h:108:41
#2 0x5648f3080b3f in Allocator<false, false, false,
DefaultMemoryAllocator>::free(void*, unsigned long)
doris/be/src/vec/common/allocator.h:323:13
#3 0x5648f30b6dee in doris::vectorized::Arena::Chunk::~Chunk()
doris/be/src/vec/common/arena.h:77:31
#4 0x5648f30b6d1f in doris::vectorized::Arena::~Arena()
doris/be/src/vec/common/arena.h:151:16
#5 0x5648f30b695a in
std::default_delete<doris::vectorized::Arena>::operator()(doris::vectorized::Arena*)
const
env/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/bits/unique_ptr.h:99:2
#6 0x5648f30b67c8 in std::__uniq_ptr_impl<doris::vectorized::Arena,
std::default_delete<doris::vectorized::Arena>>::reset(doris::vectorized::Arena*)
env/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/bits/unique_ptr.h:211:4
#7 0x5648f30b5d8c in std::unique_ptr<doris::vectorized::Arena,
std::default_delete<doris::vectorized::Arena>>::reset(doris::vectorized::Arena*)
env/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/bits/unique_ptr.h:509:7
#8 0x5648f684253b in
doris::MemTable::_to_block(std::unique_ptr<doris::vectorized::Block,
std::default_delete<doris::vectorized::Block>>*)
doris/be/src/olap/memtable.cpp:522:12
#9 0x5648f6842ac5 in
doris::MemTable::to_block(std::unique_ptr<doris::vectorized::Block,
std::default_delete<doris::vectorized::Block>>*)
doris/be/src/olap/memtable.cpp:528:5
#10 0x5648f6907a72 in
doris::FlushToken::_do_flush_memtable(doris::MemTable*, int, long*)
doris/be/src/olap/memtable_flush_executor.cpp:144:9
#11 0x5648f690932c in
doris::FlushToken::_flush_memtable(std::shared_ptr<doris::MemTable>,
int, long) doris/be/src/olap/memtable_flush_executor.cpp:183:16
#12 0x5648f6915d18 in doris::MemtableFlushTask::run()
doris/be/src/olap/memtable_flush_executor.cpp:60:20
github-actions bot pushed a commit that referenced this issue Jan 15, 2025
@yiguolei
[bugfix](memtable) arena is freed early and will cause use after free (…
8aa5d66 
…#46997)

### What problem does this PR solve?

Related PR: #40912

Problem Summary:

Do not reset _arena in MemTable::to_block(), because it is still used in
~MemTable() when releasing agg places

Fix the following use-after-free

Use:

==3628099==ERROR: AddressSanitizer: heap-use-after-free on address
0x52100381be60 at pc 0x5648f30893f8 bp 0x7f8842433310 sp 0x7f8842433308
READ of size 8 at 0x52100381be60 thread T4767 (wg_flush_broker)
#0 0x5648f30893f7 in
phmap::priv::raw_hash_set<phmap::priv::FlatHashSetPolicy<unsigned long>,
phmap::Hash<unsigned long>, phmap::EqualTo<unsigned long>,
std::allocator<unsigned long>>::destroy_slots()
doris/thirdparty/installed/include/parallel_hashmap/phmap.h:1992:14
#1 0x5648f30936f6 in
phmap::priv::raw_hash_set<phmap::priv::FlatHashSetPolicy<unsigned long>,
phmap::Hash<unsigned long>, phmap::EqualTo<unsigned long>,
std::allocator<unsigned long>>::~raw_hash_set()
doris/thirdparty/installed/include/parallel_hashmap/phmap.h:1236:23
#2 0x5648f3089276 in phmap::flat_hash_set<unsigned long,
phmap::Hash<unsigned long>, phmap::EqualTo<unsigned long>,
std::allocator<unsigned long>>::~flat_hash_set()
doris/thirdparty/installed/include/parallel_hashmap/phmap.h:4577:7
#3 0x5648f308922a in doris::BitmapValue::~BitmapValue()
doris/be/src/util/bitmap_value.h:824:7
#4 0x56490d319fa6 in
doris::vectorized::AggregateFunctionBitmapData<doris::vectorized::AggregateFunctionBitmapUnionOp>::~AggregateFunctionBitmapData()
doris/be/src/vec/aggregate_functions/aggregate_function_bitmap.h:127:8
#5 0x56490d49636a in
doris::vectorized::IAggregateFunctionDataHelper<doris::vectorized::AggregateFunctionBitmapData<doris::vectorized::AggregateFunctionBitmapUnionOp>,
doris::vectorized::AggregateFunctionBitmapOp<doris::vectorized::AggregateFunctionBitmapUnionOp>>::destroy(char*)
const doris/be/src/vec/aggregate_functions/aggregate_function.h:563:92
#6 0x5648f68376e9 in doris::MemTable::~MemTable()
doris/be/src/olap/memtable.cpp:159:27
Free:

0x52100381be60 is located 352 bytes inside of 4096-byte region
[0x52100381bd00,0x52100381cd00)
freed by thread T4767 (wg_flush_broker) here:
#0 0x5648f2f3ee46 in free (doris/output/be/lib/doris_be+0x57418e46)
(BuildId: 298b9c91a1ec8fe0)
#1 0x5648f3080dfc in DefaultMemoryAllocator::free(void*)
doris/be/src/vec/common/allocator.h:108:41
#2 0x5648f3080b3f in Allocator<false, false, false,
DefaultMemoryAllocator>::free(void*, unsigned long)
doris/be/src/vec/common/allocator.h:323:13
#3 0x5648f30b6dee in doris::vectorized::Arena::Chunk::~Chunk()
doris/be/src/vec/common/arena.h:77:31
#4 0x5648f30b6d1f in doris::vectorized::Arena::~Arena()
doris/be/src/vec/common/arena.h:151:16
#5 0x5648f30b695a in
std::default_delete<doris::vectorized::Arena>::operator()(doris::vectorized::Arena*)
const
env/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/bits/unique_ptr.h:99:2
#6 0x5648f30b67c8 in std::__uniq_ptr_impl<doris::vectorized::Arena,
std::default_delete<doris::vectorized::Arena>>::reset(doris::vectorized::Arena*)
env/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/bits/unique_ptr.h:211:4
#7 0x5648f30b5d8c in std::unique_ptr<doris::vectorized::Arena,
std::default_delete<doris::vectorized::Arena>>::reset(doris::vectorized::Arena*)
env/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/bits/unique_ptr.h:509:7
#8 0x5648f684253b in
doris::MemTable::_to_block(std::unique_ptr<doris::vectorized::Block,
std::default_delete<doris::vectorized::Block>>*)
doris/be/src/olap/memtable.cpp:522:12
#9 0x5648f6842ac5 in
doris::MemTable::to_block(std::unique_ptr<doris::vectorized::Block,
std::default_delete<doris::vectorized::Block>>*)
doris/be/src/olap/memtable.cpp:528:5
#10 0x5648f6907a72 in
doris::FlushToken::_do_flush_memtable(doris::MemTable*, int, long*)
doris/be/src/olap/memtable_flush_executor.cpp:144:9
#11 0x5648f690932c in
doris::FlushToken::_flush_memtable(std::shared_ptr<doris::MemTable>,
int, long) doris/be/src/olap/memtable_flush_executor.cpp:183:16
#12 0x5648f6915d18 in doris::MemtableFlushTask::run()
doris/be/src/olap/memtable_flush_executor.cpp:60:20
HappenLee added a commit that referenced this issue Feb 14, 2025
@HappenLee
[Bug](cache) Fix query cache report error message (#47883)
c58fc18 
the plan：
```
 QUERY_CACHE:                                                                                                                                                                                                                                                                                                                           |
|     CACHE_NODE_ID: 2                                                                                                                                                                                                                                                                                                                     |
|     DIGEST: 557413bec3209f50fd640eb3b0534c12ccd9df35f9d4c86620416936985e2679                                                                                                                                                                                                                                                             |
|                                                                                                                                                                                                                                                                                                                                          |
|   STREAM DATA SINK                                                                                                                                                                                                                                                                                                                       |
|     EXCHANGE ID: 03                                                                                                                                                                                                                                                                                                                      |
|     UNPARTITIONED                                                                                                                                                                                                                                                                                                                        |
|                                                                                                                                                                                                                                                                                                                                          |
|   2:VAGGREGATE (merge finalize)(273)                                                                                                                                                                                                                                                                                                     |
|   |  output: min(partial_min(pk)[#6])[#9], max(partial_max(pk)[#7])[#10]                                                                                                                                                                                                                                                                 |
|   |  group by: pk[#5]                                                                                                                                                                                                                                                                                                                    |
|   |  sortByGroupKey:false                                                                                                                                                                                                                                                                                                                |
|   |  cardinality=0                                                                                                                                                                                                                                                                                                                       |
|   |  final projections: field1[#9], field2[#10], pk[#8]                                                                                                                                                                                                                                                                                  |
|   |  final project output tuple id: 4                                                                                                                                                                                                                                                                                                    |
|   |  distribute expr lists: pk[#5]                                                                                                                                                                                                                                                                                                       |
|   |                                                                                                                                                                                                                                                                                                                                      |
|   1:VAGGREGATE (update serialize)(268)                                                                                                                                                                                                                                                                                                   |
|   |  output: partial_min(pk[#4])[#6], partial_max(pk[#4])[#7]                                                                                                                                                                                                                                                                            |
|   |  group by: pk[#4]                                                                                                                                                                                                                                                                                                                    |
|   |  sortByGroupKey:false                                                                                                                                                                                                                                                                                                                |
|   |  cardinality=0                                                                                                                                                                                                                                                                                                                       |
|   |  distribute expr lists: pk[#4]                                                                                                                                                                                                                                                                                                       |
|   |                                                                                                                                                                                                                                                                                           
```

FE choose the id agg 2, but BE still use id agg 1. BE should support the case
xinyiZzz added a commit that referenced this issue Feb 19, 2025
@xinyiZzz
[fix](resource) Fix register ResourceContext to RuntimeQueryStatistic…
e7c966c 
…sMgr (#48004)

### What problem does this PR solve?

Wait for QueryTaskController to be initialized before register
ResourceContext

fix
```
/root/doris/be/src/runtime/workload_management/task_controller.h:59:44: runtime error: load of value 3200171710, which is not a valid value for type 'TQueryType::type'
    #0 0x555ff9336162 in doris::TaskController::query_type() /root/doris/be/src/runtime/workload_management/task_controller.h:59:44
    #1 0x555ff9336162 in doris::RuntimeQueryStatisticsMgr::report_runtime_query_statistics() /root/doris/be/src/runtime/runtime_query_statistics_mgr.cpp:346:50
    #2 0x555ff50d789d in doris::Daemon::report_runtime_query_statistics_thread() /root/doris/be/src/common/daemon.cpp:487:65
    #3 0x555ff9a3efae in std::function<void ()>::operator()() const /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/std_function.h:560:9
    #4 0x555ff9a3efae in doris::Thread::supervise_thread(void*) /root/doris/be/src/util/thread.cpp:498:5
    #5 0x7f4935d00608 in start_thread /build/glibc-SzIz7B/glibc-2.31/nptl/pthread_create.c:477:8
    #6 0x7f4935fad132 in __clone /build/glibc-SzIz7B/glibc-2.31/misc/../sysdeps/unix/sysv/linux/x86_64/clone.S:95
```
@Gabriel39 Gabriel39 mentioned this issue Feb 21, 2025
Merged
16 tasks
Gabriel39 added a commit that referenced this issue Feb 21, 2025
@Gabriel39
[fix](scheduler) Fix invalid access after freed (#48168)
b7c973e 
WRITE of size 1 at 0x6160007e86f0 thread T1983 (Pipe_normal [wo)
#0 0x55fc8065b975 in std::__atomic_base<bool>::store(bool,
std::memory_order)
/var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/atomic_base.h:457:2
#1 0x55fc8065b975 in std::__atomic_base<bool>::operator=(bool)
/var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/atomic_base.h:349:2
#2 0x55fc8065b975 in std::atomic<bool>::operator=(bool)
/var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/atomic:80:22
#3 0x55fc8065b975 in doris::pipeline::PipelineTask::set_running(bool)
/root/doris/be/src/pipeline/pipeline_task.h:192:47
#4 0x55fc8065b975 in
doris::pipeline::TaskScheduler::_do_work(int)::$_0::operator()() const
/root/doris/be/src/pipeline/task_scheduler.cpp:121:23
#5 0x55fc8065b975 in
doris::Defer<doris::pipeline::TaskScheduler::_do_work(int)::$_0>::~Defer()
/root/doris/be/src/util/defer_op.h:37:16
#6 0x55fc8065b975 in doris::pipeline::TaskScheduler::_do_work(int)
/root/doris/be/src/pipeline/task_scheduler.cpp:162:5
#7 0x55fc4c57cd19 in doris::ThreadPool::dispatch_thread()
/root/doris/be/src/util/threadpool.cpp:608:24
#8 0x55fc4c55395e in std::function<void ()>::operator()() const
/var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/std_function.h:560:9
#9 0x55fc4c55395e in doris::Thread::supervise_thread(void*)
/root/doris/be/src/util/thread.cpp:498:5
#10 0x7f9ee3d25608 in start_thread
/build/glibc-SzIz7B/glibc-2.31/nptl/pthread_create.c:477:8
#11 0x7f9ee3fd2132 in __clone
/build/glibc-SzIz7B/glibc-2.31/misc/../sysdeps/unix/sysv/linux/x86_64/clone.S:95

0x6160007e86f0 is located 624 bytes inside of 632-byte region
[0x6160007e8480,0x6160007e86f8)
freed by thread T1981 (Pipe_normal [wo) here:
#0 0x55fc47aa680d in operator delete(void*)
(/mnt/ssd01/pipline/OpenSourceDoris/clusterEnv/P0/Cluster0/be/lib/doris_be+0x3376e80d)
(BuildId: 865149e62959581e)
#1 0x55fc8059db84 in
std::default_delete<doris::pipeline::PipelineTask>::operator()(doris::pipeline::PipelineTask*)
const
/var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/unique_ptr.h:85:2
#2 0x55fc8059db84 in std::unique_ptr<doris::pipeline::PipelineTask,
std::default_delete<doris::pipeline::PipelineTask> >::~unique_ptr()
/var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/unique_ptr.h:361:4
#3 0x55fc8059db84 in void
std::destroy_at<std::unique_ptr<doris::pipeline::PipelineTask,
std::default_delete<doris::pipeline::PipelineTask> >
>(std::unique_ptr<doris::pipeline::PipelineTask,
std::default_delete<doris::pipeline::PipelineTask> >*)
/var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/stl_construct.h:88:15
#4 0x55fc8059db84 in void
std::_Destroy<std::unique_ptr<doris::pipeline::PipelineTask,
std::default_delete<doris::pipeline::PipelineTask> >
>(std::unique_ptr<doris::pipeline::PipelineTask,
std::default_delete<doris::pipeline::PipelineTask> >*)
/var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/stl_construct.h:138:7
#5 0x55fc8059db84 in void
std::_Destroy_aux<false>::__destroy<std::unique_ptr<doris::pipeline::PipelineTask,
std::default_delete<doris::pipeline::PipelineTask>
>*>(std::unique_ptr<doris::pipeline::PipelineTask,
std::default_delete<doris::pipeline::PipelineTask> >*,
std::unique_ptr<doris::pipeline::PipelineTask,
std::default_delete<doris::pipeline::PipelineTask> >*)
/var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/stl_construct.h:152:6
#6 0x55fc8059db84 in void
std::_Destroy<std::unique_ptr<doris::pipeline::PipelineTask,
std::default_delete<doris::pipeline::PipelineTask>
>*>(std::unique_ptr<doris::pipeline::PipelineTask,
std::default_delete<doris::pipeline::PipelineTask> >*,
std::unique_ptr<doris::pipeline::PipelineTask,
std::default_delete<doris::pipeline::PipelineTask> >*)
/var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/stl_construct.h:184:7
#7 0x55fc8059db84 in void
std::_Destroy<std::unique_ptr<doris::pipeline::PipelineTask,
std::default_delete<doris::pipeline::PipelineTask> >*,
std::unique_ptr<doris::pipeline::PipelineTask,
std::default_delete<doris::pipeline::PipelineTask> >
>(std::unique_ptr<doris::pipeline::PipelineTask,
std::default_delete<doris::pipeline::PipelineTask> >*,
std::unique_ptr<doris::pipeline::PipelineTask,
std::default_delete<doris::pipeline::PipelineTask> >*,
std::allocator<std::unique_ptr<doris::pipeline::PipelineTask,
std::default_delete<doris::pipeline::PipelineTask> > >&)
/var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/alloc_traits.h:746:7
#8 0x55fc8059db84 in
std::vector<std::unique_ptr<doris::pipeline::PipelineTask,
std::default_delete<doris::pipeline::PipelineTask> >,
std::allocator<std::unique_ptr<doris::pipeline::PipelineTask,
std::default_delete<doris::pipeline::PipelineTask> > > >::~vector()
/var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/stl_vector.h:680:2
#9 0x55fc8052571c in void
std::destroy_at<std::vector<std::unique_ptr<doris::pipeline::PipelineTask,
std::default_delete<doris::pipeline::PipelineTask> >,
std::allocator<std::unique_ptr<doris::pipeline::PipelineTask,
std::default_delete<doris::pipeline::PipelineTask> > > >
>(std::vector<std::unique_ptr<doris::pipeline::PipelineTask,
std::default_delete<doris::pipeline::PipelineTask> >,
std::allocator<std::unique_ptr<doris::pipeline::PipelineTask,
std::default_delete<doris::pipeline::PipelineTask> > > >*)
/var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/stl_construct.h:88:15
#10 0x55fc8052571c in void
std::_Destroy<std::vector<std::unique_ptr<doris::pipeline::PipelineTask,
std::default_delete<doris::pipeline::PipelineTask> >,
std::allocator<std::unique_ptr<doris::pipeline::PipelineTask,
std::default_delete<doris::pipeline::PipelineTask> > > >
>(std::vector<std::unique_ptr<doris::pipeline::PipelineTask,
std::default_delete<doris::pipeline::PipelineTask> >,
std::allocator<std::unique_ptr<doris::pipeline::PipelineTask,
std::default_delete<doris::pipeline::PipelineTask> > > >*)
/var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/stl_construct.h:138:7
#11 0x55fc8052571c in void
std::_Destroy_aux<false>::__destroy<std::vector<std::unique_ptr<doris::pipeline::PipelineTask,
std::default_delete<doris::pipeline::PipelineTask> >,
std::allocator<std::unique_ptr<doris::pipeline::PipelineTask,
std::default_delete<doris::pipeline::PipelineTask> > >
>*>(std::vector<std::unique_ptr<doris::pipeline::PipelineTask,
std::default_delete<doris::pipeline::PipelineTask> >,
std::allocator<std::unique_ptr<doris::pipeline::PipelineTask,
std::default_delete<doris::pipeline::PipelineTask> > > >*,
std::vector<std::unique_ptr<doris::pipeline::PipelineTask,
std::default_delete<doris::pipeline::PipelineTask> >,
std::allocator<std::unique_ptr<doris::pipeline::PipelineTask,
std::default_delete<doris::pipeline::PipelineTask> > > >*)
/var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/stl_construct.h:152:6
#12 0x55fc8052571c in void
std::_Destroy<std::vector<std::unique_ptr<doris::pipeline::PipelineTask,
std::default_delete<doris::pipeline::PipelineTask> >,
std::allocator<std::unique_ptr<doris::pipeline::PipelineTask,
std::default_delete<doris::pipeline::PipelineTask> > >
>*>(std::vector<std::unique_ptr<doris::pipeline::PipelineTask,
std::default_delete<doris::pipeline::PipelineTask> >,
std::allocator<std::unique_ptr<doris::pipeline::PipelineTask,
std::default_delete<doris::pipeline::PipelineTask> > > >*,
std::vector<std::unique_ptr<doris::pipeline::PipelineTask,
std::default_delete<doris::pipeline::PipelineTask> >,
std::allocator<std::unique_ptr<doris::pipeline::PipelineTask,
std::default_delete<doris::pipeline::PipelineTask> > > >*)
/var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/stl_construct.h:184:7
#13 0x55fc8052571c in void
std::_Destroy<std::vector<std::unique_ptr<doris::pipeline::PipelineTask,
std::default_delete<doris::pipeline::PipelineTask> >,
std::allocator<std::unique_ptr<doris::pipeline::PipelineTask,
std::default_delete<doris::pipeline::PipelineTask> > > >*,
std::vector<std::unique_ptr<doris::pipeline::PipelineTask,
std::default_delete<doris::pipeline::PipelineTask> >,
std::allocator<std::unique_ptr<doris::pipeline::PipelineTask,
std::default_delete<doris::pipeline::PipelineTask> > > >
>(std::vector<std::unique_ptr<doris::pipeline::PipelineTask,
std::default_delete<doris::pipeline::PipelineTask> >,
std::allocator<std::unique_ptr<doris::pipeline::PipelineTask,
std::default_delete<doris::pipeline::PipelineTask> > > >*,
std::vector<std::unique_ptr<doris::pipeline::PipelineTask,
std::default_delete<doris::pipeline::PipelineTask> >,
std::allocator<std::unique_ptr<doris::pipeline::PipelineTask,
std::default_delete<doris::pipeline::PipelineTask> > > >*,
std::allocator<std::vector<std::unique_ptr<doris::pipeline::PipelineTask,
std::default_delete<doris::pipeline::PipelineTask> >,
std::allocator<std::unique_ptr<doris::pipeline::PipelineTask,
std::default_delete<doris::pipeline::PipelineTask> > > > >&)
/var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/alloc_traits.h:746:7
#14 0x55fc8052571c in
std::vector<std::vector<std::unique_ptr<doris::pipeline::PipelineTask,
std::default_delete<doris::pipeline::PipelineTask> >,
std::allocator<std::unique_ptr<doris::pipeline::PipelineTask,
std::default_delete<doris::pipeline::PipelineTask> > > >,
std::allocator<std::vector<std::unique_ptr<doris::pipeline::PipelineTask,
std::default_delete<doris::pipeline::PipelineTask> >,
std::allocator<std::unique_ptr<doris::pipeline::PipelineTask,
std::default_delete<doris::pipeline::PipelineTask> > > > >
>::_M_erase_at_end(std::vector<std::unique_ptr<doris::pipeline::PipelineTask,
std::default_delete<doris::pipeline::PipelineTask> >,
std::allocator<std::unique_ptr<doris::pipeline::PipelineTask,
std::default_delete<doris::pipeline::PipelineTask> > > >*)
/var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/stl_vector.h:1796:6
#15 0x55fc8052571c in
std::vector<std::vector<std::unique_ptr<doris::pipeline::PipelineTask,
std::default_delete<doris::pipeline::PipelineTask> >,
std::allocator<std::unique_ptr<doris::pipeline::PipelineTask,
std::default_delete<doris::pipeline::PipelineTask> > > >,
std::allocator<std::vector<std::unique_ptr<doris::pipeline::PipelineTask,
std::default_delete<doris::pipeline::PipelineTask> >,
std::allocator<std::unique_ptr<doris::pipeline::PipelineTask,
std::default_delete<doris::pipeline::PipelineTask> > > > > >::clear()
/var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/stl_vector.h:1499:9
#16 0x55fc8052571c in
doris::pipeline::PipelineFragmentContext::~PipelineFragmentContext()
/root/doris/be/src/pipeline/pipeline_fragment_context.cpp:142:12
#17 0x55fc47ad30cc in
std::_Sp_counted_base<(__gnu_cxx::_Lock_policy)2>::_M_release()
/var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/shared_ptr_base.h:168:6
#18 0x55fc80658d57 in
std::__shared_count<(__gnu_cxx::_Lock_policy)2>::~__shared_count()
/var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/shared_ptr_base.h:702:11
#19 0x55fc80658d57 in std::__shared_ptr<doris::TaskExecutionContext,
(__gnu_cxx::_Lock_policy)2>::~__shared_ptr()
/var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/shared_ptr_base.h:1149:31
#20 0x55fc80658d57 in
doris::pipeline::close_task(doris::pipeline::PipelineTask*,
doris::Status) /root/doris/be/src/pipeline/task_scheduler.cpp:100:1
#21 0x55fc8065aa17 in doris::pipeline::TaskScheduler::_do_work(int)
/root/doris/be/src/pipeline/task_scheduler.cpp:160:36
#22 0x55fc4c57cd19 in doris::ThreadPool::dispatch_thread()
/root/doris/be/src/util/threadpool.cpp:608:24
#23 0x55fc4c55395e in std::function<void ()>::operator()() const
/var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/std_function.h:560:9
#24 0x55fc4c55395e in doris::Thread::supervise_thread(void*)
/root/doris/be/src/util/thread.cpp:498:5
#25 0x7f9ee3d25608 in start_thread
/build/glibc-SzIz7B/glibc-2.31/nptl/pthread_create.c:477:8
lzyy2024 pushed a commit to lzyy2024/doris that referenced this issue Feb 21, 2025
@yiguolei @lzyy2024
[bugfix](memtable) arena is freed early and will cause use after free (…
193cecd 
…apache#46997)

### What problem does this PR solve?

Related PR: apache#40912

Problem Summary:

Do not reset _arena in MemTable::to_block(), because it is still used in
~MemTable() when releasing agg places

Fix the following use-after-free

Use:

==3628099==ERROR: AddressSanitizer: heap-use-after-free on address
0x52100381be60 at pc 0x5648f30893f8 bp 0x7f8842433310 sp 0x7f8842433308
READ of size 8 at 0x52100381be60 thread T4767 (wg_flush_broker)
#0 0x5648f30893f7 in
phmap::priv::raw_hash_set<phmap::priv::FlatHashSetPolicy<unsigned long>,
phmap::Hash<unsigned long>, phmap::EqualTo<unsigned long>,
std::allocator<unsigned long>>::destroy_slots()
doris/thirdparty/installed/include/parallel_hashmap/phmap.h:1992:14
apache#1 0x5648f30936f6 in
phmap::priv::raw_hash_set<phmap::priv::FlatHashSetPolicy<unsigned long>,
phmap::Hash<unsigned long>, phmap::EqualTo<unsigned long>,
std::allocator<unsigned long>>::~raw_hash_set()
doris/thirdparty/installed/include/parallel_hashmap/phmap.h:1236:23
apache#2 0x5648f3089276 in phmap::flat_hash_set<unsigned long,
phmap::Hash<unsigned long>, phmap::EqualTo<unsigned long>,
std::allocator<unsigned long>>::~flat_hash_set()
doris/thirdparty/installed/include/parallel_hashmap/phmap.h:4577:7
apache#3 0x5648f308922a in doris::BitmapValue::~BitmapValue()
doris/be/src/util/bitmap_value.h:824:7
apache#4 0x56490d319fa6 in
doris::vectorized::AggregateFunctionBitmapData<doris::vectorized::AggregateFunctionBitmapUnionOp>::~AggregateFunctionBitmapData()
doris/be/src/vec/aggregate_functions/aggregate_function_bitmap.h:127:8
apache#5 0x56490d49636a in
doris::vectorized::IAggregateFunctionDataHelper<doris::vectorized::AggregateFunctionBitmapData<doris::vectorized::AggregateFunctionBitmapUnionOp>,
doris::vectorized::AggregateFunctionBitmapOp<doris::vectorized::AggregateFunctionBitmapUnionOp>>::destroy(char*)
const doris/be/src/vec/aggregate_functions/aggregate_function.h:563:92
apache#6 0x5648f68376e9 in doris::MemTable::~MemTable()
doris/be/src/olap/memtable.cpp:159:27
Free:

0x52100381be60 is located 352 bytes inside of 4096-byte region
[0x52100381bd00,0x52100381cd00)
freed by thread T4767 (wg_flush_broker) here:
#0 0x5648f2f3ee46 in free (doris/output/be/lib/doris_be+0x57418e46)
(BuildId: 298b9c91a1ec8fe0)
apache#1 0x5648f3080dfc in DefaultMemoryAllocator::free(void*)
doris/be/src/vec/common/allocator.h:108:41
apache#2 0x5648f3080b3f in Allocator<false, false, false,
DefaultMemoryAllocator>::free(void*, unsigned long)
doris/be/src/vec/common/allocator.h:323:13
apache#3 0x5648f30b6dee in doris::vectorized::Arena::Chunk::~Chunk()
doris/be/src/vec/common/arena.h:77:31
apache#4 0x5648f30b6d1f in doris::vectorized::Arena::~Arena()
doris/be/src/vec/common/arena.h:151:16
apache#5 0x5648f30b695a in
std::default_delete<doris::vectorized::Arena>::operator()(doris::vectorized::Arena*)
const
env/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/bits/unique_ptr.h:99:2
apache#6 0x5648f30b67c8 in std::__uniq_ptr_impl<doris::vectorized::Arena,
std::default_delete<doris::vectorized::Arena>>::reset(doris::vectorized::Arena*)
env/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/bits/unique_ptr.h:211:4
apache#7 0x5648f30b5d8c in std::unique_ptr<doris::vectorized::Arena,
std::default_delete<doris::vectorized::Arena>>::reset(doris::vectorized::Arena*)
env/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/bits/unique_ptr.h:509:7
apache#8 0x5648f684253b in
doris::MemTable::_to_block(std::unique_ptr<doris::vectorized::Block,
std::default_delete<doris::vectorized::Block>>*)
doris/be/src/olap/memtable.cpp:522:12
apache#9 0x5648f6842ac5 in
doris::MemTable::to_block(std::unique_ptr<doris::vectorized::Block,
std::default_delete<doris::vectorized::Block>>*)
doris/be/src/olap/memtable.cpp:528:5
apache#10 0x5648f6907a72 in
doris::FlushToken::_do_flush_memtable(doris::MemTable*, int, long*)
doris/be/src/olap/memtable_flush_executor.cpp:144:9
apache#11 0x5648f690932c in
doris::FlushToken::_flush_memtable(std::shared_ptr<doris::MemTable>,
int, long) doris/be/src/olap/memtable_flush_executor.cpp:183:16
apache#12 0x5648f6915d18 in doris::MemtableFlushTask::run()
doris/be/src/olap/memtable_flush_executor.cpp:60:20
lzyy2024 pushed a commit to lzyy2024/doris that referenced this issue Feb 21, 2025
@HappenLee @lzyy2024
[Bug](cache) Fix query cache report error message (apache#47883)
63dabf7 
the plan：
```
 QUERY_CACHE:                                                                                                                                                                                                                                                                                                                           |
|     CACHE_NODE_ID: 2                                                                                                                                                                                                                                                                                                                     |
|     DIGEST: 557413bec3209f50fd640eb3b0534c12ccd9df35f9d4c86620416936985e2679                                                                                                                                                                                                                                                             |
|                                                                                                                                                                                                                                                                                                                                          |
|   STREAM DATA SINK                                                                                                                                                                                                                                                                                                                       |
|     EXCHANGE ID: 03                                                                                                                                                                                                                                                                                                                      |
|     UNPARTITIONED                                                                                                                                                                                                                                                                                                                        |
|                                                                                                                                                                                                                                                                                                                                          |
|   2:VAGGREGATE (merge finalize)(273)                                                                                                                                                                                                                                                                                                     |
|   |  output: min(partial_min(pk)[apache#6])[apache#9], max(partial_max(pk)[apache#7])[apache#10]                                                                                                                                                                                                                                                                 |
|   |  group by: pk[apache#5]                                                                                                                                                                                                                                                                                                                    |
|   |  sortByGroupKey:false                                                                                                                                                                                                                                                                                                                |
|   |  cardinality=0                                                                                                                                                                                                                                                                                                                       |
|   |  final projections: field1[apache#9], field2[apache#10], pk[apache#8]                                                                                                                                                                                                                                                                                  |
|   |  final project output tuple id: 4                                                                                                                                                                                                                                                                                                    |
|   |  distribute expr lists: pk[apache#5]                                                                                                                                                                                                                                                                                                       |
|   |                                                                                                                                                                                                                                                                                                                                      |
|   1:VAGGREGATE (update serialize)(268)                                                                                                                                                                                                                                                                                                   |
|   |  output: partial_min(pk[apache#4])[apache#6], partial_max(pk[apache#4])[apache#7]                                                                                                                                                                                                                                                                            |
|   |  group by: pk[apache#4]                                                                                                                                                                                                                                                                                                                    |
|   |  sortByGroupKey:false                                                                                                                                                                                                                                                                                                                |
|   |  cardinality=0                                                                                                                                                                                                                                                                                                                       |
|   |  distribute expr lists: pk[apache#4]                                                                                                                                                                                                                                                                                                       |
|   |                                                                                                                                                                                                                                                                                           
```

FE choose the id agg 2, but BE still use id agg 1. BE should support the case
lzyy2024 pushed a commit to lzyy2024/doris that referenced this issue Feb 21, 2025
@xinyiZzz @lzyy2024
[fix](resource) Fix register ResourceContext to RuntimeQueryStatistic…
2265362 
…sMgr (apache#48004)

### What problem does this PR solve?

Wait for QueryTaskController to be initialized before register
ResourceContext

fix
```
/root/doris/be/src/runtime/workload_management/task_controller.h:59:44: runtime error: load of value 3200171710, which is not a valid value for type 'TQueryType::type'
    #0 0x555ff9336162 in doris::TaskController::query_type() /root/doris/be/src/runtime/workload_management/task_controller.h:59:44
    apache#1 0x555ff9336162 in doris::RuntimeQueryStatisticsMgr::report_runtime_query_statistics() /root/doris/be/src/runtime/runtime_query_statistics_mgr.cpp:346:50
    apache#2 0x555ff50d789d in doris::Daemon::report_runtime_query_statistics_thread() /root/doris/be/src/common/daemon.cpp:487:65
    apache#3 0x555ff9a3efae in std::function<void ()>::operator()() const /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/std_function.h:560:9
    apache#4 0x555ff9a3efae in doris::Thread::supervise_thread(void*) /root/doris/be/src/util/thread.cpp:498:5
    apache#5 0x7f4935d00608 in start_thread /build/glibc-SzIz7B/glibc-2.31/nptl/pthread_create.c:477:8
    apache#6 0x7f4935fad132 in __clone /build/glibc-SzIz7B/glibc-2.31/misc/../sysdeps/unix/sysv/linux/x86_64/clone.S:95
```
lzyy2024 pushed a commit to lzyy2024/doris that referenced this issue Feb 21, 2025
@Gabriel39 @lzyy2024
[fix](scheduler) Fix invalid access after freed (apache#48168)
448aa8d 
WRITE of size 1 at 0x6160007e86f0 thread T1983 (Pipe_normal [wo)
#0 0x55fc8065b975 in std::__atomic_base<bool>::store(bool,
std::memory_order)
/var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/atomic_base.h:457:2
apache#1 0x55fc8065b975 in std::__atomic_base<bool>::operator=(bool)
/var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/atomic_base.h:349:2
apache#2 0x55fc8065b975 in std::atomic<bool>::operator=(bool)
/var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/atomic:80:22
apache#3 0x55fc8065b975 in doris::pipeline::PipelineTask::set_running(bool)
/root/doris/be/src/pipeline/pipeline_task.h:192:47
apache#4 0x55fc8065b975 in
doris::pipeline::TaskScheduler::_do_work(int)::$_0::operator()() const
/root/doris/be/src/pipeline/task_scheduler.cpp:121:23
apache#5 0x55fc8065b975 in
doris::Defer<doris::pipeline::TaskScheduler::_do_work(int)::$_0>::~Defer()
/root/doris/be/src/util/defer_op.h:37:16
apache#6 0x55fc8065b975 in doris::pipeline::TaskScheduler::_do_work(int)
/root/doris/be/src/pipeline/task_scheduler.cpp:162:5
apache#7 0x55fc4c57cd19 in doris::ThreadPool::dispatch_thread()
/root/doris/be/src/util/threadpool.cpp:608:24
apache#8 0x55fc4c55395e in std::function<void ()>::operator()() const
/var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/std_function.h:560:9
apache#9 0x55fc4c55395e in doris::Thread::supervise_thread(void*)
/root/doris/be/src/util/thread.cpp:498:5
apache#10 0x7f9ee3d25608 in start_thread
/build/glibc-SzIz7B/glibc-2.31/nptl/pthread_create.c:477:8
apache#11 0x7f9ee3fd2132 in __clone
/build/glibc-SzIz7B/glibc-2.31/misc/../sysdeps/unix/sysv/linux/x86_64/clone.S:95

0x6160007e86f0 is located 624 bytes inside of 632-byte region
[0x6160007e8480,0x6160007e86f8)
freed by thread T1981 (Pipe_normal [wo) here:
#0 0x55fc47aa680d in operator delete(void*)
(/mnt/ssd01/pipline/OpenSourceDoris/clusterEnv/P0/Cluster0/be/lib/doris_be+0x3376e80d)
(BuildId: 865149e62959581e)
apache#1 0x55fc8059db84 in
std::default_delete<doris::pipeline::PipelineTask>::operator()(doris::pipeline::PipelineTask*)
const
/var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/unique_ptr.h:85:2
apache#2 0x55fc8059db84 in std::unique_ptr<doris::pipeline::PipelineTask,
std::default_delete<doris::pipeline::PipelineTask> >::~unique_ptr()
/var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/unique_ptr.h:361:4
apache#3 0x55fc8059db84 in void
std::destroy_at<std::unique_ptr<doris::pipeline::PipelineTask,
std::default_delete<doris::pipeline::PipelineTask> >
>(std::unique_ptr<doris::pipeline::PipelineTask,
std::default_delete<doris::pipeline::PipelineTask> >*)
/var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/stl_construct.h:88:15
apache#4 0x55fc8059db84 in void
std::_Destroy<std::unique_ptr<doris::pipeline::PipelineTask,
std::default_delete<doris::pipeline::PipelineTask> >
>(std::unique_ptr<doris::pipeline::PipelineTask,
std::default_delete<doris::pipeline::PipelineTask> >*)
/var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/stl_construct.h:138:7
apache#5 0x55fc8059db84 in void
std::_Destroy_aux<false>::__destroy<std::unique_ptr<doris::pipeline::PipelineTask,
std::default_delete<doris::pipeline::PipelineTask>
>*>(std::unique_ptr<doris::pipeline::PipelineTask,
std::default_delete<doris::pipeline::PipelineTask> >*,
std::unique_ptr<doris::pipeline::PipelineTask,
std::default_delete<doris::pipeline::PipelineTask> >*)
/var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/stl_construct.h:152:6
apache#6 0x55fc8059db84 in void
std::_Destroy<std::unique_ptr<doris::pipeline::PipelineTask,
std::default_delete<doris::pipeline::PipelineTask>
>*>(std::unique_ptr<doris::pipeline::PipelineTask,
std::default_delete<doris::pipeline::PipelineTask> >*,
std::unique_ptr<doris::pipeline::PipelineTask,
std::default_delete<doris::pipeline::PipelineTask> >*)
/var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/stl_construct.h:184:7
apache#7 0x55fc8059db84 in void
std::_Destroy<std::unique_ptr<doris::pipeline::PipelineTask,
std::default_delete<doris::pipeline::PipelineTask> >*,
std::unique_ptr<doris::pipeline::PipelineTask,
std::default_delete<doris::pipeline::PipelineTask> >
>(std::unique_ptr<doris::pipeline::PipelineTask,
std::default_delete<doris::pipeline::PipelineTask> >*,
std::unique_ptr<doris::pipeline::PipelineTask,
std::default_delete<doris::pipeline::PipelineTask> >*,
std::allocator<std::unique_ptr<doris::pipeline::PipelineTask,
std::default_delete<doris::pipeline::PipelineTask> > >&)
/var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/alloc_traits.h:746:7
apache#8 0x55fc8059db84 in
std::vector<std::unique_ptr<doris::pipeline::PipelineTask,
std::default_delete<doris::pipeline::PipelineTask> >,
std::allocator<std::unique_ptr<doris::pipeline::PipelineTask,
std::default_delete<doris::pipeline::PipelineTask> > > >::~vector()
/var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/stl_vector.h:680:2
apache#9 0x55fc8052571c in void
std::destroy_at<std::vector<std::unique_ptr<doris::pipeline::PipelineTask,
std::default_delete<doris::pipeline::PipelineTask> >,
std::allocator<std::unique_ptr<doris::pipeline::PipelineTask,
std::default_delete<doris::pipeline::PipelineTask> > > >
>(std::vector<std::unique_ptr<doris::pipeline::PipelineTask,
std::default_delete<doris::pipeline::PipelineTask> >,
std::allocator<std::unique_ptr<doris::pipeline::PipelineTask,
std::default_delete<doris::pipeline::PipelineTask> > > >*)
/var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/stl_construct.h:88:15
apache#10 0x55fc8052571c in void
std::_Destroy<std::vector<std::unique_ptr<doris::pipeline::PipelineTask,
std::default_delete<doris::pipeline::PipelineTask> >,
std::allocator<std::unique_ptr<doris::pipeline::PipelineTask,
std::default_delete<doris::pipeline::PipelineTask> > > >
>(std::vector<std::unique_ptr<doris::pipeline::PipelineTask,
std::default_delete<doris::pipeline::PipelineTask> >,
std::allocator<std::unique_ptr<doris::pipeline::PipelineTask,
std::default_delete<doris::pipeline::PipelineTask> > > >*)
/var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/stl_construct.h:138:7
apache#11 0x55fc8052571c in void
std::_Destroy_aux<false>::__destroy<std::vector<std::unique_ptr<doris::pipeline::PipelineTask,
std::default_delete<doris::pipeline::PipelineTask> >,
std::allocator<std::unique_ptr<doris::pipeline::PipelineTask,
std::default_delete<doris::pipeline::PipelineTask> > >
>*>(std::vector<std::unique_ptr<doris::pipeline::PipelineTask,
std::default_delete<doris::pipeline::PipelineTask> >,
std::allocator<std::unique_ptr<doris::pipeline::PipelineTask,
std::default_delete<doris::pipeline::PipelineTask> > > >*,
std::vector<std::unique_ptr<doris::pipeline::PipelineTask,
std::default_delete<doris::pipeline::PipelineTask> >,
std::allocator<std::unique_ptr<doris::pipeline::PipelineTask,
std::default_delete<doris::pipeline::PipelineTask> > > >*)
/var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/stl_construct.h:152:6
apache#12 0x55fc8052571c in void
std::_Destroy<std::vector<std::unique_ptr<doris::pipeline::PipelineTask,
std::default_delete<doris::pipeline::PipelineTask> >,
std::allocator<std::unique_ptr<doris::pipeline::PipelineTask,
std::default_delete<doris::pipeline::PipelineTask> > >
>*>(std::vector<std::unique_ptr<doris::pipeline::PipelineTask,
std::default_delete<doris::pipeline::PipelineTask> >,
std::allocator<std::unique_ptr<doris::pipeline::PipelineTask,
std::default_delete<doris::pipeline::PipelineTask> > > >*,
std::vector<std::unique_ptr<doris::pipeline::PipelineTask,
std::default_delete<doris::pipeline::PipelineTask> >,
std::allocator<std::unique_ptr<doris::pipeline::PipelineTask,
std::default_delete<doris::pipeline::PipelineTask> > > >*)
/var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/stl_construct.h:184:7
apache#13 0x55fc8052571c in void
std::_Destroy<std::vector<std::unique_ptr<doris::pipeline::PipelineTask,
std::default_delete<doris::pipeline::PipelineTask> >,
std::allocator<std::unique_ptr<doris::pipeline::PipelineTask,
std::default_delete<doris::pipeline::PipelineTask> > > >*,
std::vector<std::unique_ptr<doris::pipeline::PipelineTask,
std::default_delete<doris::pipeline::PipelineTask> >,
std::allocator<std::unique_ptr<doris::pipeline::PipelineTask,
std::default_delete<doris::pipeline::PipelineTask> > > >
>(std::vector<std::unique_ptr<doris::pipeline::PipelineTask,
std::default_delete<doris::pipeline::PipelineTask> >,
std::allocator<std::unique_ptr<doris::pipeline::PipelineTask,
std::default_delete<doris::pipeline::PipelineTask> > > >*,
std::vector<std::unique_ptr<doris::pipeline::PipelineTask,
std::default_delete<doris::pipeline::PipelineTask> >,
std::allocator<std::unique_ptr<doris::pipeline::PipelineTask,
std::default_delete<doris::pipeline::PipelineTask> > > >*,
std::allocator<std::vector<std::unique_ptr<doris::pipeline::PipelineTask,
std::default_delete<doris::pipeline::PipelineTask> >,
std::allocator<std::unique_ptr<doris::pipeline::PipelineTask,
std::default_delete<doris::pipeline::PipelineTask> > > > >&)
/var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/alloc_traits.h:746:7
apache#14 0x55fc8052571c in
std::vector<std::vector<std::unique_ptr<doris::pipeline::PipelineTask,
std::default_delete<doris::pipeline::PipelineTask> >,
std::allocator<std::unique_ptr<doris::pipeline::PipelineTask,
std::default_delete<doris::pipeline::PipelineTask> > > >,
std::allocator<std::vector<std::unique_ptr<doris::pipeline::PipelineTask,
std::default_delete<doris::pipeline::PipelineTask> >,
std::allocator<std::unique_ptr<doris::pipeline::PipelineTask,
std::default_delete<doris::pipeline::PipelineTask> > > > >
>::_M_erase_at_end(std::vector<std::unique_ptr<doris::pipeline::PipelineTask,
std::default_delete<doris::pipeline::PipelineTask> >,
std::allocator<std::unique_ptr<doris::pipeline::PipelineTask,
std::default_delete<doris::pipeline::PipelineTask> > > >*)
/var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/stl_vector.h:1796:6
apache#15 0x55fc8052571c in
std::vector<std::vector<std::unique_ptr<doris::pipeline::PipelineTask,
std::default_delete<doris::pipeline::PipelineTask> >,
std::allocator<std::unique_ptr<doris::pipeline::PipelineTask,
std::default_delete<doris::pipeline::PipelineTask> > > >,
std::allocator<std::vector<std::unique_ptr<doris::pipeline::PipelineTask,
std::default_delete<doris::pipeline::PipelineTask> >,
std::allocator<std::unique_ptr<doris::pipeline::PipelineTask,
std::default_delete<doris::pipeline::PipelineTask> > > > > >::clear()
/var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/stl_vector.h:1499:9
apache#16 0x55fc8052571c in
doris::pipeline::PipelineFragmentContext::~PipelineFragmentContext()
/root/doris/be/src/pipeline/pipeline_fragment_context.cpp:142:12
apache#17 0x55fc47ad30cc in
std::_Sp_counted_base<(__gnu_cxx::_Lock_policy)2>::_M_release()
/var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/shared_ptr_base.h:168:6
apache#18 0x55fc80658d57 in
std::__shared_count<(__gnu_cxx::_Lock_policy)2>::~__shared_count()
/var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/shared_ptr_base.h:702:11
apache#19 0x55fc80658d57 in std::__shared_ptr<doris::TaskExecutionContext,
(__gnu_cxx::_Lock_policy)2>::~__shared_ptr()
/var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/shared_ptr_base.h:1149:31
apache#20 0x55fc80658d57 in
doris::pipeline::close_task(doris::pipeline::PipelineTask*,
doris::Status) /root/doris/be/src/pipeline/task_scheduler.cpp:100:1
apache#21 0x55fc8065aa17 in doris::pipeline::TaskScheduler::_do_work(int)
/root/doris/be/src/pipeline/task_scheduler.cpp:160:36
apache#22 0x55fc4c57cd19 in doris::ThreadPool::dispatch_thread()
/root/doris/be/src/util/threadpool.cpp:608:24
apache#23 0x55fc4c55395e in std::function<void ()>::operator()() const
/var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/std_function.h:560:9
apache#24 0x55fc4c55395e in doris::Thread::supervise_thread(void*)
/root/doris/be/src/util/thread.cpp:498:5
apache#25 0x7f9ee3d25608 in start_thread
/build/glibc-SzIz7B/glibc-2.31/nptl/pthread_create.c:477:8
@Gabriel39 Gabriel39 mentioned this issue Feb 26, 2025
Merged
16 tasks
Gabriel39 added a commit that referenced this issue Feb 26, 2025
@Gabriel39
[fix](schema scan) Fix invalid pointer access (#48313)
961a844 
Schema scanner runs on a separate thread which is executed
asynchronously. We should make sure all context used not be freed once
it is scheduled.

ERROR: AddressSanitizer: heap-buffer-overflow on address 0x613002f33eb2
at pc 0x55e085dccbe3 bp 0x7f345c0e1f10 sp 0x7f345c0e1f08
READ of size 1 at 0x613002f33eb2 thread T2776 (FragmentMgrAsyn)
#0 0x55e085dccbe2 in std::__atomic_base::load(std::memory_order) const
/var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/atomic_base.h:481:9
#1 0x55e085dccbe2 in std::atomic::operator bool() const
/var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/atomic:87:22
#2 0x55e085dccbe2 in
doris::SchemaScanner::get_next_block_async(doris::RuntimeState*)::$_0::operator()()
const
/home/zcp/repo_center/doris_master/doris/be/src/exec/schema_scanner.cpp:118:5
#3 0x55e085dccbe2 in void std::__invoke_impl(std::__invoke_other,
doris::SchemaScanner::get_next_block_async(doris::RuntimeState*)::$_0&)
/var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/invoke.h:61:14
#4 0x55e085dccbe2 in std::enable_if, void>::type
std::__invoke_r(doris::SchemaScanner::get_next_block_async(doris::RuntimeState*)::$_0&)
/var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/invoke.h:111:2
#5 0x55e085dccbe2 in std::_Function_handler::_M_invoke(std::_Any_data
const&)
/var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/std_function.h:291:9
#6 0x55e050f081ca in doris::ThreadPool::dispatch_thread()
/home/zcp/repo_center/doris_master/doris/be/src/util/threadpool.cpp:608:24
#7 0x55e050ede467 in doris::Thread::supervise_thread(void*)
/home/zcp/repo_center/doris_master/doris/be/src/util/thread.cpp:498:5
    #8 0x7f376ef5aac2 in start_thread nptl/pthread_create.c:442:8
#9 0x7f376efec84f misc/../sysdeps/unix/sysv/linux/x86_64/clone3.S:81
github-actions bot pushed a commit that referenced this issue Feb 26, 2025
@Gabriel39
[fix](schema scan) Fix invalid pointer access (#48313)
3284277 
Schema scanner runs on a separate thread which is executed
asynchronously. We should make sure all context used not be freed once
it is scheduled.

ERROR: AddressSanitizer: heap-buffer-overflow on address 0x613002f33eb2
at pc 0x55e085dccbe3 bp 0x7f345c0e1f10 sp 0x7f345c0e1f08
READ of size 1 at 0x613002f33eb2 thread T2776 (FragmentMgrAsyn)
#0 0x55e085dccbe2 in std::__atomic_base::load(std::memory_order) const
/var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/atomic_base.h:481:9
#1 0x55e085dccbe2 in std::atomic::operator bool() const
/var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/atomic:87:22
#2 0x55e085dccbe2 in
doris::SchemaScanner::get_next_block_async(doris::RuntimeState*)::$_0::operator()()
const
/home/zcp/repo_center/doris_master/doris/be/src/exec/schema_scanner.cpp:118:5
#3 0x55e085dccbe2 in void std::__invoke_impl(std::__invoke_other,
doris::SchemaScanner::get_next_block_async(doris::RuntimeState*)::$_0&)
/var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/invoke.h:61:14
#4 0x55e085dccbe2 in std::enable_if, void>::type
std::__invoke_r(doris::SchemaScanner::get_next_block_async(doris::RuntimeState*)::$_0&)
/var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/invoke.h:111:2
#5 0x55e085dccbe2 in std::_Function_handler::_M_invoke(std::_Any_data
const&)
/var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/std_function.h:291:9
#6 0x55e050f081ca in doris::ThreadPool::dispatch_thread()
/home/zcp/repo_center/doris_master/doris/be/src/util/threadpool.cpp:608:24
#7 0x55e050ede467 in doris::Thread::supervise_thread(void*)
/home/zcp/repo_center/doris_master/doris/be/src/util/thread.cpp:498:5
    #8 0x7f376ef5aac2 in start_thread nptl/pthread_create.c:442:8
#9 0x7f376efec84f misc/../sysdeps/unix/sysv/linux/x86_64/clone3.S:81
github-actions bot pushed a commit that referenced this issue Feb 26, 2025
@Gabriel39
[fix](schema scan) Fix invalid pointer access (#48313)
c01234d 
Schema scanner runs on a separate thread which is executed
asynchronously. We should make sure all context used not be freed once
it is scheduled.

ERROR: AddressSanitizer: heap-buffer-overflow on address 0x613002f33eb2
at pc 0x55e085dccbe3 bp 0x7f345c0e1f10 sp 0x7f345c0e1f08
READ of size 1 at 0x613002f33eb2 thread T2776 (FragmentMgrAsyn)
#0 0x55e085dccbe2 in std::__atomic_base::load(std::memory_order) const
/var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/atomic_base.h:481:9
#1 0x55e085dccbe2 in std::atomic::operator bool() const
/var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/atomic:87:22
#2 0x55e085dccbe2 in
doris::SchemaScanner::get_next_block_async(doris::RuntimeState*)::$_0::operator()()
const
/home/zcp/repo_center/doris_master/doris/be/src/exec/schema_scanner.cpp:118:5
#3 0x55e085dccbe2 in void std::__invoke_impl(std::__invoke_other,
doris::SchemaScanner::get_next_block_async(doris::RuntimeState*)::$_0&)
/var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/invoke.h:61:14
#4 0x55e085dccbe2 in std::enable_if, void>::type
std::__invoke_r(doris::SchemaScanner::get_next_block_async(doris::RuntimeState*)::$_0&)
/var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/invoke.h:111:2
#5 0x55e085dccbe2 in std::_Function_handler::_M_invoke(std::_Any_data
const&)
/var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/std_function.h:291:9
#6 0x55e050f081ca in doris::ThreadPool::dispatch_thread()
/home/zcp/repo_center/doris_master/doris/be/src/util/threadpool.cpp:608:24
#7 0x55e050ede467 in doris::Thread::supervise_thread(void*)
/home/zcp/repo_center/doris_master/doris/be/src/util/thread.cpp:498:5
    #8 0x7f376ef5aac2 in start_thread nptl/pthread_create.c:442:8
#9 0x7f376efec84f misc/../sysdeps/unix/sysv/linux/x86_64/clone3.S:81
@Gabriel39 Gabriel39 mentioned this issue Feb 26, 2025
Merged
16 tasks
Gabriel39 added a commit that referenced this issue Feb 26, 2025
@Gabriel39
[fix](schema scan) Fix invalid pointer access (#48370)
a3eb715 
==14365==ERROR: AddressSanitizer: heap-buffer-overflow on address
0x613002873c70 at pc 0x5559a290ec05 bp 0x7f6c40385e90 sp 0x7f6c40385e88
READ of size 2 at 0x613002873c70 thread T4019 (FragmentMgrAsyn)
#0 0x5559a290ec04 in std::__atomic_base::load(std::memory_order) const
/var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/atomic_base.h:481:9
#1 0x5559a290ec04 in doris::AtomicStatus::update(doris::Status const&)
/home/zcp/repo_center/doris_master/doris/be/src/common/status.h:601:44
#2 0x5559d8f289c0 in
doris::SchemaScanner::get_next_block_async(doris::RuntimeState*)::$_0::operator()()
const
/home/zcp/repo_center/doris_master/doris/be/src/exec/schema_scanner.cpp:118:5
#3 0x5559d8f289c0 in void std::__invoke_impl(std::__invoke_other,
doris::SchemaScanner::get_next_block_async(doris::RuntimeState*)::$_0&)
/var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/invoke.h:61:14
#4 0x5559d8f289c0 in std::enable_if, void>::type
std::__invoke_r(doris::SchemaScanner::get_next_block_async(doris::RuntimeState*)::$_0&)
/var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/invoke.h:111:2
#5 0x5559d8f289c0 in std::_Function_handler::_M_invoke(std::_Any_data
const&)
/var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/std_function.h:291:9
#6 0x5559a4140e4a in doris::ThreadPool::dispatch_thread()
/home/zcp/repo_center/doris_master/doris/be/src/util/threadpool.cpp:608:24
#7 0x5559a4117877 in doris::Thread::supervise_thread(void*)
/home/zcp/repo_center/doris_master/doris/be/src/util/thread.cpp:498:5
    #8 0x7f7912f84ac2 in start_thread nptl/pthread_create.c:442:8
#9 0x7f791301684f misc/../sysdeps/unix/sysv/linux/x86_64/clone3.S:81
github-actions bot pushed a commit that referenced this issue Feb 26, 2025
@Gabriel39
[fix](schema scan) Fix invalid pointer access (#48370)
116d912 
==14365==ERROR: AddressSanitizer: heap-buffer-overflow on address
0x613002873c70 at pc 0x5559a290ec05 bp 0x7f6c40385e90 sp 0x7f6c40385e88
READ of size 2 at 0x613002873c70 thread T4019 (FragmentMgrAsyn)
#0 0x5559a290ec04 in std::__atomic_base::load(std::memory_order) const
/var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/atomic_base.h:481:9
#1 0x5559a290ec04 in doris::AtomicStatus::update(doris::Status const&)
/home/zcp/repo_center/doris_master/doris/be/src/common/status.h:601:44
#2 0x5559d8f289c0 in
doris::SchemaScanner::get_next_block_async(doris::RuntimeState*)::$_0::operator()()
const
/home/zcp/repo_center/doris_master/doris/be/src/exec/schema_scanner.cpp:118:5
#3 0x5559d8f289c0 in void std::__invoke_impl(std::__invoke_other,
doris::SchemaScanner::get_next_block_async(doris::RuntimeState*)::$_0&)
/var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/invoke.h:61:14
#4 0x5559d8f289c0 in std::enable_if, void>::type
std::__invoke_r(doris::SchemaScanner::get_next_block_async(doris::RuntimeState*)::$_0&)
/var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/invoke.h:111:2
#5 0x5559d8f289c0 in std::_Function_handler::_M_invoke(std::_Any_data
const&)
/var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/std_function.h:291:9
#6 0x5559a4140e4a in doris::ThreadPool::dispatch_thread()
/home/zcp/repo_center/doris_master/doris/be/src/util/threadpool.cpp:608:24
#7 0x5559a4117877 in doris::Thread::supervise_thread(void*)
/home/zcp/repo_center/doris_master/doris/be/src/util/thread.cpp:498:5
    #8 0x7f7912f84ac2 in start_thread nptl/pthread_create.c:442:8
#9 0x7f791301684f misc/../sysdeps/unix/sysv/linux/x86_64/clone3.S:81
github-actions bot pushed a commit that referenced this issue Feb 26, 2025
@Gabriel39
[fix](schema scan) Fix invalid pointer access (#48370)
63a5d04 
==14365==ERROR: AddressSanitizer: heap-buffer-overflow on address
0x613002873c70 at pc 0x5559a290ec05 bp 0x7f6c40385e90 sp 0x7f6c40385e88
READ of size 2 at 0x613002873c70 thread T4019 (FragmentMgrAsyn)
#0 0x5559a290ec04 in std::__atomic_base::load(std::memory_order) const
/var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/atomic_base.h:481:9
#1 0x5559a290ec04 in doris::AtomicStatus::update(doris::Status const&)
/home/zcp/repo_center/doris_master/doris/be/src/common/status.h:601:44
#2 0x5559d8f289c0 in
doris::SchemaScanner::get_next_block_async(doris::RuntimeState*)::$_0::operator()()
const
/home/zcp/repo_center/doris_master/doris/be/src/exec/schema_scanner.cpp:118:5
#3 0x5559d8f289c0 in void std::__invoke_impl(std::__invoke_other,
doris::SchemaScanner::get_next_block_async(doris::RuntimeState*)::$_0&)
/var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/invoke.h:61:14
#4 0x5559d8f289c0 in std::enable_if, void>::type
std::__invoke_r(doris::SchemaScanner::get_next_block_async(doris::RuntimeState*)::$_0&)
/var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/invoke.h:111:2
#5 0x5559d8f289c0 in std::_Function_handler::_M_invoke(std::_Any_data
const&)
/var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/std_function.h:291:9
#6 0x5559a4140e4a in doris::ThreadPool::dispatch_thread()
/home/zcp/repo_center/doris_master/doris/be/src/util/threadpool.cpp:608:24
#7 0x5559a4117877 in doris::Thread::supervise_thread(void*)
/home/zcp/repo_center/doris_master/doris/be/src/util/thread.cpp:498:5
    #8 0x7f7912f84ac2 in start_thread nptl/pthread_create.c:442:8
#9 0x7f791301684f misc/../sysdeps/unix/sysv/linux/x86_64/clone3.S:81
zhiqiang-hhhh pushed a commit to zhiqiang-hhhh/doris that referenced this issue Feb 27, 2025
@Gabriel39 @zhiqiang-hhhh
[fix](schema scan) Fix invalid pointer access (apache#48313)
0ca0d43 
Schema scanner runs on a separate thread which is executed
asynchronously. We should make sure all context used not be freed once
it is scheduled.

ERROR: AddressSanitizer: heap-buffer-overflow on address 0x613002f33eb2
at pc 0x55e085dccbe3 bp 0x7f345c0e1f10 sp 0x7f345c0e1f08
READ of size 1 at 0x613002f33eb2 thread T2776 (FragmentMgrAsyn)
#0 0x55e085dccbe2 in std::__atomic_base::load(std::memory_order) const
/var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/atomic_base.h:481:9
#1 0x55e085dccbe2 in std::atomic::operator bool() const
/var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/atomic:87:22
apache#2 0x55e085dccbe2 in
doris::SchemaScanner::get_next_block_async(doris::RuntimeState*)::$_0::operator()()
const
/home/zcp/repo_center/doris_master/doris/be/src/exec/schema_scanner.cpp:118:5
apache#3 0x55e085dccbe2 in void std::__invoke_impl(std::__invoke_other,
doris::SchemaScanner::get_next_block_async(doris::RuntimeState*)::$_0&)
/var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/invoke.h:61:14
apache#4 0x55e085dccbe2 in std::enable_if, void>::type
std::__invoke_r(doris::SchemaScanner::get_next_block_async(doris::RuntimeState*)::$_0&)
/var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/invoke.h:111:2
apache#5 0x55e085dccbe2 in std::_Function_handler::_M_invoke(std::_Any_data
const&)
/var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/std_function.h:291:9
apache#6 0x55e050f081ca in doris::ThreadPool::dispatch_thread()
/home/zcp/repo_center/doris_master/doris/be/src/util/threadpool.cpp:608:24
apache#7 0x55e050ede467 in doris::Thread::supervise_thread(void*)
/home/zcp/repo_center/doris_master/doris/be/src/util/thread.cpp:498:5
    apache#8 0x7f376ef5aac2 in start_thread nptl/pthread_create.c:442:8
apache#9 0x7f376efec84f misc/../sysdeps/unix/sysv/linux/x86_64/clone3.S:81
zhiqiang-hhhh pushed a commit to zhiqiang-hhhh/doris that referenced this issue Feb 27, 2025
@Gabriel39 @zhiqiang-hhhh
[fix](schema scan) Fix invalid pointer access (apache#48370)
0ef1690 
==14365==ERROR: AddressSanitizer: heap-buffer-overflow on address
0x613002873c70 at pc 0x5559a290ec05 bp 0x7f6c40385e90 sp 0x7f6c40385e88
READ of size 2 at 0x613002873c70 thread T4019 (FragmentMgrAsyn)
#0 0x5559a290ec04 in std::__atomic_base::load(std::memory_order) const
/var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/atomic_base.h:481:9
#1 0x5559a290ec04 in doris::AtomicStatus::update(doris::Status const&)
/home/zcp/repo_center/doris_master/doris/be/src/common/status.h:601:44
apache#2 0x5559d8f289c0 in
doris::SchemaScanner::get_next_block_async(doris::RuntimeState*)::$_0::operator()()
const
/home/zcp/repo_center/doris_master/doris/be/src/exec/schema_scanner.cpp:118:5
apache#3 0x5559d8f289c0 in void std::__invoke_impl(std::__invoke_other,
doris::SchemaScanner::get_next_block_async(doris::RuntimeState*)::$_0&)
/var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/invoke.h:61:14
apache#4 0x5559d8f289c0 in std::enable_if, void>::type
std::__invoke_r(doris::SchemaScanner::get_next_block_async(doris::RuntimeState*)::$_0&)
/var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/invoke.h:111:2
apache#5 0x5559d8f289c0 in std::_Function_handler::_M_invoke(std::_Any_data
const&)
/var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/std_function.h:291:9
apache#6 0x5559a4140e4a in doris::ThreadPool::dispatch_thread()
/home/zcp/repo_center/doris_master/doris/be/src/util/threadpool.cpp:608:24
apache#7 0x5559a4117877 in doris::Thread::supervise_thread(void*)
/home/zcp/repo_center/doris_master/doris/be/src/util/thread.cpp:498:5
    apache#8 0x7f7912f84ac2 in start_thread nptl/pthread_create.c:442:8
apache#9 0x7f791301684f misc/../sysdeps/unix/sysv/linux/x86_64/clone3.S:81
seawinde pushed a commit to seawinde/doris that referenced this issue Feb 28, 2025
@Gabriel39 @seawinde
[fix](schema scan) Fix invalid pointer access (apache#48313)
7f18b1b 
Schema scanner runs on a separate thread which is executed
asynchronously. We should make sure all context used not be freed once
it is scheduled.

ERROR: AddressSanitizer: heap-buffer-overflow on address 0x613002f33eb2
at pc 0x55e085dccbe3 bp 0x7f345c0e1f10 sp 0x7f345c0e1f08
READ of size 1 at 0x613002f33eb2 thread T2776 (FragmentMgrAsyn)
#0 0x55e085dccbe2 in std::__atomic_base::load(std::memory_order) const
/var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/atomic_base.h:481:9
#1 0x55e085dccbe2 in std::atomic::operator bool() const
/var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/atomic:87:22
apache#2 0x55e085dccbe2 in
doris::SchemaScanner::get_next_block_async(doris::RuntimeState*)::$_0::operator()()
const
/home/zcp/repo_center/doris_master/doris/be/src/exec/schema_scanner.cpp:118:5
apache#3 0x55e085dccbe2 in void std::__invoke_impl(std::__invoke_other,
doris::SchemaScanner::get_next_block_async(doris::RuntimeState*)::$_0&)
/var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/invoke.h:61:14
apache#4 0x55e085dccbe2 in std::enable_if, void>::type
std::__invoke_r(doris::SchemaScanner::get_next_block_async(doris::RuntimeState*)::$_0&)
/var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/invoke.h:111:2
apache#5 0x55e085dccbe2 in std::_Function_handler::_M_invoke(std::_Any_data
const&)
/var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/std_function.h:291:9
apache#6 0x55e050f081ca in doris::ThreadPool::dispatch_thread()
/home/zcp/repo_center/doris_master/doris/be/src/util/threadpool.cpp:608:24
apache#7 0x55e050ede467 in doris::Thread::supervise_thread(void*)
/home/zcp/repo_center/doris_master/doris/be/src/util/thread.cpp:498:5
    apache#8 0x7f376ef5aac2 in start_thread nptl/pthread_create.c:442:8
apache#9 0x7f376efec84f misc/../sysdeps/unix/sysv/linux/x86_64/clone3.S:81
seawinde pushed a commit to seawinde/doris that referenced this issue Feb 28, 2025
@Gabriel39 @seawinde
[fix](schema scan) Fix invalid pointer access (apache#48370)
c70400d 
==14365==ERROR: AddressSanitizer: heap-buffer-overflow on address
0x613002873c70 at pc 0x5559a290ec05 bp 0x7f6c40385e90 sp 0x7f6c40385e88
READ of size 2 at 0x613002873c70 thread T4019 (FragmentMgrAsyn)
#0 0x5559a290ec04 in std::__atomic_base::load(std::memory_order) const
/var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/atomic_base.h:481:9
#1 0x5559a290ec04 in doris::AtomicStatus::update(doris::Status const&)
/home/zcp/repo_center/doris_master/doris/be/src/common/status.h:601:44
apache#2 0x5559d8f289c0 in
doris::SchemaScanner::get_next_block_async(doris::RuntimeState*)::$_0::operator()()
const
/home/zcp/repo_center/doris_master/doris/be/src/exec/schema_scanner.cpp:118:5
apache#3 0x5559d8f289c0 in void std::__invoke_impl(std::__invoke_other,
doris::SchemaScanner::get_next_block_async(doris::RuntimeState*)::$_0&)
/var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/invoke.h:61:14
apache#4 0x5559d8f289c0 in std::enable_if, void>::type
std::__invoke_r(doris::SchemaScanner::get_next_block_async(doris::RuntimeState*)::$_0&)
/var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/invoke.h:111:2
apache#5 0x5559d8f289c0 in std::_Function_handler::_M_invoke(std::_Any_data
const&)
/var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/std_function.h:291:9
apache#6 0x5559a4140e4a in doris::ThreadPool::dispatch_thread()
/home/zcp/repo_center/doris_master/doris/be/src/util/threadpool.cpp:608:24
apache#7 0x5559a4117877 in doris::Thread::supervise_thread(void*)
/home/zcp/repo_center/doris_master/doris/be/src/util/thread.cpp:498:5
    apache#8 0x7f7912f84ac2 in start_thread nptl/pthread_create.c:442:8
apache#9 0x7f791301684f misc/../sysdeps/unix/sysv/linux/x86_64/clone3.S:81
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@maruyue @luyee