Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

tiflash crash frequently with error of TryFlushData faild #5602

Closed
lilinghai opened this issue Aug 11, 2022 · 1 comment · Fixed by #5603
Closed

tiflash crash frequently with error of TryFlushData faild #5602

lilinghai opened this issue Aug 11, 2022 · 1 comment · Fixed by #5603

Comments

@lilinghai
Copy link

lilinghai commented Aug 11, 2022

Bug Report

Please answer these questions before submitting your issue. Thanks!

1. Minimal reproduce step (Required)

[2022/08/11 01:56:46.760 +08:00] [ERROR] [Exception.cpp:85] ["uint8_t DB::TryFlushData(DB::EngineStoreServerWrap *, uint64_t, uint8_t, uint64_t, uint64_t):Code: 0, e.displayText() = DB::Exception: region not found when trying flush, e.what() = DB::Exception, Stack trace:

       0x1a91a98    DB::Exception::Exception(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, int) [tiflash+27859608]
                    dbms/src/Common/Exception.h:41
       0x651ab64    DB::KVStore::canFlushRegionDataImpl(std::__1::shared_ptr<DB::Region> const&, unsigned char, bool, DB::TMTContext&, DB::RegionTaskLock const&, unsigned long, unsigned long) [tiflash+106015588]
                    dbms/src/Storages/Transaction/KVStore.cpp:348
       0x651ad18    DB::KVStore::tryFlushRegionData(unsigned long, bool, DB::TMTContext&, unsigned long, unsigned long) [tiflash+106016024]
                    dbms/src/Storages/Transaction/KVStore.cpp:341
       0x65310f8    TryFlushData [tiflash+106107128]
                    dbms/src/Storages/Transaction/ProxyFFI.cpp:150
  0xffffacadfafc    _$LT$raftstore..engine_store_ffi..observer..TiFlashObserver$u20$as$u20$raftstore..coprocessor..AdminObserver$GT$::pre_exec_admin::hfd267e9df0a1ada2 [[libtiflash_proxy.so](http://libtiflash_proxy.so/)+30669564]
  0xffffac476d38    raftstore::store::fsm::apply::ApplyDelegate$LT$EK$GT$::process_raft_cmd::ha0396bd1778fd5cd [[libtiflash_proxy.so](http://libtiflash_proxy.so/)+23948600]
  0xffffac475f34    raftstore::store::fsm::apply::ApplyDelegate$LT$EK$GT$::handle_raft_committed_entries::h0aac2d505272f3fb [[libtiflash_proxy.so](http://libtiflash_proxy.so/)+23945012]
  0xffffac491034    raftstore::store::fsm::apply::ApplyFsm$LT$EK$GT$::handle_apply::h74c0b72499b90339 [[libtiflash_proxy.so](http://libtiflash_proxy.so/)+24055860]
  0xffffac4939f8    raftstore::store::fsm::apply::ApplyFsm$LT$EK$GT$::handle_tasks::hc57505dc8bb700b6 [[libtiflash_proxy.so](http://libtiflash_proxy.so/)+24066552]
  0xffffac496324    _$LT$raftstore..store..fsm..apply..ApplyPoller$LT$EK$GT$$u20$as$u20$batch_system..batch..PollHandler$LT$raftstore..store..fsm..apply..ApplyFsm$LT$EK$GT$$C$raftstore..store..fsm..apply..ControlFsm$GT$$GT$::handle_normal::ha4565cf6507a3216 [[libtiflash_proxy.so](http://libtiflash_proxy.so/)+24077092]
  0xffffac0a5508    batch_system::batch::Poller$LT$N$C$C$C$Handler$GT$::poll::hc14ea824bead376d [[libtiflash_proxy.so](http://libtiflash_proxy.so/)+19944712]
  0xffffac77136c    std::sys_common::backtrace::__rust_begin_short_backtrace::h80953f8e29109df0 [[libtiflash_proxy.so](http://libtiflash_proxy.so/)+27071340]
  0xffffabf96908    core::ops::function::FnOnce::call_once$u7b$$u7b$vtable.shim$u7d$$u7d$::h8b14e8f2aab21868 [[libtiflash_proxy.so](http://libtiflash_proxy.so/)+18835720]
  0xfffface2f01c    std::sys::unix::thread::Thread::new::thread_start::h4e461bcfb14a5bff [[libtiflash_proxy.so](http://libtiflash_proxy.so/)+34140188]
  0xffffaac37d38    start_thread [libpthread.so.0+32056]
  0xffffaab3f690    thread_start [libc.so.6+915088]"] [thread_id=93]
[2022/08/11 01:56:46.762 +08:00] [ERROR] [BaseDaemon.cpp:377] [BaseDaemon:########################################] [thread_id=239]
[2022/08/11 01:56:46.763 +08:00] [ERROR] [BaseDaemon.cpp:378] ["BaseDaemon:(from thread 178) Received signal Segmentation fault(11)."] [thread_id=239]
[2022/08/11 01:56:46.763 +08:00] [ERROR] [BaseDaemon.cpp:408] ["BaseDaemon:Address: 0x1390"] [thread_id=239]
[2022/08/11 01:56:46.763 +08:00] [ERROR] [BaseDaemon.cpp:423] ["BaseDaemon:Address not mapped to object."] [thread_id=239]
[2022/08/11 01:56:46.765 +08:00] [WARN] [CoprocessorHandler.cpp:143] ["CoprocessorHandler:RegionException: region 22449, message: NOT_FOUND"] [thread_id=117]
[2022/08/11 01:56:46.765 +08:00] [ERROR] [BaseDaemon.cpp:570] ["BaseDaemon:
       0x5a1cad0    faultSignalHandler(int, siginfo_t*, void*) [tiflash+94489296]
                    libs/libdaemon/src/BaseDaemon.cpp:221
  0xffffae11066c    <unknown symbol> [+1644]"] [thread_id=239]

2. What did you expect to see? (Required)

3. What did you see instead (Required)

4. What is your TiFlash version? (Required)

@CalvinNeo
Copy link
Member

/run-build-arm64-release comment=true

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants