Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Nebula Graphd Coredump when time-out #5747

Closed
flymysql opened this issue Oct 30, 2023 · 0 comments
Closed

Nebula Graphd Coredump when time-out #5747

flymysql opened this issue Oct 30, 2023 · 0 comments
Labels
affects/none PR/issue: this bug affects none version. process/fixed Process of bug severity/none Severity of bug type/bug Type: something is unexpected

Comments

@flymysql
Copy link
Contributor

flymysql commented Oct 30, 2023

Please check the FAQ documentation before raising an issue

I have the same problem as in the link below, but I am not using the nebula-go client side, I am using the nebula-java client side, but the call stack of graphd crash is still the same.

Describe the bug (required)
https://discuss.nebula-graph.com.cn/t/topic/10101/13
#4635

  • nebula version v3.2.1
#0  0x0000000005e86ba9 in apache::thrift::Cpp2Connection::stop() ()
    at /install_temp/gcc-10.3.0/gcc-10.3.0/x86_64-pc-linux-gnu/libstdc++-v3/include/bits/shared_ptr_base.h:1183
#1  0x0000000005e88889 in apache::thrift::Cpp2Connection::channelClosed(folly::exception_wrapper&&) ()
    at /install_temp/gcc-10.3.0/gcc-10.3.0/x86_64-pc-linux-gnu/libstdc++-v3/include/bits/shared_ptr_base.h:1183
#2  0x0000000005e9c567 in apache::thrift::HeaderServerChannel::messageChannelEOF() ()
    at /install_temp/gcc-10.3.0/gcc-10.3.0/x86_64-pc-linux-gnu/libstdc++-v3/include/bits/shared_ptr_base.h:1183
#3  0x0000000005e401d6 in apache::thrift::Cpp2Channel::processReadEOF() ()
    at /install_temp/gcc-10.3.0/gcc-10.3.0/x86_64-pc-linux-gnu/libstdc++-v3/include/bits/shared_ptr_base.h:1183
#4  0x0000000005e47132 in non-virtual thunk to wangle::ContextImpl<apache::thrift::Cpp2Channel>::readEOF() ()
    at /install_temp/gcc-10.3.0/gcc-10.3.0/x86_64-pc-linux-gnu/libstdc++-v3/include/bits/shared_ptr_base.h:1183
#5  0x0000000005e4568d in non-virtual thunk to wangle::ContextImpl<apache::thrift::FramingHandler>::readEOF() ()
    at /install_temp/gcc-10.3.0/gcc-10.3.0/x86_64-pc-linux-gnu/libstdc++-v3/include/bits/shared_ptr_base.h:1183
#6  0x0000000005e4550a in wangle::ContextImpl<apache::thrift::TAsyncTransportHandler>::fireReadEOF() ()
    at /install_temp/gcc-10.3.0/gcc-10.3.0/x86_64-pc-linux-gnu/libstdc++-v3/include/bits/shared_ptr_base.h:1183
#7  0x0000000005e43570 in non-virtual thunk to apache::thrift::TAsyncTransportHandler::readEOF() ()
    at /install_temp/gcc-10.3.0/gcc-10.3.0/x86_64-pc-linux-gnu/libstdc++-v3/include/bits/shared_ptr_base.h:1183
#8  0x0000000006013bae in folly::AsyncSocket::handleRead() ()
    at /install_temp/gcc-10.3.0/gcc-10.3.0/x86_64-pc-linux-gnu/libstdc++-v3/include/bits/shared_ptr_base.h:1183
#9  0x000000000600898a in folly::AsyncSocket::ioReady(unsigned short) ()
    at /install_temp/gcc-10.3.0/gcc-10.3.0/x86_64-pc-linux-gnu/libstdc++-v3/include/bits/shared_ptr_base.h:1183
#10 0x00000000060e2315 in ?? () at /install_temp/gcc-10.3.0/gcc-10.3.0/x86_64-pc-linux-gnu/libstdc++-v3/include/bits/shared_ptr_base.h:1183
#11 0x00000000060e297f in event_base_loop ()
    at /install_temp/gcc-10.3.0/gcc-10.3.0/x86_64-pc-linux-gnu/libstdc++-v3/include/bits/shared_ptr_base.h:1183
#12 0x000000000602180e in folly::EventBase::loopBody(int, bool) ()
    at /install_temp/gcc-10.3.0/gcc-10.3.0/x86_64-pc-linux-gnu/libstdc++-v3/include/bits/shared_ptr_base.h:1183
#13 0x00000000060220fe in folly::EventBase::loop() ()
    at /install_temp/gcc-10.3.0/gcc-10.3.0/x86_64-pc-linux-gnu/libstdc++-v3/include/bits/shared_ptr_base.h:1183
#14 0x0000000006024d38 in folly::EventBase::loopForever() ()
    at /install_temp/gcc-10.3.0/gcc-10.3.0/x86_64-pc-linux-gnu/libstdc++-v3/include/bits/shared_ptr_base.h:1183
#15 0x0000000005faf549 in folly::IOThreadPoolExecutor::threadRun(std::shared_ptr<folly::ThreadPoolExecutor::Thread>) ()
    at /install_temp/gcc-10.3.0/gcc-10.3.0/x86_64-pc-linux-gnu/libstdc++-v3/include/bits/shared_ptr_base.h:1183
#16 0x0000000005fbdab7 in void folly::detail::function::FunctionTraits<void ()>::callBig<std::_Bind<void (folly::ThreadPoolExecutor::*(folly::ThreadPoolExecutor*, std::shared_ptr<folly::ThreadPoolExecutor::Thread>))(std::shared_ptr<folly::ThreadPoolExecutor::Thread>)> >(folly::detail::function::Data&) () at /install_temp/gcc-10.3.0/gcc-10.3.0/x86_64-pc-linux-gnu/libstdc++-v3/include/bits/shared_ptr_base.h:1183
#17 0x0000000003eb1430 in folly::detail::function::FunctionTraits<void ()>::operator()() (this=0x7f9f43212410)
    at ../../../third-party/third-party/include/folly/Function.h:400
#18 0x0000000003f3a70a in folly::NamedThreadFactory::newThread(folly::Function<void ()>&&)::{lambda()#1}::operator()() (
--Type <RET> for more, q to quit, c to continue without paging--
    __closure=0x7f9f43212410) at ../../../../third-party/third-party/include/folly/executors/thread_factory/NamedThreadFactory.h:40
#19 0x0000000003f7a23b in std::__invoke_impl<void, folly::NamedThreadFactory::newThread(folly::Function<void ()>&&)::{lambda()#1}>(std::__invoke_other, folly::NamedThreadFactory::newThread(folly::Function<void ()>&&)::{lambda()#1}&&) (__f=...)
    at /opt/buildtools/gcc-10.3.0/include/c++/10.3.0/bits/invoke.h:60
#20 0x0000000003f79d05 in std::__invoke<folly::NamedThreadFactory::newThread(folly::Function<void ()>&&)::{lambda()#1}>(std::__invoke_result&&, (folly::NamedThreadFactory::newThread(folly::Function<void ()>&&)::{lambda()#1}&&)...) (__fn=...)
    at /opt/buildtools/gcc-10.3.0/include/c++/10.3.0/bits/invoke.h:95
#21 0x0000000003f79a00 in std::thread::_Invoker<std::tuple<folly::NamedThreadFactory::newThread(folly::Function<void ()>&&)::{lambda()#1}> >::_M_invoke<0ul>(std::_Index_tuple<0ul>) (this=0x7f9f43212410) at /opt/buildtools/gcc-10.3.0/include/c++/10.3.0/thread:264
#22 0x0000000003f7963c in std::thread::_Invoker<std::tuple<folly::NamedThreadFactory::newThread(folly::Function<void ()>&&)::{lambda()#1}> >::operator()() (this=0x7f9f43212410) at /opt/buildtools/gcc-10.3.0/include/c++/10.3.0/thread:271
#23 0x0000000003f78f7e in std::thread::_State_impl<std::thread::_Invoker<std::tuple<folly::NamedThreadFactory::newThread(folly::Function<void ()>&&)::{lambda()#1}> > >::_M_run() (this=0x7f9f43212400) at /opt/buildtools/gcc-10.3.0/include/c++/10.3.0/thread:215
#24 0x00000000065f1e7e in std::execute_native_thread_routine (__p=0x7f9f43212400) at ../../../.././libstdc++-v3/src/c++11/thread.cc:78
#25 0x00007f9f43884f3b in ?? () from /usr/lib64/libpthread.so.0
#26 0x00007f9f437bc840 in clone () from /usr/lib64/libc.so.6

Your Environments (required)

  • OS: uname -a
Linux ncn4a-wisemlopsdppservice-32-203-100 4.18.0-147.5.1.6.h934.eulerosv2r9.x86_64 #1 SMP Sat Feb 4 09:00:27 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
  • Compiler: g++ --version or clang++ --version
g++ (Ubuntu 10.5.0-1ubuntu1~20.04) 10.5.0
  • CPU: lscpu
Architecture:                    x86_64
CPU op-mode(s):                  32-bit, 64-bit
Byte Order:                      Little Endian
Address sizes:                   42 bits physical, 48 bits virtual
CPU(s):                          16
On-line CPU(s) list:             0-15
Thread(s) per core:              2
Core(s) per socket:              8
Socket(s):                       1
NUMA node(s):                    1
Vendor ID:                       GenuineIntel
CPU family:                      6
Model:                           85
Model name:                      Intel(R) Xeon(R) Gold 6266C CPU @ 3.
                                 00GHz
Stepping:                        7
CPU MHz:                         3000.000
BogoMIPS:                        6000.00
Hypervisor vendor:               KVM
Virtualization type:             full
L1d cache:                       256 KiB
L1i cache:                       256 KiB
L2 cache:                        8 MiB
L3 cache:                        30.3 MiB
NUMA node0 CPU(s):               0-15
Vulnerability Itlb multihit:     Processor vulnerable
Vulnerability L1tf:              Not affected
Vulnerability Mds:               Not affected
Vulnerability Meltdown:          Not affected
Vulnerability Mmio stale data:   Vulnerable: Clear CPU buffers attemp
                                 ted, no microcode; SMT Host state un
                                 known
Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass
                                  disabled via prctl and seccomp
Vulnerability Spectre v1:        Mitigation; usercopy/swapgs barriers
                                  and __user pointer sanitization
Vulnerability Spectre v2:        Vulnerable, IBPB: disabled, STIBP: d
                                 isabled, PBRSB-eIBRS: Vulnerable
Vulnerability Srbds:             Not affected
Vulnerability Tsx async abort:   Vulnerable: Clear CPU buffers attemp
                                 ted, no microcode; SMT Host state un
                                 known
Flags:                           fpu vme de pse tsc msr pae mce cx8 a
                                 pic sep mtrr pge mca cmov pat pse36 
                                 clflush mmx fxsr sse sse2 ss ht sysc
                                 all nx pdpe1gb rdtscp lm constant_ts
                                 c rep_good nopl xtopology nonstop_ts
                                 c cpuid tsc_known_freq pni pclmulqdq
                                  ssse3 fma cx16 pcid sse4_1 sse4_2 x
                                 2apic movbe popcnt tsc_deadline_time
                                 r aes xsave avx f16c rdrand hypervis
                                 or lahf_lm abm 3dnowprefetch invpcid
                                 _single ssbd ibrs ibpb stibp ibrs_en
                                 hanced fsgsbase tsc_adjust bmi1 hle 
                                 avx2 smep bmi2 erms invpcid rtm mpx 
                                 avx512f avx512dq rdseed adx smap clf
                                 lushopt clwb avx512cd avx512bw avx51
                                 2vl xsaveopt xsavec xgetbv1 arat avx
                                 512_vnni md_clear flush_l1d arch_cap
                                 abilities

  • Commit id (e.g. a3ffc7d8)
version v3.2.1
nebula-graphd version Git: bb2e684

How To Reproduce(required)

Steps to reproduce the behavior:

  1. Step 1
  2. Step 2
  3. Step 3

Expected behavior

Additional context

@flymysql flymysql added the type/bug Type: something is unexpected label Oct 30, 2023
@github-actions github-actions bot added affects/none PR/issue: this bug affects none version. severity/none Severity of bug labels Oct 30, 2023
@flymysql flymysql changed the title Occasional Graphd Node Crash in NebulaGraph V3.2 Nebula Graphd Coredump when time-out Nov 2, 2023
@flymysql flymysql closed this as completed Nov 2, 2023
@github-actions github-actions bot added the process/fixed Process of bug label Nov 2, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
affects/none PR/issue: this bug affects none version. process/fixed Process of bug severity/none Severity of bug type/bug Type: something is unexpected
Projects
None yet
Development

No branches or pull requests

1 participant