SIGSEGV, Segmentation fault when running pipeline multiple times

[Sweehoe]

Env:
DS 6.2
DeepStream-Yolo commit ab6de54

I have a segmentation fault error when running multiple pipelines with YoloV7 model sequentially. Running the app through gdb captured the bt when the fault occurred.

It seems to be related to this reported issue. https://forums.developer.nvidia.com/t/yolo-inference-library-issue/154891/6.
Do you think it is similar issue? Would you know which static variable the post is referring to?

Thread 1 "python3" received signal SIGSEGV, Segmentation fault.
0x00007f976c2d16f4 in ?? () from /usr/lib/x86_64-linux-gnu/libnvinfer.so.8
(gdb) bt
#0 0x00007f976c2d16f4 in () at /usr/lib/x86_64-linux-gnu/libnvinfer.so.8
#1 0x00007f971db5cfdd in nvinfer1::PluginRegistrar::PluginRegistrar() () at /usr/local/lib/libnvdsinfer_custom_impl_Yolo.so
#2 0x00007f971db5c8d3 in __static_initialization_and_destruction_0(int, int) () at /usr/local/lib/libnvdsinfer_custom_impl_Yolo.so
#3 0x00007f971db5c90a in _GLOBAL__sub_I_yoloPlugins.cpp () at /usr/local/lib/libnvdsinfer_custom_impl_Yolo.so
#4 0x00007f9796a36b9a in () at /lib64/ld-linux-x86-64.so.2
#5 0x00007f9796a36ca1 in () at /lib64/ld-linux-x86-64.so.2
#6 0x00007f9796980985 in _dl_catch_exception () at /usr/lib/x86_64-linux-gnu/libc.so.6
#7 0x00007f9796a3b0cf in () at /lib64/ld-linux-x86-64.so.2
#8 0x00007f9796980928 in _dl_catch_exception () at /usr/lib/x86_64-linux-gnu/libc.so.6
#9 0x00007f9796a3a60a in () at /lib64/ld-linux-x86-64.so.2
#10 0x00007f97967f834c in () at /usr/lib/x86_64-linux-gnu/libdl.so.2
#11 0x00007f9796980928 in _dl_catch_exception () at /usr/lib/x86_64-linux-gnu/libc.so.6
#12 0x00007f97969809f3 in _dl_catch_error () at /usr/lib/x86_64-linux-gnu/libc.so.6
#13 0x00007f97967f8b59 in () at /usr/lib/x86_64-linux-gnu/libdl.so.2
#14 0x00007f97967f83da in dlopen () at /usr/lib/x86_64-linux-gnu/libdl.so.2
#15 0x00007f978be94000 in nvdsinfer::DlLibHandle::DlLibHandle(std::__cxx11::basic_string<char, std::char_traits, std::allocator > const&, int)
(this=0x2519c550, path="/usr/local/lib/libnvdsinfer_custom_impl_Yolo.so", mode=1) at nvdsinfer_func_utils.cpp:27
#16 0x00007f978be60789 in std::make_unique<nvdsinfer::DlLibHandle, char (&) [4096], int>(char (&) [4096], int&&) () at /usr/include/c++/9/bits/unique_ptr.h:857
#17 0x00007f978be506aa in nvdsinfer::NvDsInferContextImpl::initialize(_NvDsInferContextInitParams&, void*, void ()(INvDsInferContext, unsigned int, NvDsInferLogLevel, char const*, void*)) (this=0x196ab2d0, initParams=..., userCtx=0x196fcaa0, logFunc=0x7f9793ef7950 <gst_nvinfer_logger>) at nvdsinfer_context_impl.cpp:1254

[marcoslucianops]

How your pipeline works? I tried to set the pipeline state to NULL in the EOS and re-init it sending PLAYING state and it worked. I think a better practice is changing the source in the EOS instead to re-init or change pipelines.

[Sweehoe]

My app will tear down the pipeline by setting the pipeline state set to NULL and call Gst.parse_launch to create a new pipeline again. As the number of video sources is dynamic, I will need to add/remove the video source to the streammux dynamically, so recreating the pipeline seems to be the easier path.

[marcoslucianops]

Take a look on: https://github.com/NVIDIA-AI-IOT/deepstream_python_apps/tree/master/apps/runtime_source_add_delete

[Sweehoe]

Thank you for the link