Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

SIGSEGV in Vulkan replay of vulkan-smoketest #2307

Open
pmuetschard opened this issue Oct 19, 2018 · 8 comments
Open

SIGSEGV in Vulkan replay of vulkan-smoketest #2307

pmuetschard opened this issue Oct 19, 2018 · 8 comments
Labels
bug GAPIR linux triaged Vulkan

Comments

@pmuetschard
Copy link
Member

Capture the vulkan-smoketest application from the sdk with

gapit trace --capture-frames 3 -api vulkan -workingdir . -out smoketest.gfxtrace /usr/bin/vulkan-smoketest

Run a replay of the first frame with

gapit screenshot --frame 1 smoketest.gfxtrace

Observe that the replay crashes. Oddly though, gapit video succeeds.

Below is the stack trace on my machine:

Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0x7f8ec6ffd700 (LWP 104709)]
0x00007f8ec55ea508 in ?? () from /usr/lib/x86_64-linux-gnu/libnvidia-glcore.so.384.111
(gdb) bt
#0  0x00007f8ec55ea508 in ?? () from /usr/lib/x86_64-linux-gnu/libnvidia-glcore.so.384.111
#1  0x00007f8ec55fd2b3 in ?? () from /usr/lib/x86_64-linux-gnu/libnvidia-glcore.so.384.111
#2  0x0000562e66ec20bd in gapir::LazyResolved<void (*)(unsigned long, unsigned int, gapir::Vulkan::VkWriteDescriptorSet const*, unsigned int, gapir::Vulkan::VkCopyDescriptorSet const*)>::operator()<unsigned long&, unsigned int&, gapir::Vulkan::VkWriteDescriptorSet const*&, unsigned int&, gapir::Vulkan::VkCopyDescriptorSet const*&> (this=0x7f8ea03123a0, args#0=@0x7f8ec6ffc898: 140250550242272, args#1=@0x7f8ec6ffc884: 2, args#2=@0x7f8ec6ffc890: 0x7f8f8df1f9b0, args#3=@0x7f8ec6ffc880: 0, args#4=@0x7f8ec6ffc888: 0x0) at ./gapir/cc/gfx_api.h:58
#3  0x0000562e66ea66bd in gapir::Vulkan::callVkUpdateDescriptorSets (this=0x7f8ea00008d0, cmdLabel=403, stack=0x7f8ec1a391a0, pushReturn=false) at bazel-out/k8-dbg/genfiles/gapir/cc/vulkan_gfx_api.cpp:1159
#4  0x0000562e66ecf770 in std::__invoke_impl<bool, bool (gapir::Vulkan::*&)(unsigned int, gapir::Stack*, bool), gapir::Vulkan*&, unsigned int, gapir::Stack*, bool>(std::__invoke_memfun_deref, bool (gapir::Vulkan::*&)(unsigned int, gapir::Stack*, bool), gapir::Vulkan*&, unsigned int&&, gapir::Stack*&&, bool&&) (__f=@0x7f8ea0001890: (bool (gapir::Vulkan::*)(gapir::Vulkan * const, unsigned int, gapir::Stack *, bool)) 0x562e66ea6522 <gapir::Vulkan::callVkUpdateDescriptorSets(unsigned int, gapir::Stack*, bool)>, __t=@0x7f8ea00018a0: 0x7f8ea00008d0, 
    __args#0=<unknown type in bin/pkg/gapir, CU 0x3876b1, DIE 0x3bb8cd>, 
    __args#1=<unknown type in bin/pkg/gapir, CU 0x3876b1, DIE 0x3bb8dc>, 
    __args#2=<unknown type in bin/pkg/gapir, CU 0x3876b1, DIE 0x3bb8eb>) at /usr/include/c++/7/bits/invoke.h:73
#5  0x0000562e66ecf691 in std::__invoke<bool (gapir::Vulkan::*&)(unsigned int, gapir::Stack*, bool), gapir::Vulkan*&, unsigned int, gapir::Stack*, bool>(bool (gapir::Vulkan::*&)(unsigned int, gapir::Stack*, bool), gapir::Vulkan*&, unsigned int&&, gapir::Stack*&&, bool&&) (__fn=
    @0x7f8ea0001890: (bool (gapir::Vulkan::*)(gapir::Vulkan * const, unsigned int, gapir::Stack *, bool)) 0x562e66ea6522 <gapir::Vulkan::callVkUpdateDescriptorSets(unsigned int, gapir::Stack*, bool)>, __args#0=@0x7f8ea00018a0: 0x7f8ea00008d0, 
    __args#1=<unknown type in bin/pkg/gapir, CU 0x3876b1, DIE 0x3bd259>, 
    __args#2=<unknown type in bin/pkg/gapir, CU 0x3876b1, DIE 0x3bd268>, 
    __args#3=<unknown type in bin/pkg/gapir, CU 0x3876b1, DIE 0x3bd277>) at /usr/include/c++/7/bits/invoke.h:96
#6  0x0000562e66ecf566 in std::_Bind<bool (gapir::Vulkan::*(gapir::Vulkan*, std::_Placeholder<1>, std::_Placeholder<2>, std::_Placeholder<3>))(unsigned int, gapir::Stack*, bool)>::__call<bool, unsigned int&&, gapir::Stack*&&, bool&&, 0ul, 1ul, 2ul, 3ul>(std::tuple<unsigned int&&, gapir::Stack*&&, bool&&>&&, std::_Index_tuple<0ul, 1ul, 2ul, 3ul>) (this=0x7f8ea0001890, __args=<unknown type in bin/pkg/gapir, CU 0x3876b1, DIE 0x3bf081>)
    at /usr/include/c++/7/functional:469
#7  0x0000562e66ecf349 in std::_Bind<bool (gapir::Vulkan::*(gapir::Vulkan*, std::_Placeholder<1>, std::_Placeholder<2>, std::_Placeholder<3>))(unsigned int, gapir::Stack*, bool)>::operator()<unsigned int, gapir::Stack*, bool, bool>(unsigned int&&, gapir::Stack*&&, bool&&) (
    this=0x7f8ea0001890, __args#0=<unknown type in bin/pkg/gapir, CU 0x3876b1, DIE 0x3c0b9b>, 
    __args#1=<unknown type in bin/pkg/gapir, CU 0x3876b1, DIE 0x3c0bab>, 
    __args#2=<unknown type in bin/pkg/gapir, CU 0x3876b1, DIE 0x3c0bbb>) at /usr/include/c++/7/functional:551
#8  0x0000562e66ecd1d9 in std::_Function_handler<bool (unsigned int, gapir::Stack*, bool), std::_Bind<bool (gapir::Vulkan::*(gapir::Vulkan*, std::_Placeholder<1>, std::_Placeholder<2>, std::_Placeholder<3>))(unsigned int, gapir::Stack*, bool)> >::_M_invoke(std::_Any_data const&, unsigned int&&, gapir::Stack*&&, bool&&) (__functor=..., __args#0=<unknown type in bin/pkg/gapir, CU 0x3876b1, DIE 0x3c59ce>, 
    __args#1=<unknown type in bin/pkg/gapir, CU 0x3876b1, DIE 0x3c59dd>, 
    __args#2=<unknown type in bin/pkg/gapir, CU 0x3876b1, DIE 0x3c59ed>) at /usr/include/c++/7/bits/std_function.h:302
#9  0x0000562e66e53c96 in std::function<bool (unsigned int, gapir::Stack*, bool)>::operator()(unsigned int, gapir::Stack*, bool) const (this=0x7f8ea0001860, __args#0=403, __args#1=0x7f8ec1a391a0, __args#2=false) at /usr/include/c++/7/bits/std_function.h:706
#10 0x0000562e66e503f4 in gapir::Interpreter::call (this=0x7f8ec1a39100, opcode=131105) at gapir/cc/interpreter.cpp:174
#11 0x0000562e66e5153e in gapir::Interpreter::interpret (this=0x7f8ec1a39100, opcode=131105) at gapir/cc/interpreter.cpp:474
#12 0x0000562e66e4ffea in gapir::Interpreter::exec (this=0x7f8ec1a39100) at gapir/cc/interpreter.cpp:115
#13 0x0000562e66e4ff81 in gapir::Interpreter::<lambda()>::operator()(void) const (__closure=0x7f8ec6ffcca0) at gapir/cc/interpreter.cpp:129
#14 0x0000562e66e5255b in std::_Function_handler<void(), gapir::Interpreter::exec()::<lambda()> >::_M_invoke(const std::_Any_data &) (__functor=...) at /usr/include/c++/7/bits/std_function.h:316
#15 0x0000562e66e3f648 in std::function<void ()>::operator()() const (this=0x7f8ec6ffcca0) at /usr/include/c++/7/bits/std_function.h:706
#16 0x0000562e66e3edd8 in gapir::ThreadPool::Thread::worker (thread=0x7f8ec0ed5b70) at gapir/cc/thread_pool.cpp:56
#17 0x0000562e66e400b4 in std::__invoke_impl<void, void (*)(gapir::ThreadPool::Thread*), gapir::ThreadPool::Thread*>(std::__invoke_other, void (*&&)(gapir::ThreadPool::Thread*), gapir::ThreadPool::Thread*&&) (
    __f=<unknown type in bin/pkg/gapir, CU 0x154adc, DIE 0x16499b>, 
    __args#0=<unknown type in pkg/gapir, CU 0x154adc, DIE 0x1649b6>) at /usr/include/c++/7/bits/invoke.h:60
#18 0x0000562e66e3f73e in std::__invoke<void (*)(gapir::ThreadPool::Thread*), gapir::ThreadPool::Thread*>(void (*&&)(gapir::ThreadPool::Thread*), gapir::ThreadPool::Thread*&&) (
    __fn=<unknown type in bin/pkg/gapir, CU 0x154adc, DIE 0x16564e>, 
    __args#0=<unknown type in bin/pkg/gapir, CU 0x154adc, DIE 0x165669>) at /usr/include/c++/7/bits/invoke.h:95
#19 0x0000562e66e42a39 in std::thread::_Invoker<std::tuple<void (*)(gapir::ThreadPool::Thread*), gapir::ThreadPool::Thread*> >::_M_invoke<0ul, 1ul> (this=0x7f8ec02dfba8) at /usr/include/c++/7/thread:234
#20 0x0000562e66e429ef in std::thread::_Invoker<std::tuple<void (*)(gapir::ThreadPool::Thread*), gapir::ThreadPool::Thread*> >::operator() (this=0x7f8ec02dfba8) at /usr/include/c++/7/thread:243
#21 0x0000562e66e429ce in std::thread::_State_impl<std::thread::_Invoker<std::tuple<void (*)(gapir::ThreadPool::Thread*), gapir::ThreadPool::Thread*> > >::_M_run (this=0x7f8ec02dfba0) at /usr/include/c++/7/thread:186
#22 0x00007f8f8e7097e3 in ?? () from /usr/lib/x86_64-linux-gnu/libstdc++.so.6
#23 0x00007f8f8ecdd494 in start_thread (arg=0x7f8ec6ffd700) at pthread_create.c:333
#24 0x00007f8f8e17ea8f in clone () from /lib/x86_64-linux-gnu/libc.so.6
@Qining
Copy link
Contributor

Qining commented Oct 24, 2018

Confirmed, this is a bug in the current DCE.

@Qining Qining self-assigned this Oct 24, 2018
Qining added a commit to Qining/gapid that referenced this issue Oct 24, 2018
@Qining
Fix vulkan-smoketest replay
8044b9b 
Fix issue google#2307
@Qining Qining mentioned this issue Oct 24, 2018
Merged
Qining added a commit that referenced this issue Oct 24, 2018
@Qining
Fix vulkan-smoketest replay (#2313)
0f7c31b 
Fix issue #2307
@pmuetschard
Copy link
Member Author

This appears to not be fixed quite yet. Here is another stack trace form a different machine:

#0  0x00007faeb48d2bbe in xcb_send_request_with_fds64 () from /usr/lib/x86_64-linux-gnu/libxcb.so.1
#1  0x00007faeb48d3129 in xcb_send_request () from /usr/lib/x86_64-linux-gnu/libxcb.so.1
#2  0x00007faeb48e15bf in xcb_query_extension () from /usr/lib/x86_64-linux-gnu/libxcb.so.1
#3  0x00007fae9b049a40 in ?? () from /usr/lib/x86_64-linux-gnu/libvulkan_intel.so
#4  0x00007fae9b04b0b3 in ?? () from /usr/lib/x86_64-linux-gnu/libvulkan_intel.so
#5  0x00007fae9af23fad in ?? () from /usr/lib/x86_64-linux-gnu/libvulkan_intel.so
#6  0x000055d774df174d in gapir::LazyResolved<unsigned int (*)(unsigned long, unsigned int, xcb_connection_t*, unsigned int)>::operator()<unsigned long&, unsigned int&, xcb_connection_t*&, unsigned int&> (this=0x7fae94398c48, args#0=@0x7faeb669b878: 140387787323424, args#1=@0x7faeb669b868: 0, 
    args#2=@0x7faeb669b870: 0xbadf00d, args#3=@0x7faeb669b864: 97) at ./gapir/cc/gfx_api.h:58
#7  0x000055d774de2d14 in gapir::Vulkan::callVkGetPhysicalDeviceXcbPresentationSupportKHR (this=0x7fae940008d0, cmdLabel=63, stack=0x7faea89d1760, 
    pushReturn=false) at bazel-out/k8-dbg/genfiles/gapir/cc/vulkan_gfx_api.cpp:5025
#8  0x000055d774df4774 in std::__invoke_impl<bool, bool (gapir::Vulkan::*&)(unsigned int, gapir::Stack*, bool), gapir::Vulkan*&, unsigned int, gapir::Stack*, bool>(std::__invoke_memfun_deref, bool (gapir::Vulkan::*&)(unsigned int, gapir::Stack*, bool), gapir::Vulkan*&, unsigned int&&, gapir::Stack*&&, bool&&) (__f=
    @0x7fae94005b00: (bool (gapir::Vulkan::*)(gapir::Vulkan * const, unsigned int, gapir::Stack *, bool)) 0x55d774de2afe <gapir::Vulkan::callVkGetPhysicalDeviceXcbPresentationSupportKHR(unsigned int, gapir::Stack*, bool)>, __t=@0x7fae94005b10: 0x7fae940008d0, 
    __args#0=<unknown type in bin/pkg/gapir, CU 0x3876bc, DIE 0x3bb8d8>, 
    __args#1=<unknown type in bin/pkg/gapir, CU 0x3876bc, DIE 0x3bb8e7>, 
    __args#2=<unknown type in bin/pkg/gapir, CU 0x3876bc, DIE 0x3bb8f6>) at /usr/include/c++/7/bits/invoke.h:73
#9  0x000055d774df4695 in std::__invoke<bool (gapir::Vulkan::*&)(unsigned int, gapir::Stack*, bool), gapir::Vulkan*&, unsigned int, gapir::Stack*, bool>(bool (gapir::Vulkan::*&)(unsigned int, gapir::Stack*, bool), gapir::Vulkan*&, unsigned int&&, gapir::Stack*&&, bool&&) (__fn=
    @0x7fae94005b00: (bool (gapir::Vulkan::*)(gapir::Vulkan * const, unsigned int, gapir::Stack *, bool)) 0x55d774de2afe <gapir::Vulkan::callVkGetPhysicalDeviceXcbPresentationSupportKHR(unsigned int, gapir::Stack*, bool)>, __args#0=@0x7fae94005b10: 0x7fae940008d0, 
    __args#1=<unknown type in bin/pkg/gapir, CU 0x3876bc, DIE 0x3bd264>, 
    __args#2=<unknown type in bin/pkg/gapir, CU 0x3876bc, DIE 0x3bd273>, 
    __args#3=<unknown type in bin/pkg/gapir, CU 0x3876bc, DIE 0x3bd282>) at /usr/include/c++/7/bits/invoke.h:96
#10 0x000055d774df456a in std::_Bind<bool (gapir::Vulkan::*(gapir::Vulkan*, std::_Placeholder<1>, std::_Placeholder<2>, std::_Placeholder<3>))(unsigned int, gapir::Stack*, bool)>::__call<bool, unsigned int&&, gapir::Stack*&&, bool&&, 0ul, 1ul, 2ul, 3ul>(std::tuple<unsigned int&&, gapir::Stack*&&, bool&&>&&, std::_Index_tuple<0ul, 1ul, 2ul, 3ul>) (this=0x7fae94005b00, 
    __args=<unknown type in bin/pkg/gapir, CU 0x3876bc, DIE 0x3bf08c>)
    at /usr/include/c++/7/functional:469
#11 0x000055d774df434d in std::_Bind<bool (gapir::Vulkan::*(gapir::Vulkan*, std::_Placeholder<1>, std::_Placeholder<2>, std::_Placeholder<3>))(unsigned int, gapir::Stack*, bool)>::operator()<unsigned int, gapir::Stack*, bool, bool>(unsigned int&&, gapir::Stack*&&, bool&&) (this=0x7fae94005b00, 
    __args#0=<unknown type in bin/pkg/gapir, CU 0x3876bc, DIE 0x3c0ba6>, 
    __args#1=<unknown type in bin/pkg/gapir, CU 0x3876bc, DIE 0x3c0bb6>, 
    __args#2=<unknown type in bin/pkg/gapir, CU 0x3876bc, DIE 0x3c0bc6>)
    at /usr/include/c++/7/functional:551
#12 0x000055d774df21dd in std::_Function_handler<bool (unsigned int, gapir::Stack*, bool), std::_Bind<bool (gapir::Vulkan::*(gapir::Vulkan*, std::_Placeholder<1>, std::_Placeholder<2>, std::_Placeholder<3>))(unsigned int, gapir::Stack*, bool)> >::_M_invoke(std::_Any_data const&, unsigned int&&, gapir::Stack*&&, bool&&) (__functor=..., 
    __args#0=<unknown type in bin/pkg/gapir, CU 0x3876bc, DIE 0x3c59d9>, 
    __args#1=<unknown type in bin/pkg/gapir, CU 0x3876bc, DIE 0x3c59e8>, 
    __args#2=<unknown type in bin/pkg/gapir, CU 0x3876bc, DIE 0x3c59f8>)
    at /usr/include/c++/7/bits/std_function.h:302
#13 0x000055d774d78c9a in std::function<bool (unsigned int, gapir::Stack*, bool)>::operator()(unsigned int, gapir::Stack*, bool) const (this=0x7fae94005ad0, __args#0=63, __args#1=0x7faea89d1760, 
    __args#2=false) at /usr/include/c++/7/bits/std_function.h:706
#14 0x000055d774d753f8 in gapir::Interpreter::call (this=0x7faea89d16c0, opcode=131258) at gapir/cc/interpreter.cpp:174
#15 0x000055d774d76542 in gapir::Interpreter::interpret (this=0x7faea89d16c0, opcode=131258) at gapir/cc/interpreter.cpp:474
#16 0x000055d774d74fee in gapir::Interpreter::exec (this=0x7faea89d16c0) at gapir/cc/interpreter.cpp:115
#17 0x000055d774d74f85 in gapir::Interpreter::<lambda()>::operator()(void) const (__closure=0x7faeb669bca0) at gapir/cc/interpreter.cpp:129
#18 0x000055d774d7755f in std::_Function_handler<void(), gapir::Interpreter::exec()::<lambda()> >::_M_invoke(const std::_Any_data &) (__functor=...) at /usr/include/c++/7/bits/std_function.h:316
#19 0x000055d774d6464c in std::function<void ()>::operator()() const (this=0x7faeb669bca0) at /usr/include/c++/7/bits/std_function.h:706
#20 0x000055d774d63ddc in gapir::ThreadPool::Thread::worker (thread=0x7faea89f9660) at gapir/cc/thread_pool.cpp:56
#21 0x000055d774d650b8 in std::__invoke_impl<void, void (*)(gapir::ThreadPool::Thread*), gapir::ThreadPool::Thread*>(std::__invoke_other, void (*&&)(gapir::ThreadPool::Thread*), gapir::ThreadPool::Thread*&&) (__f=<unknown type in bin/pkg/gapir, CU 0x154ae7, DIE 0x1649a6>, 
    __args#0=<unknown type in bin/pkg/gapir, CU 0x154ae7, DIE 0x1649c1>)
    at /usr/include/c++/7/bits/invoke.h:60
#22 0x000055d774d64742 in std::__invoke<void (*)(gapir::ThreadPool::Thread*), gapir::ThreadPool::Thread*>(void (*&&)(gapir::ThreadPool::Thread*), gapir::ThreadPool::Thread*&&) (
    __fn=<unknown type in bin/pkg/gapir, CU 0x154ae7, DIE 0x165659>, 
    __args#0=<unknown type in bin/pkg/gapir, CU 0x154ae7, DIE 0x165674>)
    at /usr/include/c++/7/bits/invoke.h:95
#23 0x000055d774d67a3d in std::thread::_Invoker<std::tuple<void (*)(gapir::ThreadPool::Thread*), gapir::ThreadPool::Thread*> >::_M_invoke<0ul, 1ul> (this=0x7faea83d03c8)
    at /usr/include/c++/7/thread:234
#24 0x000055d774d679f3 in std::thread::_Invoker<std::tuple<void (*)(gapir::ThreadPool::Thread*), gapir::ThreadPool::Thread*> >::operator() (this=0x7faea83d03c8) at /usr/include/c++/7/thread:243
#25 0x000055d774d679d2 in std::thread::_State_impl<std::thread::_Invoker<std::tuple<void (*)(gapir::ThreadPool::Thread*), gapir::ThreadPool::Thread*> > >::_M_run (this=0x7faea83d03c0)
    at /usr/include/c++/7/thread:186
#26 0x00007faf7a57e7e3 in ?? () from /usr/lib/x86_64-linux-gnu/libstdc++.so.6
#27 0x00007faf7ab52494 in start_thread (arg=0x7faeb669c700) at pthread_create.c:333
#28 0x00007faf79ff3a8f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:97

@Qining
Copy link
Contributor

Qining commented Oct 25, 2018

@pmuetschard Is this also triggered by gapit screenshot --frame 1 smoketest.gfxtrace? And it is not using a trace generated on another machine right?

@Qining
Copy link
Contributor

Qining commented Oct 25, 2018

@pmuetschard, by generated on another machine, I mean, the trace is captured and replayed on the same machine right?

@pmuetschard
Copy link
Member Author

The trace was taken with:
../bazel-bin/pkg/gapit trace -capture-frames 3 -api vulkan /usr/bin/vulkan-smoketest
The crash happens with
../bazel-bin/pkg/gapit screenshot vulkan-smoketest.gfxtrace

The trace and replay were done on the same machine. The crash also happens if using -noopt

@Qining
Copy link
Contributor

Qining commented Oct 25, 2018

Odd, I can not reproduce it on my machine. Let me try it later on my home desktop which uses an Intel driver.

@pmuetschard
Copy link
Member Author

Note the above crash also happens with DCE off.

@AWoloszyn
Copy link
Contributor

@pmuetschard Does this still happen with latest?

@Qining Qining assigned AWoloszyn and unassigned Qining Oct 24, 2019
@AWoloszyn AWoloszyn removed their assignment May 23, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug GAPIR linux triaged Vulkan
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@Qining @AWoloszyn @pmuetschard