i#5926 libdw: Switch drsyms to use libdw from elfutils #6534
Conversation
Adds a new API routine dr_running_under_dynamorio() which is similar to the application interface dr_app_running_under_dynamorio() but is callable from a client. The use case is code meant for us both in managed mode under DR and in standalone library mode. Sometimes the behavior for the two differs significantly, such as whether certain library routines are safe when linked statically. Adds sanity checks to some existing tests. The driving use case is in drsyms's support for libdw. The actual use of this will come in that libdw commit. Issue: #5926
Switches from elftoolchain's libdwarf, which is no longer being
updated, to elfutil's libdw, only for Linux for now. The interfaces
are slightly different but generally similar. We also switch to the
libelf from elfutil to ensure its interactions with libdw work
properly.
Adds a new third_party/elfutils submodule.
Adds the LGPL3 elfutils license to our license docs, stressing that we
are providing drsyms as a library distinct from the rest of DR.
Adds build rules to build the 4 elfutils libraries we need (libdw,
libelf, plus helpers libdwelf and libebl) as static PIC libraries.
As elfutils uses autoconf, we check in a config.h file from a local
configure believed to be general enough for our supported toolchains
to feed to our CMake build.
Adds 3 patches needed for elfutils. These are stored in the parent
dynamorio repository and applied at CMake configure time to create new
source files in the build directory. This keeps the submodule
pristine. The patches are:
1) Fix a bug in elf_memory() where an ignored "command" enum was used.
2) Shrink the stack usage of read_srclines() called by dwarf_getsrclines() to
fit on DR's small stacks.
3) Redirect zlib heap allocation.
Adds a requirement that zlib be available on Linux to build drysms, as
elfutils requires zlib for compressed debug sections.
Adds -Dmalloc=__wrap_malloc, etc. to redirect heap usage by the
elfutils static libraries to DR.
Adds a new tool.drcacheoff.check-malloc test to better check for
malloc use in the drmemtrace client. The new test explicitly invokes
drsyms via the drmemtrace function tracing feature, while disabling
lz4 compression (as it turns off the import check). This test only
passes with the -Dmalloc=__wrap_malloc defines and the zlib heap
redirect patch.
Adds a new drsyms_dw.c file, partly mirroring drsyms_dwarf.c. The
libdw and libdwarf interfaces are different enough it seemed best to
have a separate file, though the general code is similar. The
drsyms_elf.c differences are in-file via ifdef.
Removes the -gdwarf-4 workaround from the docs.
Adds new -gdwarf-4 and -gdwarf-5 drsyms tests.
The libelftc demangler is kept via the checked-in static library,
since libdw does not have its own demangler. We do not expect this
part of libelftc to need updates.
Fixes #5926
…do so for Android (once it works will copy to ci-package.yml)
|
Getting zlib installed for 32-bit Android is the blocker here. I will spend a little time on it...but Android is not really a high priority (someone who is interested in reviving Android full support should speak up...); at some point I would just drop drsyms support on Android, or more likely go back to libelftc on Android and not support dwarf-5 there. |
…tead of libdw for Android
|
drcache{sim,off}.simple 32-bit ubuntu22 are failing every time, but I suspect it is unrelated to libdw and was instead triggered by installing libz for our 32-bit testing which was not done before. I will see if I can repro. |
|
The unresolved failures are all an assert |
…ULL ref hit on privlib init SIGSEGV, to try to fix synch.c asserts
|
The 32-bit AMD GA CI also has a bunch of those synch.c asserts. But I can't repro any of them on local AMD machines. Still think it's related to the particular zlib installed here, which is done in a different way than locally b/c of the different repo on GA CI. Plus, when I put in the try-except, the failures change from the synch.c to reported DR crashes. I am separating out the try-except under #6541 as it seems a separate useful change. |
|
It is indeed from the 32-bit zlib now installed. DR's private loader crashes trying to relocate a zlib symbol "deflateResetKeep" in drsyms.so (presumably zlib is statically linked into drsyms here): |
|
The difference seems to be that on my local machine libz is not statically linked in: |
|
I should file a separate issue on zlib as this would be hit without libdw since we want to use zlib with drmemtrace, we just never tried to install it on these GA CI VMs. The relocation itself looks ok; the issue seems to be that it's relocating .text which is mapped as +rx and is not writable. I'm trying to understand how other text relocation ever worked...could it be that regular PIC libraries never have text relocs? There are security policies banning text relocs. So fundamentally the problem is not really in our loader if we only support PIC, it's that the static libz.a chosen here is not PIC? Or would ld.so go mark .txt +w temporarily? |
|
That seems to have resolved everything: the only failures left look like the #6417 32-bit ones (before that job had extra failures; now it is back to the baseline). |
|
Filed #6543 as Won't Fix to document the text relocs from non-PIC .a |
Switches from elftoolchain's libdwarf, which is no longer being
updated, to elfutil's libdw, only for Linux for now. The interfaces
are slightly different but generally similar. We also switch to the
libelf from elfutil to ensure its interactions with libdw work
properly.
Adds a new third_party/elfutils submodule.
Adds the LGPL3 elfutils license to our license docs, stressing that we
are providing drsyms as a library distinct from the rest of DR.
Adds build rules to build the 4 elfutils libraries we need (libdw,
libelf, plus helpers libdwelf and libebl) as static PIC libraries.
As elfutils uses autoconf, we check in a config.h file from a local
configure believed to be general enough for our supported toolchains
to feed to our CMake build.
Adds 4 patches needed for elfutils. These are stored in the parent
dynamorio repository and applied at CMake configure time to create new
source files in the build directory. This keeps the submodule
pristine. The patches are:
fit on DR's small stacks.
Adds a requirement that zlib be available on Linux to build drysms, as
elfutils requires zlib for compressed debug sections.
Adds -Dmalloc=__wrap_malloc, etc. to redirect heap usage by the
elfutils static libraries to DR.
Adds a new tool.drcacheoff.check-malloc test to better check for
malloc use in the drmemtrace client. The new test explicitly invokes
drsyms via the drmemtrace function tracing feature, while disabling
lz4 compression (as it turns off the import check). This test only
passes with the -Dmalloc=__wrap_malloc defines and the zlib heap
redirect patch.
Adds a new drsyms_dw.c file, partly mirroring drsyms_dwarf.c. The
libdw and libdwarf interfaces are different enough it seemed best to
have a separate file, though the general code is similar. The
drsyms_elf.c differences are in-file via ifdef.
Removes the -gdwarf-4 workaround from the docs.
Adds new -gdwarf-4 and -gdwarf-5 drsyms tests.
The libelftc demangler is kept via the checked-in static library,
since libdw does not have its own demangler. We do not expect this
part of libelftc to need updates.
Fixes #5926