SEGFAULT with auditwheel + patchelf #474
Comments
|
Would it be possible to attach de libstdc++ before and after patching? And also the patchelf command that was used? That would help a lot in the investigation. |
No problem. I'll make this |
|
Here is readelf logs Original and patched libs
I dont know, bacuse this commands make auditwheel python package |
|
Looks like the alignment of the existing LOAD segments is |
|
@brenoguim, this is already a good step towards solving the problem. |
|
Yes sure |
bash-5.1# pip install --force-reinstall cchardet-2.1.7-cp310-cp310-musllinux_1_2_armv7l.whl
Processing ./cchardet-2.1.7-cp310-cp310-musllinux_1_2_armv7l.whl
Installing collected packages: cchardet
Attempting uninstall: cchardet
Found existing installation: cchardet 2.1.7
Uninstalling cchardet-2.1.7:
Successfully uninstalled cchardet-2.1.7
Successfully installed cchardet-2.1.7bash-5.1# python -c "import cchardet"
Segmentation fault
bash-5.1# ldd /usr/local/lib/python3.10/site-packages/cchardet.libs/libstdc++-ad7c573d.so.6.0.29
/lib/ld-musl-armhf.so.1 (0x542d0000)
Error loading shared library : Invalid argument (needed by /usr/local/lib/python3.10/site-packages/cchardet.libs/libstdc++-ad7c573d.so.6.0.29)
Error loading shared library error_code: No such file or directory (needed by /usr/local/lib/python3.10/site-packages/cchardet.libs/libstdc++-ad7c573d.so.6.0.29)
Segmentation fault
bash-5.1# strace ldd /usr/local/lib/python3.10/site-packages/cchardet.libs/libstdc++-ad7c573d.so.6.0.29 bash-5.1# strace ldd /usr/local/lib/python3.10/site-packages/cchardet.libs/libstdc++-ad7c573d.so.6.0.29
execve("/usr/bin/ldd", ["ldd", "/usr/local/lib/python3.10/site-p"...], 0x7df67cf4 /* 16 vars */) = 0
set_tls(0x75a9a400) = 0
set_tid_address(0x75a997cc) = 281
brk(NULL) = 0x560e0000
brk(0x560f0000) = 0x560f0000
mmap2(0x560e0000, 32768, PROT_NONE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x560e0000
getuid32() = 0
getpid() = 281
rt_sigprocmask(SIG_UNBLOCK, [RT_1 RT_2], NULL, 8) = 0
rt_sigaction(SIGCHLD, {sa_handler=0x54327c11, sa_mask=~[RTMIN RT_1 RT_2], sa_flags=SA_RESTORER, sa_restorer=0x75a501fb}, NULL, 8) = 0
getppid() = 278
statx(AT_FDCWD, "/config", AT_STATX_SYNC_AS_STAT, STATX_BASIC_STATS, 0x7dc57698) = -1 ENOSYS (Function not implemented)
stat64("/config", {st_mode=S_IFDIR|0777, st_size=4096, ...}) = 0
statx(AT_FDCWD, ".", AT_STATX_SYNC_AS_STAT, STATX_BASIC_STATS, 0x7dc57698) = -1 ENOSYS (Function not implemented)
stat64(".", {st_mode=S_IFDIR|0777, st_size=4096, ...}) = 0
open("/usr/bin/ldd", O_RDONLY|O_LARGEFILE|O_CLOEXEC) = 3
fcntl64(3, F_SETFD, FD_CLOEXEC) = 0
fcntl64(3, F_DUPFD_CLOEXEC, 10) = 10
fcntl64(10, F_SETFD, FD_CLOEXEC) = 0
close(3) = 0
rt_sigaction(SIGINT, NULL, {sa_handler=SIG_DFL, sa_mask=[], sa_flags=0}, 8) = 0
rt_sigaction(SIGINT, {sa_handler=0x54327c11, sa_mask=~[RTMIN RT_1 RT_2], sa_flags=SA_RESTORER, sa_restorer=0x75a501fb}, NULL, 8) = 0
rt_sigaction(SIGQUIT, NULL, {sa_handler=SIG_DFL, sa_mask=[], sa_flags=0}, 8) = 0
rt_sigaction(SIGQUIT, {sa_handler=SIG_IGN, sa_mask=~[RTMIN RT_1 RT_2], sa_flags=SA_RESTORER, sa_restorer=0x75a501fb}, NULL, 8) = 0
rt_sigaction(SIGTERM, NULL, {sa_handler=SIG_DFL, sa_mask=[], sa_flags=0}, 8) = 0
read(10, "#!/bin/sh\nexec /lib/ld-musl-armh"..., 2047) = 51
rt_sigaction(SIGQUIT, {sa_handler=SIG_DFL, sa_mask=~[RTMIN RT_1 RT_2], sa_flags=SA_RESTORER, sa_restorer=0x75a501fb}, NULL, 8) = 0
execve("/lib/ld-musl-armhf.so.1", ["/lib/ld-musl-armhf.so.1", "--list", "/usr/local/lib/python3.10/site-p"...], 0x75a98414 /* 16 vars */) = 0
set_tls(0x54802400) = 0
set_tid_address(0x548017cc) = 281
open("/usr/local/lib/python3.10/site-packages/cchardet.libs/libstdc++-ad7c573d.so.6.0.29", O_RDONLY|O_LARGEFILE) = 3
read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0(\0\1\0\0\0\0\0\0\0004\0\0\0"..., 936) = 936
mmap2(NULL, 2228224, PROT_READ|PROT_EXEC, MAP_PRIVATE, 3, 0) = 0x75a78000
mmap2(0x75bc8000, 65536, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED, 3, 0x140000) = 0x75bc8000
mmap2(0x75bd0000, 458752, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED, 3, 0x148000) = 0x75bd0000
mmap2(0x75c38000, 393216, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED, 3, 0x1a8000) = 0x75c38000
close(3) = 0
writev(1, [{iov_base="\t/lib/ld-musl-armhf.so.1 (0x5477"..., iov_len=38}, {iov_base=NULL, iov_len=0}], 2 /lib/ld-musl-armhf.so.1 (0x54778000)
) = 38
brk(NULL) = 0x55000000
brk(0x55010000) = 0x55010000
mmap2(0x55000000, 32768, PROT_NONE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x55000000
writev(2, [{iov_base="Error loading shared library : I"..., iov_len=59}, {iov_base="/usr/local/lib/python3.10/site-p"..., iov_len=82}], 2Error loading shared library : Invalid argument (needed by /usr/local/lib/python3.10/site-packages/cchardet.libs/libstdc++-ad7c573d.so.6.0.29) = 141
writev(2, [{iov_base=")", iov_len=1}, {iov_base=NULL, iov_len=0}], 2)) = 1
writev(2, [{iov_base="\n", iov_len=1}, {iov_base=NULL, iov_len=0}], 2
) = 1
open("/etc/ld-musl-armhf.path", O_RDONLY|O_LARGEFILE|O_CLOEXEC) = -1 ENOENT (No such file or directory)
open("/lib/error_code", O_RDONLY|O_LARGEFILE|O_CLOEXEC) = -1 ENOENT (No such file or directory)
open("/usr/local/lib/error_code", O_RDONLY|O_LARGEFILE|O_CLOEXEC) = -1 ENOENT (No such file or directory)
open("/usr/lib/error_code", O_RDONLY|O_LARGEFILE|O_CLOEXEC) = -1 ENOENT (No such file or directory)
writev(2, [{iov_base="Error loading shared library err"..., iov_len=78}, {iov_base="/usr/local/lib/python3.10/site-p"..., iov_len=82}], 2Error loading shared library error_code: No such file or directory (needed by /usr/local/lib/python3.10/site-packages/cchardet.libs/libstdc++-ad7c573d.so.6.0.29) = 160
writev(2, [{iov_base=")", iov_len=1}, {iov_base=NULL, iov_len=0}], 2)) = 1
writev(2, [{iov_base="\n", iov_len=1}, {iov_base=NULL, iov_len=0}], 2
) = 1
--- SIGSEGV {si_signo=SIGSEGV, si_code=SEGV_MAPERR, si_addr=0x76ba40a4} ---
+++ killed by SIGSEGV +++
Segmentation fault
bash-5.1# |
|
Ouch! I'll take a look at it later on tonight. |
|
Hi @brenoguim, thank you very much for helping ..... It's very kind of you. |
|
Colorfull comparison of differences of readelf log of original and patched files |
|
The new loads (second image) are still using alignment 0x1000 instead. Could you double check you used the patchelf with my fix? |
|
ok no problem. Also colorful logs are first logs not actual. |
|
Also here grep of wheelaudit for patchelf bash-5.1# grep -r "patchelf" /usr/local/lib/python3.10/site-packages/auditwheel/*
grep: /usr/local/lib/python3.10/site-packages/auditwheel/__pycache__/patcher.cpython-310.pyc: binary file matches
/usr/local/lib/python3.10/site-packages/auditwheel/patcher.py:def _verify_patchelf() -> None:
/usr/local/lib/python3.10/site-packages/auditwheel/patcher.py: """This function looks for the ``patchelf`` external binary in the PATH,
/usr/local/lib/python3.10/site-packages/auditwheel/patcher.py: if not find_executable("patchelf"):
/usr/local/lib/python3.10/site-packages/auditwheel/patcher.py: raise ValueError("Cannot find required utility `patchelf` in PATH")
/usr/local/lib/python3.10/site-packages/auditwheel/patcher.py: version = check_output(["patchelf", "--version"]).decode("utf-8")
/usr/local/lib/python3.10/site-packages/auditwheel/patcher.py: raise ValueError("Could not call `patchelf` binary")
/usr/local/lib/python3.10/site-packages/auditwheel/patcher.py: m = re.match(r"patchelf\s+(\d+(.\d+)?)", version)
/usr/local/lib/python3.10/site-packages/auditwheel/patcher.py: f"patchelf {version} found. auditwheel repair requires " "patchelf >= 0.14."
/usr/local/lib/python3.10/site-packages/auditwheel/patcher.py: _verify_patchelf()
/usr/local/lib/python3.10/site-packages/auditwheel/patcher.py: "patchelf",
/usr/local/lib/python3.10/site-packages/auditwheel/patcher.py: check_call(["patchelf", "--set-soname", new_so_name, file_name])
/usr/local/lib/python3.10/site-packages/auditwheel/patcher.py: check_call(["patchelf", "--remove-rpath", file_name])
/usr/local/lib/python3.10/site-packages/auditwheel/patcher.py: check_call(["patchelf", "--force-rpath", "--set-rpath", rpath, file_name])
/usr/local/lib/python3.10/site-packages/auditwheel/patcher.py: check_output(["patchelf", "--print-rpath", file_name])
bash-5.1# |
|
new readelf -aW with patched alignment |
|
Original Section header string table index: 26Patched Section header string table index: 24 <----- but in real there are 26 Is this correct from orignal and patched readelf differnces? (link was changed from 3 to 26) Also in patched there are two new lines And this (link was changed from 3 to 26) |
|
Yes, exceptnfor the alignment, nothing stands out to me. Patchelf introduces new loads segments to map the modified sections into a new place. That's expected. |
Original Section header string table index: 26Patched Section header string table index: 24 <----- but in real there are 26 tooAh sorry i mismatch. this is simple pointer to string index |
|
That happens because Patchelf reorders the sections in the section header table, so it has to update some indexes in some tables. |
No, it's not this one. I thought you would compile it by cloning from https://github.com/brenoguim/patchelf/tree/breno.474 and building patchelf.
Indeed, this does not include the alignment fix. You can build Patchelf by doing: |
|
Ah ok. I'll build it by myself. I thought that there are already compiled binaries. |
|
make in progress |
|
Yes it's work you are my hero.... |
|
Awesome @magicse! I'm glad to help! |
Hm, first I need to get this merged into Patchelf. Then maybe we can generate a new release for users to pick up. |
Many thanks to you . And have nice day (tomorrow) |
You can add the needed patchelf fix into https://github.com/home-assistant/wheels |
Hi @pvizeli thank You. I'm already scared to add something there, I'm constantly scolded for being too loud. |
|
@brenoguim thank you for handling this PR. |
Use the largest segment alignment for libraries requiring non-standard alignments #474
|
Closing as completed after #475 |
This doesn't appear to have been done. Is that correct? How can I add the needed fix? Edit: Do we need to go back to installing patchelf from source? |
Describe the bug
SEGFAULT with auditwheel + patchelf
General problem that wheel libs repaired in Qemu Docker image work in Qemu but didn't work in the same image Docker on real ARMv7, and without wheelaudit+patchelf, built packages work well.
GDB show that problem with memory maping in patched /usr/local/lib/python3.10/site-packages/*.libs/libstdc++-ad7c573d.so.6.0.29 .libs/libgcc_s-8c7760c8.so.1
A clear and concise description of what the bug is.
Patched in Qemu docker ARMv7 libraries after that give SEGFAULT on real ARMv7
pypa/auditwheel#414 (comment)
Steps To Reproduce
All steps in this post
home-assistant/core#86589 (comment)
Expected behavior
Worked python wheel
patchelf --versionoutputpatchelf 0.17.2
Additional context
strace of patched lib
home-assistant/core#86589 (comment)
The text was updated successfully, but these errors were encountered: