Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Static/Dynamic Linked Libraries (Arch/Ubuntu) #54

Closed
ian-coccimiglio opened this issue Jul 30, 2024 · 5 comments
Closed

Static/Dynamic Linked Libraries (Arch/Ubuntu) #54

ian-coccimiglio opened this issue Jul 30, 2024 · 5 comments

Comments

@ian-coccimiglio
Copy link

ian-coccimiglio commented Jul 30, 2024 •
edited
Loading

I wanted to look into one of the questions brought up on image.sc about requiring libcrypt for running Jaunch.

I typically run Manjaro (based on Arch) and I can replicate his problem. Simply installing the libcrypt library solves the issue, but I wanted to do a bit more investigation, so I ran a few VMs to test this.

Tested operating systems:

  1. Ubuntu 20.04 (Failing because of a required GLIBC=2.34. I think Ubuntu version >= 22 may be required).
  2. Ubuntu 24.04 (Succeeds out of the box because Ubuntu 24 comes with LDD=2.39)
  3. Manjaro 6.6.32 (Succeeds iff libcrypt.so.1 is installed)

It seems that all the libraries are .so dynamic libraries. I'm not sure if this is intended behaviour, but it seems very consistent. (Note: I'm realizing that the issue is not that dynamic libraries are used, but rather that they should not be required at run-time).

Some information and straces are below:

Ubuntu 20.04 (failing)

ubuntu@ubuntu2004:~/Fiji.app $ ./fiji-linux-x64  
./fiji-linux-x64: /lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.34` not found (required by ./fiji-linux-x64)

ubuntu@ubuntu2004:~/Fiji.app$ ldd --version
ldd (Ubuntu GLIBC 2.31-0ubuntu9.2) 2.31
Copyright (C) 2020 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
Written by Roland McGrath and Ulrich Drepper.

ubuntu@ubuntu2004:~/Fiji.app $ strace -e trace=file,open,openat,access ./jaunch/jaunch-linux-x64 
execve("./jaunch/jaunch-linux-x64", ["./jaunch/jaunch-linux-x64"], 0x7ffc98a3d870 /* 49 vars */) = 0
open("/proc/self/exe", O_RDONLY)        = 3
readlink("/proc/self/exe", "/home/ubuntu/Fiji.app/jaunch/jau"..., 4095) = 45
open("/lib64/ld-linux-x86-64.so.2", O_RDONLY) = 4
access("/etc/ld.so.preload", R_OK)      = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/etc/ld.so.cache", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/lib/x86_64-linux-gnu/libresolv.so.2", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/lib/x86_64-linux-gnu/libm.so.6", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/lib/x86_64-linux-gnu/libpthread.so.0", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/lib/x86_64-linux-gnu/libutil.so.1", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/lib/x86_64-linux-gnu/libcrypt.so.1", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/lib/x86_64-linux-gnu/librt.so.1", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/lib/x86_64-linux-gnu/libdl.so.2", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/lib/x86_64-linux-gnu/libgcc_s.so.1", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/lib/x86_64-linux-gnu/libc.so.6", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/sys/devices/system/cpu/online", O_RDONLY|O_CLOEXEC) = 3

Ubuntu 24.04 (works out of the box)

ianfc@ianfc-VirtualBox:~/Fiji.app$ uname -rsvmo
Linux 6.8.0-39-generic #39-Ubuntu SMP PREEMPT_DYNAMIC Fri Jul  5 21:49:14 UTC 2024 x86_64 GNU/Linux

ianfc@ianfc-VirtualBox:~/Fiji.app$ ldd --version
ldd (Ubuntu GLIBC 2.39-0ubuntu8.2) 2.39
Copyright (C) 2024 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
Written by Roland McGrath and Ulrich Drepper.

ianfc@ianfc-VirtualBox:~/Fiji.app$ strace -e trace=file,open,openat,access ./jaunch/jaunch-linux-x64 
execve("./jaunch/jaunch-linux-x64", ["./jaunch/jaunch-linux-x64"], 0x7fff4819c7f0 /* 48 vars */) = 0
open("/proc/self/exe", O_RDONLY)        = 3
readlink("/proc/self/exe", "/home/ianfc/Fiji.app/jaunch/jaun"..., 4095) = 44
open("/lib64/ld-linux-x86-64.so.2", O_RDONLY) = 4
access("/etc/ld.so.preload", R_OK)      = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/etc/ld.so.cache", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/lib/x86_64-linux-gnu/libresolv.so.2", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/lib/x86_64-linux-gnu/libm.so.6", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/lib/x86_64-linux-gnu/libpthread.so.0", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/lib/x86_64-linux-gnu/libutil.so.1", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/lib/x86_64-linux-gnu/libcrypt.so.1", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/lib/x86_64-linux-gnu/librt.so.1", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/lib/x86_64-linux-gnu/libdl.so.2", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/lib/x86_64-linux-gnu/libgcc_s.so.1", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/lib/x86_64-linux-gnu/libc.so.6", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/sys/devices/system/cpu/online", O_RDONLY|O_CLOEXEC) = 3

Manjaro 6.6 (working w/ libcrypt installed via pacman)

~/Fiji.app ❯ uname -rsvmo                                                                                                                                                                    
Linux 6.6.32-1-MANJARO #1 SMP PREEMPT_DYNAMIC Mon May 27 03:41:25 UTC 2024 x86_64 GNU/Linux

~/Fiji.app ❯ ldd --version                                                                                                                                                                   
ldd (GNU libc) 2.39
Copyright (C) 2024 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
Written by Roland McGrath and Ulrich Drepper.

~/Fiji.app ❯ ./jaunch/jaunch-linux-x64                                                                                                                                                       
Hello! You have found the Jaunch configurator.
...

~/Fiji.app ❯ strace -e trace=file,open,openat,access ./jaunch/jaunch-linux-x64                                                                                                               execve("./jaunch/jaunch-linux-x64", ["./jaunch/jaunch-linux-x64"], 0x7ffdbe0db260 /* 113 vars */) = 0
open("/proc/self/exe", O_RDONLY)        = 3
readlink("/proc/self/exe", "/home/ian/Fiji.app/jaunch/jaunch"..., 4095) = 42
open("/lib64/ld-linux-x86-64.so.2", O_RDONLY) = 4
access("/etc/ld.so.preload", R_OK)      = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/opt/cuda/lib64/glibc-hwcaps/x86-64-v3/libresolv.so.2", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
newfstatat(AT_FDCWD, "/opt/cuda/lib64/glibc-hwcaps/x86-64-v3/", 0x7ffecd7a4a90, 0) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/opt/cuda/lib64/glibc-hwcaps/x86-64-v2/libresolv.so.2", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
newfstatat(AT_FDCWD, "/opt/cuda/lib64/glibc-hwcaps/x86-64-v2/", 0x7ffecd7a4a90, 0) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/opt/cuda/lib64/libresolv.so.2", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
newfstatat(AT_FDCWD, "/opt/cuda/lib64/", {st_mode=S_IFDIR|0755, st_size=4096, ...}, 0) = 0
openat(AT_FDCWD, "glibc-hwcaps/x86-64-v3/libresolv.so.2", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "glibc-hwcaps/x86-64-v2/libresolv.so.2", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "libresolv.so.2", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/etc/ld.so.cache", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/usr/lib/libresolv.so.2", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/opt/cuda/lib64/libm.so.6", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "glibc-hwcaps/x86-64-v3/libm.so.6", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "glibc-hwcaps/x86-64-v2/libm.so.6", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "libm.so.6", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/usr/lib/libm.so.6", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/opt/cuda/lib64/libpthread.so.0", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "glibc-hwcaps/x86-64-v3/libpthread.so.0", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "glibc-hwcaps/x86-64-v2/libpthread.so.0", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "libpthread.so.0", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/usr/lib/libpthread.so.0", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/opt/cuda/lib64/libutil.so.1", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "glibc-hwcaps/x86-64-v3/libutil.so.1", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "glibc-hwcaps/x86-64-v2/libutil.so.1", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "libutil.so.1", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/usr/lib/libutil.so.1", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/opt/cuda/lib64/libcrypt.so.1", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "glibc-hwcaps/x86-64-v3/libcrypt.so.1", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "glibc-hwcaps/x86-64-v2/libcrypt.so.1", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "libcrypt.so.1", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/usr/lib/libcrypt.so.1", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/opt/cuda/lib64/librt.so.1", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "glibc-hwcaps/x86-64-v3/librt.so.1", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "glibc-hwcaps/x86-64-v2/librt.so.1", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "librt.so.1", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/usr/lib/librt.so.1", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/opt/cuda/lib64/libdl.so.2", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "glibc-hwcaps/x86-64-v3/libdl.so.2", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "glibc-hwcaps/x86-64-v2/libdl.so.2", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "libdl.so.2", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/usr/lib/libdl.so.2", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/opt/cuda/lib64/libgcc_s.so.1", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "glibc-hwcaps/x86-64-v3/libgcc_s.so.1", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "glibc-hwcaps/x86-64-v2/libgcc_s.so.1", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "libgcc_s.so.1", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/usr/lib/libgcc_s.so.1", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/opt/cuda/lib64/libc.so.6", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "glibc-hwcaps/x86-64-v3/libc.so.6", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "glibc-hwcaps/x86-64-v2/libc.so.6", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "libc.so.6", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/usr/lib/libc.so.6", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/sys/devices/system/cpu/online", O_RDONLY|O_CLOEXEC) = 3
@marktsuchida
Copy link

Or, more directly (on Ubuntu 24.04/WSL2),

$ make dist
[...]
$ readelf -d dist/jaunch/jaunch-linux-x64

Dynamic section at offset 0x119d70 contains 31 entries:
  Tag        Type                         Name/Value
[...]
 0x0000000000000001 (NEEDED)             Shared library: [libresolv.so.2]
 0x0000000000000001 (NEEDED)             Shared library: [libm.so.6]
 0x0000000000000001 (NEEDED)             Shared library: [libpthread.so.0]
 0x0000000000000001 (NEEDED)             Shared library: [libutil.so.1]
 0x0000000000000001 (NEEDED)             Shared library: [libcrypt.so.1]
 0x0000000000000001 (NEEDED)             Shared library: [librt.so.1]
 0x0000000000000001 (NEEDED)             Shared library: [libdl.so.2]
 0x0000000000000001 (NEEDED)             Shared library: [libgcc_s.so.1]
 0x0000000000000001 (NEEDED)             Shared library: [libc.so.6]
 0x0000000000000001 (NEEDED)             Shared library: [ld-linux-x86-64.so.2]
[...]

Searching for why libcrypt.so.1 (part of GNU libc, which is probably why it is dynamically linked even for a statically-linked executable) might exist on some distributions and not others, I found this: pypa/manylinux#305.

But as best as I can tell jaunch-linux-x64 does not actually use any of the symbols (functions) in libcrypt:

  • nm -UDg /usr/lib/x86_64-linux-gnu/libcrypt.so.1.1.0 shows that all the symbols have crypt or setkey in their name, and
  • nm -u dist/jaunch/jaunch-linux-x64 lists no such symbols.
  • Nor is it a transitive dependency of jaunch-linxu-x64 via any of the shared libraries listed above.

So it might be that the dependency can be dropped by setting the linker flags correctly.

The GNU linker has an --as-needed option (passed as -Wl,--as-needed if specifying as an arg to gcc or g++) which avoids listing shared library dependencies that are not actually used (called). (See man 1 ld.)

How to make the Gradle/Kotlin build use that flag I am not sure.

ctrueden added a commit that referenced this issue Jul 31, 2024
@ctrueden
Add a workaround for libcrypt linking issue
95df6e5 
See #54.
@ctrueden
Copy link
Member

Thank you @ian-coccimiglio for the detailed writeup, and @marktsuchida for the additional detective work and suggestion on how to fix. I asked an all-knowing LLM (Claude) how to pass --as-needed to the Gradle build, and 95df6e5 is the result. Would you mind testing it? You can rebuild Jaunch by pulling the latest code from the main branch, setting JAVA_HOME to an OpenJDK installation, then running make dist.

@ian-coccimiglio
Copy link
Author

ian-coccimiglio commented Jul 31, 2024 •
edited
Loading

Thank you @ian-coccimiglio for the detailed writeup, and @marktsuchida for the additional detective work and suggestion on how to fix. I asked an all-knowing LLM (Claude) how to pass --as-needed to the Gradle build, and 95df6e5 is the result. Would you mind testing it? You can rebuild Jaunch by pulling the latest code from the main branch, setting JAVA_HOME to an OpenJDK installation, then running make dist.

Tested 95df6e5:

  1. Manjaro: Confirmed working. Jaunch able to be built on Manjaro with no dependency on libcrypt.
  2. Ubuntu 24.04: Confirmed working as well.
  3. Ubuntu 20.04: Not working, but not because of libcrypt (rather, just out of date glibc, I think?).

New readelf output from Ubuntu 24.04 build.

$ readelf -d jaunch-linux-x64 

Dynamic section at offset 0x119da0 contains 28 entries:
  Tag        Type                         Name/Value
 [...]
 0x0000000000000001 (NEEDED)             Shared library: [libm.so.6]
 0x0000000000000001 (NEEDED)             Shared library: [libpthread.so.0]
 0x0000000000000001 (NEEDED)             Shared library: [librt.so.1]
 0x0000000000000001 (NEEDED)             Shared library: [libdl.so.2]
 0x0000000000000001 (NEEDED)             Shared library: [libgcc_s.so.1]
 0x0000000000000001 (NEEDED)             Shared library: [libc.so.6]
 0x0000000000000001 (NEEDED)             Shared library: [ld-linux-x86-64.so.2]
[...]

Hooray!
Ian.

@marktsuchida
Copy link

Glad that worked!

As for 3., (assuming that a jaunch built on 20.04 works on 20.04) probably the easiest way to build a binary that works widely is to build using the oldest distribution (docker image) practically possible, selected based on having an older version of glibc (the approach that Python's manylinux takes).

@ctrueden
Copy link
Member

ctrueden commented Aug 1, 2024 •
edited
Loading

@marktsuchida Yes, and also -ldl to link to libdl, since older glibc hadn't merged the libdl stuff into libc yet...

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@ctrueden @marktsuchida @ian-coccimiglio