cc_import and cc_library of dylib produces LC_RPATH not LC_LOAD_DYLIB and doesn't load at runtime

[paulspixels]

Description of the problem / feature request:

Building a simple C++ application on osx that loads a third party (<something>.h + <libsomething>.dylib) DLL, I get dyld errors -> "image not found" at runtime when the DLL is loaded at runtime. I have looked at the binary with "otool -l" and found that cc_library .so files which Bazel builds from source get loaded with specific paths like this:

cmd LC_LOAD_DYLIB
      cmdsize 120
         name @loader_path/../../_solib_darwin_x86_64//<some_bazel_built_dll>.so (offset 24)

Bazel puts the third party .dylib one level deeper in <some_directory>, rather than with the rest of the .so files that bazel generates by compiling libraries. However, although the library .dylib file shows up in <some_directory>, bazel is annotating the binary with just an LC_RPATH and not an LC_LOAD_DYLIB command for this prebuilt DLL. This happens whether I use a cc_import or cc_library build rule for it. Specifically, the binary that uses the DLL library shows its last entry is my cc_import-ed DLL's directory:

          cmd LC_RPATH
      cmdsize 128
         path @loader_path/../../_solib_darwin_x86_64/<some_directory> (offset 12)

That path is valid: I have looked in bazel-bin and <some_directory>/<libsomething>.dylib exists. However, for whatever reason the runtime is not finding the <libsomething>.dylib file within that directory when it tries to actually load <libsomething>.dylib.

Feature requests: what underlying problem are you trying to solve with this feature?

Is there some difference in the linker/loader procedure for cc_import versus bazel-built DLL files? bazel has no problem generating LC_LOAD_DYLIB cmd for all the other DLLs, but seems to only generate a directory LC_RPATH cmd for the DLL that I am trying to link against / import from a third party.

Bugs: what's the simplest, easiest way to reproduce this bug? Please provide a minimal example if possible.

I posted an example here: https://forums.opalkelly.com/t/dll-import-location-for-libokfrontpanel-dylib/1661/7

It is unclear if the problem is with bazel (especially given issue 1435 linked below), but since this is a third party DLL I don't know if it could be compounded by whatever method/flags the third party used to compile <libsomething>.dylib.

What operating system are you running Bazel on?

OSX

What's the output of bazel info release?

release 1.1.0

Have you found anything relevant by searching the web?

https://blog.krzyzanowskim.com/2018/12/05/rpath-what/
https://medium.com/@donblas/fun-with-rpath-otool-and-install-name-tool-e3e41ae86172

This bazel bug seems to be similar or related:
bazel-contrib/rules_go#1435

Any other information, logs, or outputs that you want to share?

The DLL is a C++ library from Opal Kelly, and it works correctly when I just put it in /usr/local/lib. I just want cc_import and/or cc_library to correctly build a working application.

[fmeum]

@oquenchil I believe that I may be hitting this issue over at bazel-contrib/rules_go#3151. Do you have any pointers for how to work around this issue or even fix it in Bazel?

[pcjanzen]

I think this is WAI from Bazel's point of view. The linker always constructs the LC_LOAD_DYLIB command from whatever is in the LC_ID_DYLIB field of the dependent library. So the only thing that Bazel can do is construct a unique directory for the imported dylib, and then add that directory to the binary's rpath.

[fmeum]

I see, thanks. I'm still not sure why I'm seeing different behavior between Linux and macOS then on bazel-contrib/rules_go#3151, but will continue to investigate.

@pcjanzen Do you happen to know whether

bazel/src/main/java/com/google/devtools/build/lib/rules/cpp/CppActionConfigs.java

Line 551 in 1a7083e

// TODO(b/27153401): This should probably be @loader_path on osx.

could be of any significance here? I am dealing with a cc_test that has a dependency on a shared library imported via cc_common.create_library_to_link.

[pcjanzen]

No, I don't think so. The sordid tale of $EXEC_ORIGIN is at #10809 (comment).

The GoLink step that generates libadder_shared.dylib is passing -install_name tests/core/c_linkmodes/adder_shared_/libadder_shared.dylib as one of the arguments. When cc_binary or cc_library generates a shared library, it doesn't do this, so then the LC_ID_DYLIB is the same as the argument of -o, so it's a path relative to the execroot.

The macos cc_wrapper then fixes up all the execroot-relative paths in the binary: https://cs.opensource.google/bazel/bazel/+/master:tools/cpp/osx_cc_wrapper.sh
but this won't happen if the LC_LOAD_DYLIB command refers to a string that does not correspond to a file that's accessible from the execroot.

[jheaff1]

I see, thanks. I'm still not sure why I'm seeing different behavior between Linux and macOS then on bazelbuild/rules_go#3151, but will continue to investigate.

@pcjanzen Do you happen to know whether

bazel/src/main/java/com/google/devtools/build/lib/rules/cpp/CppActionConfigs.java

Line 551 in 1a7083e

// TODO(b/27153401): This should probably be @loader_path on osx.

could be of any significance here? I am dealing with a cc_test that has a dependency on a shared library imported via cc_common.create_library_to_link.

I'm facing the same issue when using rules_foreign_cc. It seems that if a cc_test depends on a shared library built by rules_foreign_cc, the required dylibs aren't loaded. rules_foreign_cc too uses cc_common.create_libraries_to_link. Do you happen to have any decent workaround?

[github-actions]

Thank you for contributing to the Bazel repository! This issue has been marked as stale since it has not had any activity in the last 1+ years. It will be closed in the next 90 days unless any other activity occurs or one of the following labels is added: "not stale", "awaiting-bazeler". Please reach out to the triage team (@bazelbuild/triage) if you think this issue is still relevant or you are interested in getting the issue resolved.

[github-actions]

This issue has been automatically closed due to inactivity. If you're still interested in pursuing this, please post @bazelbuild/triage in a comment here and we'll take a look. Thanks!

[fmeum]

@bazelbuild/triage not stale

[ashi009]

not stale but closed?