Android NDK: Default linker flags that might save people pain

[cpsauer]

Hi wonderful Bazel team,

I was playing around with the Android NDK. Along the way, I ran into a couple of build surprises that I think could be easily fixed for other people going down the same path, with just a change to the default linker options.

In particular, I think adding ["--no-undefined", "-lm", "-ldl"] to the default linkopts would make things easier for people. Reasoning below:

First, the linker seems to have undefined errors off by default (!) and will therefore happily produce .so's that can't be loaded... We can make this trigger a linker error, as you'd expect on other platforms, by adding --no-undefined.

And second, the basics you'd expect, like the math part of libc (libm) and shared library support (libdl) aren't linked by default, though libc is, and its headers refer to symbols in those libraries. Adding -lm and -ldl fixes the surprises there (which are themselves much easier to find once --no-undefined has been passed). Maybe we intentionally don't want to link to those implicitly, but I'm guessing we do, since bazel automatically links to other NDK libraries, and I don't think you normally have to manually link in libm or libdl on other platforms.

Thanks for your consideration,
Chris

[shamtron]

@cpsauer I'm upgrading from bazel 0.23.2 to 1.2.1 and I'm also running it into what I think are issues related to linker flags. A build that used to work on 0.23.3 now breaks on 1.2.1 with the following error:

ERROR: /private/var/tmp/_bazel_przemeklach/11140cf4f4caa3f85496ce83177eef7f/external/remote_java_tools_linux/BUILD:399:1: Linking of rule '@remote_java_tools_linux//:ijar_cc_binary' failed (Exit 1) clang failed: error executing command external/androidndk/ndk/toolchains/llvm/prebuilt/darwin-x86_64/bin/clang -o bazel-out/arm64-v8a-fastbuild/bin/external/remote_java_tools_linux/ijar_cc_binary ... (remaining 23 argument(s) skipped)

Use --sandbox_debug to see verbose messages from the sandbox
external/androidndk/ndk/toolchains/aarch64-linux-android-4.9/prebuilt/darwin-x86_64/lib/gcc/aarch64-linux-android/4.9.x/libgcc.a(unwind-dw2-fde-dip.o): In function `_Unwind_Find_FDE':
/Volumes/Android/buildbot/src/android/gcc/toolchain/build/../gcc/gcc-4.9/libgcc/unwind-dw2-fde-dip.c:461: undefined reference to `dl_iterate_phdr'
clang: error: linker command failed with exit code 1 (use -v to see invocation)

Did one of your build surprises look like this? I'm at a loss on what to do here.

[cpsauer]

Yup! Add -ldl to linkopts to link libdl.

dl_iterate_phdr looks like a dl (dynamic library) call to me!

[shamtron]

No still doesn't work. I added -ldl at my top level BUILD file and every BUILD file that builds any C++ and it still fails. I don't know where else to go from here. The error message does not reference any of my code. Only the toolchain.

[cpsauer]

(Replied to @shamtron's question in the separate issue he filed.)

[cpsauer]

Another thought, after some more time working around/with NDK builds:
Perhaps we should auto-stripping the .so libraries in opt (release) builds, using -Wl,--strip-all or similar? Looks like iOS is moving this direction, and I'm seeing ~24x size reductions from doing so. Plus, released binaries probably shouldn't have debug info or a local symbol table by default.

In the meantime, if anyone wants an easy fix for all this--just a rule you depend on and if fixes linker defaults plus android pthread issue--lmk. Happy to share.

[cpsauer]

These flags were in Google's internal NDK toolchain, newly released by @ahumesky as https://github.com/bazelbuild/rules_android_ndk, to replace the existing.

It makes me so happy to close this. I bet the change will save pain for almost everyone booting up on the NDK with Bazel!