[Bug]: unable to update gems on android 10

[gouravkhunger]

Problem description

This issue is a continuation from #20359. Once I fixed that and tried gem update, the gems that have native C based implementations and need to compile things on device caused errors. To keep it concise, my hunch is that it is related to the -fno-openmp-implicit-rpath argument on clang. Related: #20039, #19766

Some gems that I faced issue with were bigdecimal, ffi and nokogiri. All of them had these lines in their mkmf.log that describes the error caused while updating:

...
aarch64-linux-android-clang: error: unknown argument: '-fno-openmp-implicit-rpath'
aarch64-linux-android-clang: error: unable to make temporary file: No such file or directory
...

More precisely, there were some specifics issues too:

bigdecimal:

current directory: /data/data/sh.gourav.jekyllex/files/usr/lib/ruby/gems/3.2.0/gems/bigdecimal-3.1.8/ext/bigdecimal
/data/data/sh.gourav.jekyllex/files/usr/bin/ruby extconf.rb
checking for __builtin_clz()... *** extconf.rb failed ***
Could not create Makefile due to some reason, probably lack of necessary
libraries and/or headers.  Check the mkmf.log file for more details.  You may
need configuration options.

...

/data/data/sh.gourav.jekyllex/files/usr/lib/ruby/3.2.0/mkmf.rb:490:in `try_do': The compiler failed to generate an executable file. (RuntimeError)
You have to install development tools first.

	from /data/data/sh.gourav.jekyllex/files/usr/lib/ruby/3.2.0/mkmf.rb:616:in `block in try_compile'
	from /data/data/sh.gourav.jekyllex/files/usr/lib/ruby/3.2.0/mkmf.rb:565:in `with_werror'
...

nokogiri

current directory: /data/data/sh.gourav.jekyllex/files/usr/lib/ruby/gems/3.2.0/gems/nokogiri-1.16.5/ext/nokogiri
/data/data/sh.gourav.jekyllex/files/usr/bin/ruby extconf.rb
checking for whether -std=c99 is accepted as CFLAGS... *** extconf.rb failed ***
Could not create Makefile due to some reason, probably lack of necessary
libraries and/or headers.  Check the mkmf.log file for more details.  You may
need configuration options.

...

/data/data/sh.gourav.jekyllex/files/usr/lib/ruby/3.2.0/mkmf.rb:490:in `try_do': The compiler failed to generate an executable file. (RuntimeError)
You have to install development tools first.

	from /data/data/sh.gourav.jekyllex/files/usr/lib/ruby/3.2.0/mkmf.rb:616:in `block in try_compile'
	from /data/data/sh.gourav.jekyllex/files/usr/lib/ruby/3.2.0/mkmf.rb:563:in `with_werror'
...

Issue with __builtin_clz(), -std=c99, I guess something is wrong with clang. Do I need to downgrade, patch something, or add some other development tools to my bootstrap? Any help would be really appreciated. Thanks!

What steps will reproduce the bug?

I have a custom prefixed bootstrap so maybe it is hard to build and reproduce my exact setup. But this specific issue can be replicated using the android 10 ruby demo I have for com.termux prefixed bootstraps and termux apk I built using workflows here. It is caused when running gem update in the app-debug.apk linked in the release.

What is the expected behavior?

No response

System information

:/data/data/sh.gourav.jekyllex/files/home $ ruby -v
ruby 3.2.2 (2023-03-30 revision e51014f9c0) [aarch64-linux-android]
:/data/data/sh.gourav.jekyllex/files/home $ gem -v
3.4.10
:/data/data/sh.gourav.jekyllex/files/home $ make -v
make: TMPDIR value /data/data/sh.gourav.jekyllex/files/usr/tmp: No such file or directory
make: using default temporary directory '/data/data/sh.gourav.jekyllex/files/usr/tmp/'
GNU Make 4.4.1
Built for aarch64-unknown-linux-android
Copyright (C) 1988-2023 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <https://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
:/data/data/sh.gourav.jekyllex/files/home $ clang -v
clang version 18.1.6
Target: aarch64-unknown-linux-android24
Thread model: posix
InstalledDir: /data/data/sh.gourav.jekyllex/files/usr/bin


My bootstrap has: git, tar, dash, make, ruby, libxslt, libllvm, coreutils, binutils-libs

[licy183]

It should be fixed. Feel free to reopen it if it is not fixed yet. Thanks!

[gouravkhunger]

Hi @licy183 Thanks a lot for the PR! I don't seem to get -fno-openmp-implicit-rpath now. But there is still this error in installing/updating popular gems. Could you please point out what could have gone wrong?

...
In file included from conftest.c:1:
In file included from /data/data/sh.gourav.jekyllex/files/usr/include/ruby-3.2.0/ruby.h:38:
/data/data/sh.gourav.jekyllex/files/usr/include/ruby-3.2.0/ruby/ruby.h:23:10: fatal error: 'stdarg.h' file not found
   23 | #include <stdarg.h>
      |          ^~~~~~~~~~
1 error generated.
...

And I don't see the option to re-open this issue...

[licy183]

stdarg.h should exist in $PREFIX/lib/clang/18/include/stdarg.h. Could try to compile a .c file with stdarg.h with clang?

echo '#include <stdarg.h>' > test.c
clang --verbose test.c

[gouravkhunger]

The $PREFIX/lib/clang/18/include/stdarg.h file does exist, but the compilation fails.

127|:/data/data/sh.gourav.jekyllex/files/home $
echo '#include <stdarg.h>' > test.c
lang --verbose test.c                                 <
clang version 18.1.7
Target: aarch64-unknown-linux-android24
Thread model: posix
InstalledDir: /data/data/sh.gourav.jekyllex/files/usr/bin
 "/data/app/~~_1YePu7TvJMluBP7hcym-Q==/sh.gourav.jekyllex-a71yPQcBmXVmYSW_4f0Ozg==/lib/arm64/lib262.so" -cc1 -triple aarch64-unknown-linux-android24 -emit-obj -mrelax-all -dumpdir a- -disable-free -clear-ast-before-backend -disable-llvm-verifier -discard-value-names -main-file-name test.c -mrelocation-model pic -pic-level 2 -pic-is-pie -mframe-pointer=non-leaf -ffp-contract=on -fno-rounding-math -mconstructor-aliases -funwind-tables=2 -target-cpu generic -target-feature +v8a -target-feature +fp-armv8 -target-feature +neon -target-feature +fix-cortex-a53-835769 -target-abi aapcs -debugger-tuning=gdb -fdebug-compilation-dir=/data/data/sh.gourav.jekyllex/files/home -v -fcoverage-compilation-dir=/data/data/sh.gourav.jekyllex/files/home -resource-dir /data/app/~~_1YePu7TvJMluBP7hcym-Q==/sh.gourav.jekyllex-a71yPQcBmXVmYSW_4f0Ozg==/lib/lib/clang/18 -isysroot /data/data/sh.gourav.jekyllex/files -internal-isystem /data/app/~~_1YePu7TvJMluBP7hcym-Q==/sh.gourav.jekyllex-a71yPQcBmXVmYSW_4f0Ozg==/lib/lib/clang/18/include -internal-isystem /data/data/sh.gourav.jekyllex/files/usr/local/include -internal-externc-isystem /data/data/sh.gourav.jekyllex/files/usr/include/aarch64-linux-android -internal-externc-isystem /data/data/sh.gourav.jekyllex/files/include -internal-externc-isystem /data/data/sh.gourav.jekyllex/files/usr/include -ferror-limit 19 -femulated-tls -fno-signed-char -fgnuc-version=4.2.1 -fskip-odr-check-in-gmf -fcolor-diagnostics -target-feature +outline-atomics -D__GCC_HAVE_DWARF2_CFI_ASM=1 -o /data/data/sh.gourav.jekyllex/files/usr/tmp/test-3b4dd1.o -x c test.c
clang -cc1 version 18.1.7 based upon LLVM 18.1.7 default target aarch64-unknown-linux-android24
ignoring nonexistent directory "/data/app/~~_1YePu7TvJMluBP7hcym-Q==/sh.gourav.jekyllex-a71yPQcBmXVmYSW_4f0Ozg==/lib/lib/clang/18/include"
ignoring nonexistent directory "/data/data/sh.gourav.jekyllex/files/usr/local/include"
ignoring nonexistent directory "/data/data/sh.gourav.jekyllex/files/include"
#include "..." search starts here:
#include <...> search starts here:
 /data/data/sh.gourav.jekyllex/files/usr/include/aarch64-linux-android
 /data/data/sh.gourav.jekyllex/files/usr/include
End of search list.
test.c:1:10: fatal error: 'stdarg.h' file not found
    1 | #include <stdarg.h>
      |          ^~~~~~~~~~
1 error generated.
1|:/data/data/sh.gourav.jekyllex/files/home $

Maybe again some android 10 path issue.

[gouravkhunger]

Tried with the -I flag:

lang --verbose test.c -I$PREFIX/lib/clang/18/include  <
clang version 18.1.7
Target: aarch64-unknown-linux-android24
Thread model: posix
InstalledDir: /data/data/sh.gourav.jekyllex/files/usr/bin
 "/data/app/~~_1YePu7TvJMluBP7hcym-Q==/sh.gourav.jekyllex-a71yPQcBmXVmYSW_4f0Ozg==/lib/arm64/lib262.so" -cc1 -triple aarch64-unknown-linux-android24 -emit-obj -mrelax-all -dumpdir a- -disable-free -clear-ast-before-backend -disable-llvm-verifier -discard-value-names -main-file-name test.c -mrelocation-model pic -pic-level 2 -pic-is-pie -mframe-pointer=non-leaf -ffp-contract=on -fno-rounding-math -mconstructor-aliases -funwind-tables=2 -target-cpu generic -target-feature +v8a -target-feature +fp-armv8 -target-feature +neon -target-feature +fix-cortex-a53-835769 -target-abi aapcs -debugger-tuning=gdb -fdebug-compilation-dir=/data/data/sh.gourav.jekyllex/files/home -v -fcoverage-compilation-dir=/data/data/sh.gourav.jekyllex/files/home -resource-dir /data/app/~~_1YePu7TvJMluBP7hcym-Q==/sh.gourav.jekyllex-a71yPQcBmXVmYSW_4f0Ozg==/lib/lib/clang/18 -I /data/data/sh.gourav.jekyllex/files/usr/lib/clang/18/include -isysroot /data/data/sh.gourav.jekyllex/files -internal-isystem /data/app/~~_1YePu7TvJMluBP7hcym-Q==/sh.gourav.jekyllex-a71yPQcBmXVmYSW_4f0Ozg==/lib/lib/clang/18/include -internal-isystem /data/data/sh.gourav.jekyllex/files/usr/local/include -internal-externc-isystem /data/data/sh.gourav.jekyllex/files/usr/include/aarch64-linux-android -internal-externc-isystem /data/data/sh.gourav.jekyllex/files/include -internal-externc-isystem /data/data/sh.gourav.jekyllex/files/usr/include -ferror-limit 19 -femulated-tls -fno-signed-char -fgnuc-version=4.2.1 -fskip-odr-check-in-gmf -fcolor-diagnostics -target-feature +outline-atomics -D__GCC_HAVE_DWARF2_CFI_ASM=1 -o /data/data/sh.gourav.jekyllex/files/usr/tmp/test-528f86.o -x c test.c
clang -cc1 version 18.1.7 based upon LLVM 18.1.7 default target aarch64-unknown-linux-android24
ignoring nonexistent directory "/data/app/~~_1YePu7TvJMluBP7hcym-Q==/sh.gourav.jekyllex-a71yPQcBmXVmYSW_4f0Ozg==/lib/lib/clang/18/include"
ignoring nonexistent directory "/data/data/sh.gourav.jekyllex/files/usr/local/include"
ignoring nonexistent directory "/data/data/sh.gourav.jekyllex/files/include"
#include "..." search starts here:
#include <...> search starts here:
 /data/data/sh.gourav.jekyllex/files/usr/lib/clang/18/include
 /data/data/sh.gourav.jekyllex/files/usr/include/aarch64-linux-android
 /data/data/sh.gourav.jekyllex/files/usr/include
End of search list.
 "/data/data/sh.gourav.jekyllex/files/usr/bin/ld.lld" --sysroot=/data/data/sh.gourav.jekyllex/files -EL --fix-cortex-a53-843419 -z now -z relro -z max-page-size=4096 --hash-style=gnu -rpath=/data/data/sh.gourav.jekyllex/files/usr/lib --eh-frame-hdr -m aarch64linux -pie -dynamic-linker /system/bin/linker64 -o a.out /data/data/sh.gourav.jekyllex/files/usr/lib/crtbegin_dynamic.o -L/data/data/sh.gourav.jekyllex/files/usr/lib -L/system/lib64 /data/data/sh.gourav.jekyllex/files/usr/tmp/test-528f86.o /data/app/~~_1YePu7TvJMluBP7hcym-Q==/sh.gourav.jekyllex-a71yPQcBmXVmYSW_4f0Ozg==/lib/lib/clang/18/lib/linux/libclang_rt.builtins-aarch64-android.a -l:libunwind.a -ldl -lc /data/app/~~_1YePu7TvJMluBP7hcym-Q==/sh.gourav.jekyllex-a71yPQcBmXVmYSW_4f0Ozg==/lib/lib/clang/18/lib/linux/libclang_rt.builtins-aarch64-android.a -l:libunwind.a -ldl /data/data/sh.gourav.jekyllex/files/usr/lib/crtend_android.o
ld.lld: error: cannot open /data/app/~~_1YePu7TvJMluBP7hcym-Q==/sh.gourav.jekyllex-a71yPQcBmXVmYSW_4f0Ozg==/lib/lib/clang/18/lib/linux/libclang_rt.builtins-aarch64-android.a: No such file or directory
ld.lld: error: cannot open /data/app/~~_1YePu7TvJMluBP7hcym-Q==/sh.gourav.jekyllex-a71yPQcBmXVmYSW_4f0Ozg==/lib/lib/clang/18/lib/linux/libclang_rt.builtins-aarch64-android.a: No such file or directory
clang: error: linker command failed with exit code 1 (use -v to see invocation)
1|:/data/data/sh.gourav.jekyllex/files/home $

[licy183]

clang will search $clang_path/../lib/clang/$clang_major_version/include. It seems that you install the clang binary in /data/app/~~_1YePu7TvJMluBP7hcym-Q==/sh.gourav.jekyllex-a71yPQcBmXVmYSW_4f0Ozg==/lib/arm64/ and then it will search /data/app/~~_1YePu7TvJMluBP7hcym-Q==/sh.gourav.jekyllex-a71yPQcBmXVmYSW_4f0Ozg==/lib/lib/clang/18/include, which doesn't exist.

[gouravkhunger]

Hi @licy183 Thank you for pointing that out! I was able to hardcode my binary path. Not really a good approach, maybe I could use TERMUX_PREFIX from the environment to make for a better patch.

clang still could not find the default temporary path so I manually did mkdir $PREFIX/tmp and all things began to work!

I could compile a simple program and use /system/bin/linker64 to execute it:

test.c

#include <stdarg.h>
#include <stdio.h>

int main() {
   printf("Hello, World!\n");
   return 0;
}

:/data/data/sh.gourav.jekyllex/files/home $ clang test.c
:/data/data/sh.gourav.jekyllex/files/home $ /system/bin/linker64 $HOME/a.out
Hello, World!

But something to note is that all this makes updating the default gems possible. I still can't gem install ... and expect any new binaries installed to work because of targeting android 10, unless the required gems are built and packaged by default.