lldb: get lldb API tests working with newer Android NDKs #106443
Conversation
Android introduced a unified tools layout in NDK r19 (2019) and removed the old layout in r22 (2021). Running lldb tests with NDK r22 or newer fails when compiling the test inferiors. This change updates `Android.rules` to match the newer unified tools structure introduced in NDK r19, which is described in more detail at android/ndk#780. NOTE: After this change, ONLY NDK r19c and later can be used when running the lldb tests. The pre-2019 NDK structure is no longer supported.
Provide an error message when compiling LLDB API test inferiors against an older Android NDK without the unified toolchain layout. This message will help guide developers to install and use an NDK r19 or newer.
|
@llvm/pr-subscribers-lldb Author: Andrew Rogers (andrurogerz) ChangesPurposeRunning the LLDB API tests against a remote Android target with NDK version r22 or later fails to compile the test inferiors. NDK r21 from 2021 is the most recent NDK that still works with the LLDB API tests. This PR updates the Android make rules to support newer Android NDK versions (r19 and later). Overview
Problem DetailsAndroid introduced a unified tools layout in NDK r19 (2019) and removed the old layout in r22 (2021). Releases r19, r20, and r21 support both the old and new layout side-by-side. More details are in #106270. ValidationRan a sub-set of the LLDB API tests against remote Android targets for the four primary architectures i386, x86_64, arm, and aarch64. No validation was done against riscv targets. For each case, ran the copy of Ran tests with both r19 (the oldest supported) and r26 (more recent, unified layout only) NDK versions. Example test command for aarch64: NOTE: there are a lot of test failures when running the full suite (especially against 32-bit ARM target). These failures occur independent of this change. Verified the expected error message appears when attempting to run using NDK r18 ImpactThis change explicitly removes support for the pre-2019 NDK structure. Only NDK r19 (from 2019) and later can be used when running the LLDB API tests. If the maintainers object, we can easily support both the old and new NDK toolchain layouts side-by-side at the cost of readability/maintainability. Since this change only impacts tests, I don't see much value in supporting NDKs that are over 5 years old. Guidance to Reviewers
Full diff: https://github.com/llvm/llvm-project/pull/106443.diff 1 Files Affected:
diff --git a/lldb/packages/Python/lldbsuite/test/make/Android.rules b/lldb/packages/Python/lldbsuite/test/make/Android.rules
index cd7d8ae74d6bf3..3bece0d0ae8045 100644
--- a/lldb/packages/Python/lldbsuite/test/make/Android.rules
+++ b/lldb/packages/Python/lldbsuite/test/make/Android.rules
@@ -1,81 +1,55 @@
NDK_ROOT := $(shell dirname $(CC))/../../../../..
-ifeq "$(findstring 64, $(ARCH))" "64"
- # lowest 64-bit API level
- API_LEVEL := 21
-else ifeq "$(ARCH)" "i386"
- # clone(2) declaration is present only since this api level
- API_LEVEL := 17
+ifeq "$(HOST_OS)" "Linux"
+ HOST_TAG := linux-x86_64
+else ifeq "$(HOST_OS)" "Darwin"
+ HOST_TAG := darwin-x86_64
else
- # lowest supported 32-bit API level
- API_LEVEL := 16
+ HOST_TAG := windows-x86_64
+endif
+
+TOOLCHAIN_SYSROOT := $(NDK_ROOT)/toolchains/llvm/prebuilt/$(HOST_TAG)/sysroot
+
+ifeq "$(wildcard $(TOOLCHAIN_SYSROOT)/.)" ""
+# Compiling test inferiors for Android requires an NDK with the unified
+# toolchain introduced in version r19.
+$(error "No unified toolchain sysroot found in $(NDK_ROOT). NDK must be r19 or later.")
endif
ifeq "$(ARCH)" "arm"
- SYSROOT_ARCH := arm
- STL_ARCH := armeabi-v7a
TRIPLE := armv7-none-linux-androideabi
ARCH_CFLAGS += -march=armv7-a -mfloat-abi=softfp -mfpu=vfpv3-d16 -marm
else ifeq "$(ARCH)" "aarch64"
- SYSROOT_ARCH := arm64
- STL_ARCH := arm64-v8a
TRIPLE := aarch64-none-linux-android
else ifeq "$(ARCH)" "i386"
- SYSROOT_ARCH := x86
- STL_ARCH := x86
TRIPLE := i686-none-linux-android
else
- SYSROOT_ARCH := $(ARCH)
- STL_ARCH := $(ARCH)
TRIPLE := $(ARCH)-none-linux-android
endif
-ifeq "$(findstring 86,$(ARCH))" "86"
- TOOLCHAIN_DIR := $(STL_ARCH)-4.9
-else ifeq "$(ARCH)" "arm"
- TOOLCHAIN_DIR := arm-linux-androideabi-4.9
-else
- TOOLCHAIN_DIR := $(subst -none,,$(TRIPLE))-4.9
-endif
+# lowest 64-bit API level
+API_LEVEL := 21
ifeq "$(ARCH)" "arm"
- TOOL_PREFIX := arm-linux-androideabi
-else
- TOOL_PREFIX := $(subst -none,,$(TRIPLE))
-endif
-
-ifeq "$(HOST_OS)" "Linux"
- HOST_TAG := linux-x86_64
-else ifeq "$(HOST_OS)" "Darwin"
- HOST_TAG := darwin-x86_64
+ ARCH_DIR := arm-linux-androideabi
else
- HOST_TAG := windows-x86_64
-endif
-
-GCC_TOOLCHAIN = $(NDK_ROOT)/toolchains/$(TOOLCHAIN_DIR)/prebuilt/$(HOST_TAG)
-
-OBJCOPY ?= $(GCC_TOOLCHAIN)/bin/$(TOOL_PREFIX)-objcopy
-ARCHIVER ?= $(GCC_TOOLCHAIN)/bin/$(TOOL_PREFIX)-ar
-
-ifeq "$(findstring clang,$(CC))" "clang"
- ARCH_CFLAGS += -target $(TRIPLE) --gcc-toolchain=$(GCC_TOOLCHAIN)
- ARCH_LDFLAGS += -target $(TRIPLE) --gcc-toolchain=$(GCC_TOOLCHAIN)
+ ARCH_DIR := $(subst -none,,$(TRIPLE))
endif
-ARCH_CFLAGS += --sysroot=$(NDK_ROOT)/sysroot \
- -isystem $(NDK_ROOT)/sysroot/usr/include/$(TOOL_PREFIX) \
- -D__ANDROID_API__=$(API_LEVEL) \
- -isystem $(NDK_ROOT)/platforms/android-$(API_LEVEL)/arch-$(SYSROOT_ARCH)/usr/include
-
-ARCH_LDFLAGS += --sysroot=$(NDK_ROOT)/platforms/android-$(API_LEVEL)/arch-$(SYSROOT_ARCH) -lm
+ARCH_CFLAGS += \
+ --target=$(TRIPLE) \
+ --sysroot=$(TOOLCHAIN_SYSROOT) \
+ -D __ANDROID_API__=$(API_LEVEL) \
ARCH_CXXFLAGS += \
- -isystem $(NDK_ROOT)/sources/cxx-stl/llvm-libc++/include \
- -isystem $(NDK_ROOT)/sources/android/support/include \
- -isystem $(NDK_ROOT)/sources/cxx-stl/llvm-libc++abi/include
+ -isystem $(TOOLCHAIN_SYSROOT)/usr/include/c++/v1 \
ARCH_LDFLAGS += \
- -L$(NDK_ROOT)/sources/cxx-stl/llvm-libc++/libs/$(STL_ARCH) \
- $(NDK_ROOT)/sources/cxx-stl/llvm-libc++/libs/$(STL_ARCH)/libc++_static.a \
+ --target=$(TRIPLE) \
+ --sysroot=$(TOOLCHAIN_SYSROOT) \
+ --prefix=$(TOOLCHAIN_SYSROOT)/usr/lib/$(ARCH_DIR)/$(API_LEVEL) \
+ --library-directory=$(TOOLCHAIN_SYSROOT)/usr/lib/$(ARCH_DIR)/$(API_LEVEL) \
+ $(TOOLCHAIN_SYSROOT)/usr/lib/$(ARCH_DIR)/libc++_static.a \
+ -lm \
-lc++abi \
- -nostdlib++
+ -nostdlib++ \
|
The NDK unified toolchain layout includes `llvm-objcopy` and `llvm-ar` host executables. A few of the lldb API tests rely on these tools. Previously, the GCC verions of these tools were used from their old NDK location which no longer exists in post-r22 NDKs. Instead, use the llvm versions of these tools which are included in the unified toolchain.
|
I applied feedback from @labath (thank you!) and I am happy to leave this PR up for a while in case anyone else has thoughts. I also made one improvement: It now sets |
|
@labath there have been no additional comments; is there anyone specifically you'd like me to seek feedback from? @JDevlieghere would you have a moment to take a look at this PR soon? I really appreciate it. @compnerd FYI |
No, this is fine. Thanks for reminding me. |
|
Going to go ahead and merge this since there/s not been any additional feedback. |
Purpose
Running the LLDB API tests against a remote Android target with NDK version r22 or later fails to compile the test inferiors. NDK r21 from 2021 is the most recent NDK that still works with the LLDB API tests. This PR updates the Android make rules to support newer Android NDK versions (r19 and later).
Overview
Android.rulesto match the newer Android NDK unified toolchain layout introduced in NDK r19OBJCOPYandARCHIVERenv vars, required by a few test cases, to theirllvm-versions in the unified toolchainProblem Details
Android introduced a unified tools layout in NDK r19 (2019) and removed the old layout in r22 (2021). Releases r19, r20, and r21 support both the old and new layout side-by-side. More details are in #106270.
Validation
Ran a sub-set of the LLDB API tests against remote Android targets for the four primary architectures i386, x86_64, arm, and aarch64. No validation was done against riscv targets.
For each case, ran the copy of
lldb-serverfrom the Android NDK on the device with the latest LLDB test cases in llvm-projectRan tests with both r19 (the oldest supported) and r26 (more recent, unified layout only) NDK versions.
Example test command for aarch64:
NOTE: there are a lot of test failures when running the full suite (especially against 32-bit ARM target). These failures occur independent of this change.
Verified the expected error message appears when attempting to run using NDK r18
Impact
This change explicitly removes support for the pre-2019 NDK structure. Only NDK r19 (from 2019) and later can be used when running the LLDB API tests. If the maintainers object, we can easily support both the old and new NDK toolchain layouts side-by-side at the cost of readability/maintainability. Since this change only impacts tests, I don't see much value in supporting NDKs that are over 5 years old.
Guidance to Reviewers
clangarguments so if anything looks off let me know.