Place Swift runtime libraries in a separate directory than the buildtime libraries in the SDK bundle

[marcprux]

In the SDK bundle, all the libraries that are required to be included with a program (like libswiftCore.so) are included in the same folder as the core NDK libraries that are needed for linking (like libc.so). This makes it difficult to choose which libraries that need to be packaged with a Swift program, either as peers for a command-line executable or embedded in an .apk.

Specifically, the SDK folder swift-6.0.2-RELEASE-android-24-0.1.artifactbundle/swift-6.0.2-release-android-24-sdk/android-27c-sysroot/usr/lib/aarch64-linux-android/24 currently contains:

• libBlocksRuntime.so
• libEGL.so
• libFoundation.so
• libFoundationEssentials.so
• libFoundationInternationalization.so
• libFoundationNetworking.so
• libFoundationXML.so
• libGLESv1_CM.so
• libGLESv2.so
• libGLESv3.so
• libOpenMAXAL.so
• libOpenSLES.so
• libXCTest.so
• lib_FoundationICU.so
• libandroid-spawn.so
• libandroid.so
• libc++.so
• libc++_shared.so
• libc.so
• libcamera2ndk.so
• libcharset.so
• libcrypto.so
• libcurl.so
• libdispatch.so
• libdl.so
• libiconv.so
• libjnigraphics.so
• liblog.so
• liblzma.so
• libm.so
• libmediandk.so
• libnghttp2.so
• libnghttp3.so
• libssh2.so
• libssl.so
• libstdc++.so
• libswiftAndroid.so
• libswiftCore.so
• libswiftDispatch.so
• libswiftDistributed.so
• libswiftObservation.so
• libswiftRegexBuilder.so
• libswiftSwiftOnoneSupport.so
• libswiftSynchronization.so
• libswift_Builtin_float.so
• libswift_Concurrency.so
• libswift_Differentiation.so
• libswift_RegexParser.so
• libswift_StringProcessing.so
• libswift_math.so
• libvulkan.so
• libxml2.so
• libz.so

Could we split these into two separate folders, like we do with the individual SDK builds? Maybe with the link dependencies (libEGL.so, libGLESv1_CM.so, libGLESv2.so, libGLESv3.so, libOpenMAXAL.so, libOpenSLES.so, libandroid.so, libc++.so, libc.so, libcamera2ndk.so, libdl.so, libjnigraphics.so, liblog.so, libm.so, libmediandk.so, libstdc++.so, libvulkan.so, libz.so) going in usr/lib/aarch64-linux-android/24/ and the runtime dependencies going in usr/lib/aarch64-linux-android/swift/? We might need to add an extra librarySearchPaths flag to swift-sdk.json, but it seems like that might be the only change that would be needed.

[finagolfin]

What do you foresee that helping with? You had no problem separating the stub libraries from the NDK out from the Swift and Termux libraries that are needed at runtime: why would anyone else?

I chose to put the Swift runtime libraries in usr/lib/aarch64-linux-android/24 because the clang compiler is hard-coded to look in there for Android libraries to link against, so no additional config is needed for Swift. I already put a line in the doc to remove a few NDK stub libraries that might interfere when running the tests.

I understand that it's a lot of libraries to deal with and figure out, but I don't think this single split helps much.

[marcprux]

You had no problem separating the stub libraries from the NDK out

It actually is a bit troublesome – my test runner harness that I'm using for skip android test … manually filters the library based on hardwired library names (e.g., if it is called "libcamera2ndk.so", then don't copy it). It works, but if a future NDK version includes a new .so file that shouldn't be copied, then that will need to be added to the filter list.

It appears to me that the NDK's clang automatically links against both usr/lib/aarch64-linux-android/24 and the parent usr/lib/aarch64-linux-android. Could we put the Swift libraries in the parent directory instead? I did a quick experiment where I manually moved the .so files from the 24/ dir to the parent, and I'm still able to build without making any additional changes to the swift-sdk.json configuration.

[finagolfin]

It appears to me that the NDK's clang automatically links against both usr/lib/aarch64-linux-android/24 and the parent usr/lib/aarch64-linux-android. Could we put the Swift libraries in the parent directory instead?

Yeah, I saw that, thought about it, but figured it was better to place these libraries built against API 24 in that directory.

I will try moving these runtime libraries on the CI and check how well it works, let's see.

[lhoward]

I actually ran into a similar issue today trying to target a later NDK (by overriding target args), then it couldn't link against the Swift runtime libraries. In the end I had to just dlopen() the newer library.

[lhoward]

It would definitely be used to be able to choose the target level at compile time.

[finagolfin]

I actually ran into a similar issue today trying to target a later NDK

You mean a later Android API, and that's a good point. I will look into making this change for the next patch release.

[finagolfin]

Now that 6.0.3 was tagged, I had the CI build an SDK bundle with the runtime libraries moved. Try it out and let me know what you think: I will fix other issues too and post a bundle in the releases in the next week.

[marcprux]

It's working great so far in my initial testing! I just needed to make some tweaks to the tooling the handle the new location for the libraries.