Android Native C++ Libraries Build

[cortinico]

Introduction

The goal of this issue to bring awareness of what is the current status of the Android native builds, express some of the improvements we shipped, and collect feedback on some of the potential direction we might take.

With the new architecture, and specifically with the TurboModule capability (see #40) we expect more users having to deal with C++ code, therefore we believe it's important to bring clarity on this topic.

Details

Currently the react-native project needs to build both Java/Kotlin (app layer) code and C++ (native layer) in order to be shipped and be used by Android developers.
Whenever we release a new version of react-native we:

1. Build all the app layer code using the Java/Kotlin compilers
2. Build all the native code using the Android NDK (specifically we use ndk-build).
3. Bundle all the dynamic libraries (aka the .so files) together with the .class files and so on inside an Android Archive (aka a .aar file) and we place it inside the android/com/facebook/react/react-native/<version> folder of our NPM module.

This will make possible to build a React Native project with Gradle using the aforementioned folder as a Maven Repository (see here).

As long as you don't need to compile C++ code, you will be good with the current setup. If you need to build C++ code instead (say because you want to add a TurboModule or you're developing a react-native libraries that relies on native code), you need to setup a native build with the Android NDK.

Sadly, setting up an Android native build is not as easy as we would like it to be. Here are some of the friction points and potential improvements.

Discussion points

AGP Android NDK Apis

What: The ReactAndroid project is (was) not using the AGP NDK Apis
Who is affected: react-native contributors that are building the Android project.
Status: ✅ Shipped

Historically, the react-native project was using a custom Gradle task to invoke the Android NDK (see here) rather than AGP's API externalNativeBuild.

This had a variety of implications, namely:

• AGP was not aware of us running a native build (i.e. we would have to take care of the cleanups).
• There were a number of cache misses on the Gradle side (i.e. some Gradle tasks were making extensive use of doLast and would have been re-executed at every builds).
• We could not specify variant specific flags for the native build.

Some of the complexity here was that the ReactAndroid native build relies on several third party libraries that should be downloaded and prepared afterwards. We cleaned up the ReactAndroid build with the following PRs that are aimed at extracting those Gradle tasks to separate classes and fix the cache misses:

• Refactor Extract Headers and JNI from AARs to an internal task facebook/react-native#32426
• Export prepareJSC to an internal task facebook/react-native#32427
• Export prepareBoost to an internal task facebook/react-native#32424
• Export prepareLibevent to an internal task facebook/react-native#32425
• Export prepareGlog to an internal task facebook/react-native#32421

The switch to the NDK Apis was done in this PR:

• Update ReactAndroid to use the AGP NDK Apis facebook/react-native#32443

If you are a react-native contributor, please note the implications on the build time (and how to reduce it if its too long for you) in the commit message of #32443

Cmake Support

What: The react-native Android ecosystem is relying on ndk-build.
Who is affected: Any developer which is building a module with native code (e.g. a TurboModule or similar).
Status: 💬 Open for discussion ✅ Shipped in RN 0.70

Currently all the C++ code for Android is built using ndk-build. Google is anyway advertising to do not use ndk-build for new project but prefer Cmake instead (see here).

We haven't yet evaluated what would be the impact of migrating the builds to Cmake and we're happy to collect feedback from the community.

Prefab Support

What: The react-native .aar is not exposing headers/dynamic libraries and is forcing consumers to rebuild everything from source.
Who is affected: Any developer which is building a module with native code (e.g. a TurboModule or similar).
Status: 💬 Open for discussion ✅ Shipped in RN 0.71

AGP 4.1 added the support for publishing prefabs inside an .aar (see here). We're not using prefab in any from at this stage:

Consuming Prefabs

Some of the native libraries we depend on are offering prefab support, but we're not actively using it. An example is fbjni - see facebookincubator/fbjni#35. We're instead relying on a custom Gradle machinery to extract headers and .sos from the fbjni tarball: facebook/react-native#32426

Publishing Prefabs

Publishing prefabs inside the react-native .aar was blocked by the AGP Android NDK Apis task.

We should investigate what would be the impact of shipping all/some of the native dependencies that we're bundling inside the react-native .aar. Specifically users that are adding a TurboModule will need to have the following .so files + related headers: libjsi, libfbjni, libglog, libfolly_json, libyoga, libreact_nativemodule_core, libturbomodulejsijni, librrc_view, libreact_render_core, libreact_render_graphics, libreact_codegen_rncore.

The impact of this in terms of bundle size and build time hasn't been assessed yet.

Specifically if you're a react-native library author you might benefit from depending on react-native as you won't need to fetch and build all the shared libraries (such as Glog, Folly and so on). This has however implications in terms of ABI stability that we haven't yet fully assessed.

Build from source vs build from artifacts.

There has been some discussions and confusions in the community between prefabs and the build from source v. build from artifacts. We would like to clarify that we believe in:

• Sticking ReactAndroid builds to build from source, with all the dependencies code available locally. The Consuming Prefabs point presented earlier should be considered as an improvement to speedup built time and will anyway be an opt-in feature (i.e. you will still be able to build everything from source).
• Delivering the best build experiences to our consumer. Currently react-native is distributed as prebuilt (an .aar) and, as we evolve our framework/architecture, we will try to reduce the build impact on consumers as much as possible while still allowing the flexibility to build from source to some extent (see this wiki page).

Variant Aware Packaging

What: The react-native .aar needs to be manually patched to remove the unnecessary dynamic libaries.
Who is affected: All the react-native users to some extent.
Status: 💬 Open for discussion ✅ Shipped in RN 0.71

We currently rely on the enableVmCleanup flag (enabled by default) to run a cleanup of all the .so files that we suppose they won't be needed in the final APK. For instance if you're building a Debug build using Hermes, you will not need libhermes-executor-debug.so, and so on.

This has to be performed manually with a FileTree.visit during the build which is really error prone. Some alternatives to this approach could be:

• Investigate if this can be moved to use AGP 7's new Artifact API
• Release variant-aware version of the react-native .aar so that they will be matched accordingly to the app build (e.g. a react-native-debug.aar and so on).

The impact of this hasn't been assessed yet.

[likern]

One thing of pain point I've found is relative paths used by native code. They expect to be in exact location while this is not always true. Especially if workspaces are used (which makes packages be symlinked).

So in monorepo native package have different path then when installed as a dependency. It makes native development broken.

See this example
https://github.com/ospfranco/react-native-quick-sqlite/blob/e736497166481bbb14a6a9da1df6691ccbee1204/android/CMakeLists.txt#L37

And experience
mmazzarolo/react-native-universal-monorepo#22 (comment)

Would be very cool to think about this use case (and make yarn v3 completely working).

This would allow testing code under different RN versions easily.
This is example https://github.com/breeffy/react-native-monorepo/tree/main/packages

[cortinico]

Cmake Support

Hey all,
I'd like to share a small update here, as I believe this might have went unnoticed.

Starting from React Native 0.70, we released the full CMake support for Android builds.

This benefits apps on New Architecture. Specifically, it means that:

• You can now use a CMakeLists.txt file instead of an Android.mk file for anything Android/Native related in your project.
• The CMake file to create in your app is way smaller (3 lines of code versus 50+ for Android.mk files) so this makes for a easier update experience between RN versions and less code to maintain on your end.
• The react-native-codegen is generating both Android.mk and CMakeLists.txt, so libraries should not worry about doing anything if they're using the default setup we provide for New Architecture libraries.
• The Auto-linking coming with RN 0.70 will work both CMake and Android.mk files out of the box.
• Apps will be free to use either Android.mk or CMake files, though the recommended solution in the future would be CMake files (specifically because Android.mk are considered legacy by Android).

Moreover, we migrated the whole RN build to CMake.

This comes with a couple of nice benefits:

• In general CMake files are easier to understand than Make files. If you're a contributor, it will be easier for you to understand the build setup.
• CMake in general works better on Windows compared to Android.mk files.
• It will be easier for external tools to plug our build and invoke it, and it will be easier for us to plug external builds (e.g. the Hermes build is in CMake and we can now directly reference it).

EDIT: Typo

[leotm]

awesome ^ think meant

CMake files are easier to understand than Make files

[cortinico]

Hey all,
I'm closing this as all the remaining points were implemented and will be released in React Native 0.71. The main driver was the New Architecture adoption, but this will benefit users on both Old/New Arch.

Prefab Support

In 0.71, the React Native Android artifacts will be prefab capable. This allowed us to:

• Cleanup a lot of unnecessary code from .gradle files in the template
• Let libraries depend direclty on our .so/headers file. For instance, react-native-mmkv and Reanimated are already using those new APIs resulting in a much cleaner & robust build infrastructure (see Add support for React Native 0.71 software-mansion/react-native-reanimated#3745).
• As libraries don't need to re-download native code anymore (like the Glog or Folly .zip file), this will result in a significant reduction of disk usage for users. For instance, the Boost .zip file was occupying ~200Mb for every library that was re-downloading it. This won't be needed anymore.

Build from source vs build from artifacts.

In 0.71, the New Architecture setup for Android will be using prebuilts and not building React Native from source anymore. This allowed us to reduce the build time of Android Apps on New Architecture drastically (5x and more). We had reports of user waiting more than 1h to finish their build. In 0.71, building Android app in Old/New Arch will similar build times.

Variant Aware Packaging

In 0.71, we're distributing a debug and a release prebuilt. This means that:

• The React Native NPM package size is reduced from ~75Mb to ~20Mb (compressed). This will result in faster yarn install for users.
• The Android prebuilts will be downloaded once per React Native version and cached. This means that:
  • React Native users that are not building Android app don't need to download them.
  • Users that are building multiple apps on the same React Native version don't need to re-download artifacts. They also won't be occupying duplicated space in node_modules of our users.
• Users will only download the debug or release artifact, depending on which build they're running.
• We removed the FileTree.visit approach that was mentioned in this original issue and moved to use packagingOption inside the React Native Gradle Plugin. This is a more robust solution and will solve a lot of runtime crashes such as: couldn't find DSO to load: libjscexecutor.so and other library loading crashes.