Android maven resource won't build in Bazel #757

CaydenPierce · 2021-11-09T21:05:19Z

Problem

We are trying to use the android maven resource com.alphacephei:vosk-android:0.3.32 within an Android application built by Bazel.

The build is failing when com.alphacephei:vosk-android:0.3.32 is listed as a dependency.

Investigation

The error (below) has to do with vosk-android library not having access to the net.java.dev.jna:jna library during build.

Bazel uses rules_jvm_external to handle maven resources, and this is implemented in our application. I believe rules_jvm_external uses the pom.xml of the Maven resource to determine if a dependency for that library is needed during build. During system build, when com.alphacephei:vosk-android:0.3.32 is desugared in Bazel, net.java.dev.jna:jna library is not made available to it, because net.java.dev.jna:jna is not in com.alphacephei:vosk-android:0.3.32's declared dependencies.

In the Vosk android example jna is imported with Gradle as a dependency of the Android application. However, net.java.dev.jna is not a dependency of the example application (i.e. the users application that imports Vosk), it is a dependency of the vosk-android library, and so should be declared a dependency of that library (in the POM.xml of the Maven artifact).

Config

Bazel uses WORKSPACE file where we import the maven resource:

load("@rules_jvm_external//:defs.bzl", "maven_install")

# Important: there can only be one maven_install rule. Add new maven deps here.
maven_install(
    name = "maven",
    artifacts = [
        "androidx.concurrent:concurrent-futures:1.0.0-alpha03",
        "androidx.lifecycle:lifecycle-common:2.2.0",
        "androidx.annotation:annotation:aar:1.1.0",
        "androidx.appcompat:appcompat:aar:1.1.0-rc01",
        "androidx.camera:camera-core:1.0.0-beta10",
        "androidx.camera:camera-camera2:1.0.0-beta10",
        "androidx.camera:camera-lifecycle:1.0.0-beta10",
        "androidx.constraintlayout:constraintlayout:aar:1.1.3",
        "androidx.core:core:aar:1.1.0-rc03",
        "androidx.legacy:legacy-support-v4:aar:1.0.0",
        "androidx.recyclerview:recyclerview:aar:1.1.0-beta02",
        "androidx.test.espresso:espresso-core:3.1.1",
        "com.github.bumptech.glide:glide:4.11.0",
        "com.google.android.material:material:aar:1.0.0-rc01",
        "com.google.code.findbugs:jsr305:3.0.2",
        "com.google.flogger:flogger-system-backend:0.3.1",
        "com.google.flogger:flogger:0.3.1",
        "com.google.guava:guava:27.0.1-android",
        "com.google.guava:listenablefuture:1.0",
        "junit:junit:4.12",
        "org.hamcrest:hamcrest-library:1.3",
        "net.java.dev.jna:jna:aar:5.8.0",
        "com.alphacephei:vosk-android:0.3.32",
    ],
    repositories = [
        "https://jcenter.bintray.com",
        "https://maven.google.com",
        "https://dl.google.com/dl/android/maven2",
        "https://repo1.maven.org/maven2",
        "https://alphacephei.com/maven/",
    ],
    fetch_sources = True,
    version_conflict_policy = "pinned",
)

The android application Bazel BUILD file, where the application is importing those maven resources:

android_binary(
    name = "wearableai",
    srcs = glob(["*.java", "LandmarksTranslate/*.java"]),
    assets = [
        "//mediapipe/graphs/wearable_ai_gpu:wearable_ai_gpu.binarypb",
        "//mediapipe/modules/face_landmark:face_landmark.tflite",
        "//mediapipe/modules/iris_landmark:iris_landmark.tflite",
        "//mediapipe/modules/hand_landmark:hand_landmark.tflite",
        "//mediapipe/modules/facial_emotion_recognition:face_emotion.tflite",
        "//mediapipe/modules/facial_emotion_recognition:test_emotion.tflite",
        "//mediapipe/modules/facial_emotion_recognition:iris_landmark_two.tflite",
        "//mediapipe/modules/hand_landmark:handedness.txt",
        "//mediapipe/modules/holistic_landmark:hand_recrop.tflite",
        "//mediapipe/modules/pose_detection:pose_detection.tflite",
        "//mediapipe/modules/pose_landmark:pose_landmark_upper_body.tflite",
        "//mediapipe/modules/pose_landmark:pose_landmark_full_body.tflite",
        "//mediapipe/graphs/holistic_tracking:holistic_tracking_gpu.binarypb",
        "//mediapipe/modules/face_detection:face_detection_front.tflite",
    ],
    assets_dir = "",
    manifest = "AndroidManifest.xml",
    resource_files = glob(["res/**"]),
    manifest_values = {
        "applicationId": "com.google.mediapipe.apps.wearableai",
        "appName": "Wearable AI",
        "mainActivity": ".MainActivity",
        "cameraFacingFront": "True",
        "binaryGraphName": "wearable_ai_gpu.binarypb",
        "inputVideoStreamName": "input_video",
        "outputVideoStreamName": "output_video",
        "flipFramesVertically": "True",
        "converterNumBuffers": "2",
    },
    multidex = "native",
    deps = [
        ":mediapipe_jni_lib",
        "//mediapipe/framework/formats:landmark_java_proto_lite",
        "//mediapipe/java/com/google/mediapipe/components:android_camerax_helper",
        "//mediapipe/java/com/google/mediapipe/components:android_components",
        "//mediapipe/java/com/google/mediapipe/framework:android_framework",
        "//mediapipe/java/com/google/mediapipe/glutil",
        "//third_party:androidx_appcompat",
        "//third_party:androidx_core",
        "//third_party:androidx_constraint_layout",
        "//third_party:androidx_material",
        "//third_party:androidx_recyclerview",
        "//third_party:opencv",
        "//third_party:androidx_annotation",
        "@maven//:net_java_dev_jna_jna",
        "@maven//:com_alphacephei_vosk_android",
    ],
)

(The //third_party: dependencies are also Maven, I've just made our own calls directly pull from maven here for clarity.)

Error

Run the command:

bazel build -c opt --config=android_arm64 --verbose_failures mediapipe/examples/android/src/java/com/google/mediapipe/apps/wearableai:wearableai --linkopt=-s

And receive error:

Starting local Bazel server and connecting to it...
INFO: SHA256 (https://github.com/bazelbuild/rules_foreign_cc/archive/refs/tags/0.0.9.zip) = 9f96cae32dca0578960468a5a9f66e1ddb8cba8cc210e3e8198f70a20dba45d1
DEBUG: Rule 'rules_foreign_cc' indicated that a canonical reproducible form can be obtained by modifying arguments sha256 = "9f96cae32dca0578960468a5a9f66e1ddb8cba8cc210e3e8198f70a20dba45d1"
DEBUG: Repository rules_foreign_cc instantiated at:
  /home/cayden/biggem/Tester/WearableIntelligenceSystem/android_smart_phone/main/WORKSPACE:45:13: in <toplevel>
Repository rule http_archive defined at:
  /home/cayden/.cache/bazel/_bazel_cayden/568d99cd4ee70f624f9d528fbac94170/external/bazel_tools/tools/build_defs/repo/http.bzl:336:31: in <toplevel>
INFO: SHA256 (https://github.com/bazelbuild/rules_cc/archive/main.zip) = 7e3caad112850996a9c6d78b9f7593d01c030ca50872884cad35e7caf754497c
DEBUG: Rule 'rules_cc' indicated that a canonical reproducible form can be obtained by modifying arguments sha256 = "7e3caad112850996a9c6d78b9f7593d01c030ca50872884cad35e7caf754497c"
DEBUG: Repository rules_cc instantiated at:
  /home/cayden/biggem/Tester/WearableIntelligenceSystem/android_smart_phone/main/WORKSPACE:33:13: in <toplevel>
Repository rule http_archive defined at:
  /home/cayden/.cache/bazel/_bazel_cayden/568d99cd4ee70f624f9d528fbac94170/external/bazel_tools/tools/build_defs/repo/http.bzl:336:31: in <toplevel>
INFO: Analyzed target //mediapipe/examples/android/src/java/com/google/mediapipe/apps/wearableai:wearableai (180 packages loaded, 12170 targets configured).
INFO: Found 1 target...
ERROR: /home/cayden/.cache/bazel/_bazel_cayden/568d99cd4ee70f624f9d528fbac94170/external/maven/BUILD:898:11: Desugaring external/maven/_aar/com_alphacephei_vosk_android/classes_and_libs_merged.jar for Android failed: Worker process did not return a WorkResponse:

---8<---8<--- Start of log, file at /home/cayden/.cache/bazel/_bazel_cayden/568d99cd4ee70f624f9d528fbac94170/bazel-workers/worker-1-Desugar.log ---8<---8<---
Exception in thread "main" java.lang.NoClassDefFoundError: com/sun/jna/PointerType
	at java.base/java.lang.ClassLoader.defineClass1(Native Method)
	at java.base/java.lang.ClassLoader.defineClass(ClassLoader.java:1016)
	at java.base/java.lang.ClassLoader.defineClass(ClassLoader.java:877)
	at com.google.devtools.build.android.desugar.io.HeaderClassLoader.findClass(HeaderClassLoader.java:67)
	at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:588)
	at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:521)
	at java.base/sun.invoke.util.BytecodeDescriptor.parseSig(BytecodeDescriptor.java:95)
	at java.base/sun.invoke.util.BytecodeDescriptor.parseMethod(BytecodeDescriptor.java:60)
	at java.base/sun.invoke.util.BytecodeDescriptor.parseMethod(BytecodeDescriptor.java:45)
	at java.base/java.lang.invoke.MethodType.fromDescriptor(MethodType.java:1134)
	at java.base/java.lang.invoke.MethodType.fromMethodDescriptorString(MethodType.java:1113)
	at com.google.devtools.build.android.desugar.LambdaDesugaring$InvokedynamicRewriter.visitInvokeDynamicInsn(LambdaDesugaring.java:406)
	at org.objectweb.asm.MethodVisitor.visitInvokeDynamicInsn(MethodVisitor.java:462)
	at org.objectweb.asm.MethodVisitor.visitInvokeDynamicInsn(MethodVisitor.java:462)
	at com.google.devtools.build.android.desugar.strconcat.IndyStringConcatDesugaring$IndifiedStringConcatInvocationConverter.visitInvokeDynamicInsn(IndyStringConcatDesugaring.java:141)
	at org.objectweb.asm.MethodVisitor.visitInvokeDynamicInsn(MethodVisitor.java:462)
	at org.objectweb.asm.ClassReader.readCode(ClassReader.java:2451)
	at org.objectweb.asm.ClassReader.readMethod(ClassReader.java:1491)
	at org.objectweb.asm.ClassReader.accept(ClassReader.java:721)
	at com.google.devtools.build.android.desugar.Desugar.desugarClassesInInput(Desugar.java:549)
	at com.google.devtools.build.android.desugar.Desugar.desugarOneInput(Desugar.java:326)
	at com.google.devtools.build.android.desugar.Desugar.desugar(Desugar.java:246)
	at com.google.devtools.build.android.desugar.Desugar.processRequest(Desugar.java:1019)
	at com.google.devtools.build.android.desugar.Desugar.runPersistentWorker(Desugar.java:979)
	at com.google.devtools.build.android.desugar.Desugar.main(Desugar.java:957)
Caused by: java.lang.ClassNotFoundException: Class com.sun.jna.PointerType not found
	at com.google.devtools.build.android.desugar.io.HeaderClassLoader.findClass(HeaderClassLoader.java:53)
	at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:588)
	at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:521)
	... 25 more
---8<---8<--- End of log ---8<---8<---
Target //mediapipe/examples/android/src/java/com/google/mediapipe/apps/wearableai:wearableai failed to build
INFO: Elapsed time: 208.971s, Critical Path: 124.50s
INFO: 5 processes: 3 internal, 2 linux-sandbox.
FAILED: Build did NOT complete successfully

Potential Solution

Update the com.alphacephei:vosk-android:0.3.32 Maven resource to include net.java.dev.jna:jna as a dependency.

The text was updated successfully, but these errors were encountered:

nshmyrev · 2021-11-09T22:55:55Z

I released 0.3.33 with the fix. Please try and comment if needed.

nshmyrev · 2021-11-09T23:34:47Z

err, 0.3.34 actually

CaydenPierce · 2021-11-10T00:19:43Z

That worked, awesome. Good work and thanks @nshmyrev

Commit a5d788a solved this issue.

For others who want to build Vosk Android with Bazel, the results of my work will be Open Sourced here: https://github.com/emexlabs/WearableIntelligenceSystem

CaydenPierce · 2021-11-10T02:56:36Z

This has caused further errors.

While the above solution DID allow vosk to begin building with Bazel, it created issues down the line. I found these further issues working in Bazel and recreated in the original vosk-android-demo to ensure it wasn't a Bazel issue.

In vosk-android-demo, if we update to package 0.3.34, then we get Build error:

uplicate class com.sun.jna.AltCallingConvention found in modules jetified-jna-5.8.0 (net.java.dev.jna:jna:5.8.0) and jetified-jna-5.8.0-runtime (net.java.dev.jna:jna:5.8.0)
Duplicate class com.sun.jna.Callback found in modules jetified-jna-5.8.0 (net.java.dev.jna:jna:5.8.0) and jetified-jna-5.8.0-runtime (net.java.dev.jna:jna:5.8.0)
Duplicate class com.sun.jna.Callback$UncaughtExceptionHandler found in modules jetified-jna-5.8.0 (net.java.dev.jna:jna:5.8.0) and jetified-jna-5.8.0-runtime (net.java.dev.jna:jna:5.8.0)
Duplicate class com.sun.jna.CallbackParameterContext found in modules jetified-jna-5.8.0 (net.java.dev.jna:jna:5.8.0) and jetified-jna-5.8.0-runtime (net.java.dev.jna:jna:5.8.0)
Duplicate class com.sun.jna.CallbackProxy found in modules jetified-jna-5.8.0 (net.java.dev.jna:jna:5.8.0) and jetified-jna-5.8.0-runtime (net.java.dev.jna:jna:5.8.0)
Duplicate class com.sun.jna.CallbackReference found in modules jetified-jna-5.8.0 (net.java.dev.jna:jna:5.8.0) and jetified-jna-5.8.0-runtime (net.java.dev.jna:jna:5.8.0)
Duplicate class com.sun.jna.CallbackReference$AttachOptions found in modules jetified-jna-5.8.0 (net.java.dev.jna:jna:5.8.0) and jetified-jna-5.8.0-runtime (net.java.dev.jna:jna:5.8.0)
Duplicate class com.sun.jna.CallbackReference$DefaultCallbackProxy found in modules jetified-jna-5.8.0 (net.java.dev.jna:jna:5.8.0) and jetified-jna-5.8.0-runtime (net.java.dev.jna:jna:5.8.0)
Duplicate class com.sun.jna.CallbackReference$NativeFunctionHandler found in modules jetified-jna-5.8.0 (net.java.dev.jna:jna:5.8.0) and jetified-jna-5.8.0-runtime (net.java.dev.jna:jna:5.8.0)
...

To solve this, I comment out the jna dependency in app level build.gradle:

dependencies {
    implementation 'androidx.appcompat:appcompat:1.3.1'
    //implementation 'net.java.dev.jna:jna:5.8.0@aar'
    implementation group: 'com.alphacephei', name: 'vosk-android', version: '0.3.34'
    implementation project(':models')
}

This will now Gradle syncs and builds. However it raises the following runtime error (after Gradle or Bazel builds):

   java.lang.UnsatisfiedLinkError: Native library (com/sun/jna/android-aarch64/libjnidispatch.so) not found in resource path (.)
        at com.sun.jna.Native.loadNativeDispatchLibraryFromClasspath(Native.java:962)
        at com.sun.jna.Native.loadNativeDispatchLibrary(Native.java:922)
        at com.sun.jna.Native.<clinit>(Native.java:190)
        at com.sun.jna.Native.register(Native.java:1641)
        at org.vosk.LibVosk.<clinit>(LibVosk.java:16)
        at org.vosk.LibVosk.setLogLevel(LibVosk.java:58)
        at org.vosk.demo.VoskActivity.onCreate(VoskActivity.java:72)
        at android.app.Activity.performCreate(Activity.java:8006)
        at android.app.Activity.performCreate(Activity.java:7990)
        at android.app.Instrumentation.callActivityOnCreate(Instrumentation.java:1329)
        at android.app.ActivityThread.performLaunchActivity(ActivityThread.java:3584)
        at android.app.ActivityThread.handleLaunchActivity(ActivityThread.java:3775)
        at android.app.servertransaction.LaunchActivityItem.execute(LaunchActivityItem.java:85)
        at android.app.servertransaction.TransactionExecutor.executeCallbacks(TransactionExecutor.java:135)
        at android.app.servertransaction.TransactionExecutor.execute(TransactionExecutor.java:95)
        at android.app.ActivityThread$H.handleMessage(ActivityThread.java:2246)
        at android.os.Handler.dispatchMessage(Handler.java:106)
        at android.os.Looper.loop(Looper.java:233)
        at android.app.ActivityThread.main(ActivityThread.java:8010)
        at java.lang.reflect.Method.invoke(Native Method)
        at com.android.internal.os.RuntimeInit$MethodAndArgsCaller.run(RuntimeInit.java:631)
        at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:978)

CaydenPierce · 2021-11-10T05:01:49Z

If I unzip the built apk and search for *.so files, I see:

 cayden@helterserver  ~/biggem/Tester/WearableIntelligenceSystem/android_smart_phone/main/tmp   localize ● ? ⍟9  find . | grep so                                                                           ✔
./lib/armeabi-v7a/libvosk.so
./lib/armeabi-v7a/libmediapipe_jni.so
./lib/armeabi-v7a/libopencv_java3.so
./res/drawable/abc_text_cursor_material.xml
./com/sun/jna/linux-armel/libjnidispatch.so
./com/sun/jna/linux-x86-64/libjnidispatch.so
./com/sun/jna/linux-aarch64/libjnidispatch.so
./com/sun/jna/linux-ppc/libjnidispatch.so
./com/sun/jna/linux-ppc64le/libjnidispatch.so
./com/sun/jna/sunos-sparcv9/libjnidispatch.so
./com/sun/jna/openbsd-x86/libjnidispatch.so
./com/sun/jna/linux-mips64el/libjnidispatch.so
./com/sun/jna/sunos-x86-64/libjnidispatch.so
./com/sun/jna/openbsd-x86-64/libjnidispatch.so
./com/sun/jna/freebsd-x86/libjnidispatch.so
./com/sun/jna/sunos-x86/libjnidispatch.so
./com/sun/jna/linux-riscv64/libjnidispatch.so
./com/sun/jna/linux-arm/libjnidispatch.so
./com/sun/jna/sunos-sparc/libjnidispatch.so
./com/sun/jna/linux-s390x/libjnidispatch.so
./com/sun/jna/freebsd-x86-64/libjnidispatch.so
./com/sun/jna/linux-x86/libjnidispatch.so
./resources.arsc

My architecture in Bazel is set for armeabi-v7a, and the Gradle system has abiFilters, so why are libjnidispatch.so artifacts only being packaged for desktop!?

CaydenPierce · 2021-11-27T22:07:20Z

For anyone wondering... I did eventually solve this issue. The code will be made public here: https://github.com/emexlabs/WearableIntelligenceSystem and you can reach out to me (caydenpierce.com) if you need a hand.

Thanks again @nshmyrev

nshmyrev added a commit that referenced this issue Nov 9, 2021

Expose JNA dependency. Fix issue #757

7f651e1

nshmyrev closed this as completed Nov 9, 2021

cyhkelvin pushed a commit to cyhkelvin/vosk-api that referenced this issue Jan 23, 2022

Expose JNA dependency. Fix issue alphacep#757

5beb037

nshmyrev mentioned this issue Jun 19, 2022

Android Studio version for build vosk-android-demo #1029

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Android maven resource won't build in Bazel #757

Android maven resource won't build in Bazel #757

CaydenPierce commented Nov 9, 2021

nshmyrev commented Nov 9, 2021

nshmyrev commented Nov 9, 2021

CaydenPierce commented Nov 10, 2021

CaydenPierce commented Nov 10, 2021 •

edited

Loading

CaydenPierce commented Nov 10, 2021

CaydenPierce commented Nov 27, 2021

Android maven resource won't build in Bazel #757

Android maven resource won't build in Bazel #757

Comments

CaydenPierce commented Nov 9, 2021

Problem

Investigation

Config

Error

Potential Solution

nshmyrev commented Nov 9, 2021

nshmyrev commented Nov 9, 2021

CaydenPierce commented Nov 10, 2021

CaydenPierce commented Nov 10, 2021 • edited Loading

CaydenPierce commented Nov 10, 2021

CaydenPierce commented Nov 27, 2021

CaydenPierce commented Nov 10, 2021 •

edited

Loading