You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Bazel doesn't support aab generation as it can be seen here: #11497 . Based on this comment #11497 (comment), I tried to modify my build process to generate aab. But I am getting NullPointerException and bazel is crashing due to an internal error. I dont know how to dig deeper to logs. However, I believe my issue is very similar to this issue #12225. Does any one know how to solve this issue?
It is not possible for me to provide an isolated example at the moment.
hairlight_android_application.bzl
"""
Macros pertaining to building & managing Android app bundles.
"""
def _convert_apk_to_aab_module_impl(ctx):
output_file = ctx.outputs.output_file
input_file = ctx.attr.input_file.files.to_list()[0]
# See aapt2 help documentation for details on the arguments passed here.
arguments = [
"convert",
"--output-format",
"proto",
"-o",
output_file.path,
input_file.path,
]
# Reference: https://docs.bazel.build/versions/master/skylark/lib/actions.html#run.
ctx.actions.run(
outputs = [output_file],
inputs = ctx.files.input_file,
tools = [ctx.executable._aapt2_tool],
executable = ctx.executable._aapt2_tool.path,
arguments = arguments,
mnemonic = "GenerateAndroidAppBundleModuleFromApk",
progress_message = "Generating deployable AAB",
)
return DefaultInfo(
files = depset([output_file]),
runfiles = ctx.runfiles(files = [output_file]),
)
def _convert_module_aab_to_structured_zip_impl(ctx):
output_file = ctx.outputs.output_file
input_file = ctx.attr.input_file.files.to_list()[0]
command = """
# Extract AAB to working directory.
WORKING_DIR=$(mktemp -d)
unzip -d $WORKING_DIR {0}
# Create the expected directory structure for an app bundle.
# Reference for copying all other files to root: https://askubuntu.com/a/951768.
mkdir -p $WORKING_DIR/assets $WORKING_DIR/dex $WORKING_DIR/manifest $WORKING_DIR/root
mv $WORKING_DIR/*.dex $WORKING_DIR/dex/
mv $WORKING_DIR/AndroidManifest.xml $WORKING_DIR/manifest/
ls -d $WORKING_DIR/* | grep -v -w -E "res|assets|dex|manifest|root|resources.pb" | xargs -I{{}} sh -c "mv \\$0 $WORKING_DIR/root/ || exit 255" {{}} 2>&1 || exit $?
# Zip up the result--this will be used by bundletool to build a deployable AAB. Note that these
# strange file path bits are needed because zip will always retain the directory structure
# passed via arguments (necessitating changing into the working directory).
DEST_FILE_PATH="$(pwd)/{1}"
cd $WORKING_DIR
zip -r $DEST_FILE_PATH .
""".format(input_file.path, output_file.path)
# Reference: https://docs.bazel.build/versions/main/skylark/lib/actions.html#run_shell.
ctx.actions.run_shell(
outputs = [output_file],
inputs = ctx.files.input_file,
tools = [],
command = command,
mnemonic = "ConvertModuleAabToStructuredZip",
progress_message = "Generating deployable AAB",
)
return DefaultInfo(
files = depset([output_file]),
runfiles = ctx.runfiles(files = [output_file]),
)
def _bundle_module_zip_into_deployable_aab_impl(ctx):
output_file = ctx.outputs.output_file
input_file = ctx.attr.input_file.files.to_list()[0]
config_file = ctx.attr.config_file.files.to_list()[0]
# Reference: https://developer.android.com/studio/build/building-cmdline#build_your_app_bundle_using_bundletool.
arguments = [
"build-bundle",
"--modules=%s" % input_file.path,
"--config=%s" % config_file.path,
"--output=%s" % output_file.path,
]
# Reference: https://docs.bazel.build/versions/master/skylark/lib/actions.html#run.
ctx.actions.run(
outputs = [output_file],
inputs = ctx.files.input_file + ctx.files.config_file,
tools = [ctx.executable._bundletool_tool],
executable = ctx.executable._bundletool_tool.path,
arguments = arguments,
mnemonic = "GenerateDeployAabFromModuleZip",
progress_message = "Generating deployable AAB",
)
return DefaultInfo(
files = depset([output_file]),
runfiles = ctx.runfiles(files = [output_file]),
)
def _package_metadata_into_deployable_aab_impl(ctx):
output_aab_file = ctx.outputs.output_aab_file
input_aab_file = ctx.attr.input_aab_file.files.to_list()[0]
proguard_map_file = ctx.attr.proguard_map_file.files.to_list()[0]
command = """
# Extract deployable AAB to working directory.
WORKING_DIR=$(mktemp -d)
echo $WORKING_DIR
cp {0} $WORKING_DIR/temp.aab || exit 255
# Change the permissions of the AAB copy so that it can be overwritten later.
chmod 755 $WORKING_DIR/temp.aab || exit 255
# Create directory needed for storing bundle metadata.
mkdir -p $WORKING_DIR/BUNDLE-METADATA/com.android.tools.build.obfuscation
# Copy over the Proguard map file.
cp {1} $WORKING_DIR/BUNDLE-METADATA/com.android.tools.build.obfuscation/proguard.map || exit 255
$ Repackage the AAB file into the destination.
DEST_FILE_PATH="$(pwd)/{2}"
cd $WORKING_DIR
zip -Dur temp.aab BUNDLE-METADATA || exit 255
cp temp.aab $DEST_FILE_PATH || exit 255
""".format(input_aab_file.path, proguard_map_file.path, output_aab_file.path)
# Reference: https://docs.bazel.build/versions/main/skylark/lib/actions.html#run_shell.
ctx.actions.run_shell(
outputs = [output_aab_file],
inputs = ctx.files.input_aab_file + ctx.files.proguard_map_file,
tools = [],
command = command,
mnemonic = "PackageMetadataIntoDeployableAAB",
progress_message = "Generating deployable AAB",
)
return DefaultInfo(
files = depset([output_aab_file]),
runfiles = ctx.runfiles(files = [output_aab_file]),
)
def _generate_apks_and_install_impl(ctx):
input_file = ctx.attr.input_file.files.to_list()[0]
debug_keystore_file = ctx.attr.debug_keystore.files.to_list()[0]
apks_file = ctx.actions.declare_file("%s_processed.apks" % ctx.label.name)
deploy_shell = ctx.actions.declare_file("%s_run.sh" % ctx.label.name)
# Reference: https://developer.android.com/studio/command-line/bundletool#generate_apks.
# See also the Bazel BUILD file for the keystore for details on its password and alias.
generate_apks_arguments = [
"build-apks",
"--bundle=%s" % input_file.path,
"--output=%s" % apks_file.path,
"--ks=%s" % debug_keystore_file.path,
"--ks-pass=pass:android",
"--ks-key-alias=androiddebugkey",
"--key-pass=pass:android",
]
# Reference: https://docs.bazel.build/versions/master/skylark/lib/actions.html#run.
ctx.actions.run(
outputs = [apks_file],
inputs = ctx.files.input_file + ctx.files.debug_keystore,
tools = [ctx.executable._bundletool_tool],
executable = ctx.executable._bundletool_tool.path,
arguments = generate_apks_arguments,
mnemonic = "BuildApksFromDeployAab",
progress_message = "Preparing AAB deploy to device",
)
# References: https://github.com/bazelbuild/bazel/issues/7390,
# https://developer.android.com/studio/command-line/bundletool#deploy_with_bundletool, and
# https://docs.bazel.build/versions/main/skylark/rules.html#executable-rules-and-test-rules.
# Note that the bundletool can be executed directly since Bazel creates a wrapper script that
# utilizes its own internal Java toolchain.
ctx.actions.write(
output = deploy_shell,
content = """
#!/bin/sh
{0} install-apks --apks={1}
echo The APK should now be installed
""".format(ctx.executable._bundletool_tool.short_path, apks_file.short_path),
is_executable = True,
)
# Reference for including necessary runfiles for Java:
# https://github.com/bazelbuild/bazel/issues/487#issuecomment-178119424.
runfiles = ctx.runfiles(
files = [
ctx.executable._bundletool_tool,
apks_file,
],
).merge(ctx.attr._bundletool_tool.default_runfiles)
return DefaultInfo(
executable = deploy_shell,
runfiles = runfiles,
)
_convert_apk_to_module_aab = rule(
attrs = {
"input_file": attr.label(
allow_single_file = True,
mandatory = True,
),
"output_file": attr.output(
mandatory = True,
),
"_aapt2_tool": attr.label(
executable = True,
cfg = "host",
default = "@androidsdk//:aapt2_binary",
),
},
implementation = _convert_apk_to_aab_module_impl,
)
_convert_module_aab_to_structured_zip = rule(
attrs = {
"input_file": attr.label(
allow_single_file = True,
mandatory = True,
),
"output_file": attr.output(
mandatory = True,
),
},
implementation = _convert_module_aab_to_structured_zip_impl,
)
_bundle_module_zip_into_deployable_aab = rule(
attrs = {
"input_file": attr.label(
allow_single_file = True,
mandatory = True,
),
"config_file": attr.label(
allow_single_file = True,
mandatory = True,
),
"output_file": attr.output(
mandatory = True,
),
"_bundletool_tool": attr.label(
executable = True,
cfg = "host",
default = "//third_party:android_bundletool",
),
},
implementation = _bundle_module_zip_into_deployable_aab_impl,
)
_package_metadata_into_deployable_aab = rule(
attrs = {
"input_aab_file": attr.label(
allow_single_file = True,
mandatory = True,
),
"proguard_map_file": attr.label(
allow_single_file = True,
mandatory = True,
),
"output_aab_file": attr.output(
mandatory = True,
),
},
implementation = _package_metadata_into_deployable_aab_impl,
)
_generate_apks_and_install = rule(
attrs = {
"input_file": attr.label(
allow_single_file = True,
mandatory = True,
),
"debug_keystore": attr.label(
allow_single_file = True,
mandatory = True,
),
"_bundletool_tool": attr.label(
executable = True,
cfg = "host",
default = "//third_party:android_bundletool",
),
},
executable = True,
implementation = _generate_apks_and_install_impl,
)
def hairlight_android_application(name, config_file, proguard_generate_mapping, **kwargs):
"""
Creates an Android App Bundle (AAB) binary with the specified name and arguments.
Args:
name: str. The name of the Android App Bundle to build. This will corresponding to the name
of the generated .aab file.
config_file: target. The path to the .pb.json bundle configuration file for this build.
proguard_generate_mapping: boolean. Whether to perform a Proguard optimization step &
generate Proguard mapping corresponding to the obfuscation step.
**kwargs: additional arguments. See android_binary for the exact arguments that are
available.
"""
binary_name = "%s_binary" % name
module_aab_name = "%s_module_aab" % name
module_zip_name = "%s_module_zip" % name
deployable_aab_name = "%s_deployable" % name
native.android_binary(
name = binary_name,
tags = ["manual"],
proguard_generate_mapping = proguard_generate_mapping,
**kwargs
)
_convert_apk_to_module_aab(
name = module_aab_name,
input_file = ":%s.apk" % binary_name,
output_file = "%s.aab" % module_aab_name,
tags = ["manual"],
)
_convert_module_aab_to_structured_zip(
name = module_zip_name,
input_file = ":%s.aab" % module_aab_name,
output_file = "%s.zip" % module_zip_name,
tags = ["manual"],
)
# No extra package step is needed if there's no Proguard map file.
_bundle_module_zip_into_deployable_aab(
name = name,
input_file = ":%s.zip" % module_zip_name,
config_file = config_file,
output_file = "%s.aab" % name,
tags = ["manual"],
)
def declare_deployable_application(name, aab_target):
"""
Creates a new target that can be run with 'bazel run' to install an AAB file.
Example:
declare_deployable_application(
name = "install_oppia_prod",
aab_target = "//:oppia_prod",
)
$ bazel run //:install_oppia_prod
This will build (if necessary) and install the correct APK derived from the Android app bundle
on the locally attached emulator or device. Note that this command does not support targeting a
specific device so it will not work if more than one device is available via 'adb devices'.
Args:
name: str. The name of the runnable target to install an AAB file on a local device.
aab_target: target. The target (declared via oppia_android_application) that should be made
installable.
"""
_generate_apks_and_install(
name = name,
input_file = aab_target,
debug_keystore = "@bazel_tools//tools/android:debug_keystore",
tags = ["manual"],
)
Have you found anything relevant by searching the web?
I found this issue #12225 which was due to this commit (81245fd)
Any other information, logs, or outputs that you want to share?
wsluser@DESKTOP-11:/mnt/c/Users/user1/Documents/Misc/Dev/custom-mediapipe-fork$ bazel build --verbose_failures --sandbox_debug -c opt --copt -DMESA_EGL_NO_X11_HEADERS --copt -DEGL_NO_X11 --cache_computed_file_digests=1 -s mediapipe/examples/android/src/java/com/hairlight/hairsegmentation/apps/hairsegmentationgpu:hairlight_application
Starting local Bazel server and connecting to it...
INFO: SHA256 (https://github.com/bazelbuild/rules_foreign_cc/archive/0.1.0.zip) = c2cdcf55ffaf49366725639e45dedd449b8c3fe22b54e31625eb80ce3a240f1e
DEBUG: Rule 'rules_foreign_cc' indicated that a canonical reproducible form can be obtained by modifying arguments sha256 = "c2cdcf55ffaf49366725639e45dedd449b8c3fe22b54e31625eb80ce3a240f1e"
DEBUG: Repository rules_foreign_cc instantiated at:
/mnt/c/Users/user1/Documents/Misc/Dev/custom-mediapipe-fork/WORKSPACE:42:13: in <toplevel>
Repository rule http_archive defined at:
/home/wsluser/.cache/bazel/_bazel_wsluser/27576fa3ed0872646a64b44d0d0f4d1d/external/bazel_tools/tools/build_defs/repo/http.bzl:336:31: in <toplevel>
DEBUG: /home/wsluser/.cache/bazel/_bazel_wsluser/27576fa3ed0872646a64b44d0d0f4d1d/external/org_tensorflow/third_party/repo.bzl:122:14:
Warning: skipping import of repository 'com_google_absl' because it already exists.
DEBUG: /home/wsluser/.cache/bazel/_bazel_wsluser/27576fa3ed0872646a64b44d0d0f4d1d/external/org_tensorflow/third_party/repo.bzl:122:14:
Warning: skipping import of repository 'com_google_benchmark' because it already exists.
DEBUG: /home/wsluser/.cache/bazel/_bazel_wsluser/27576fa3ed0872646a64b44d0d0f4d1d/external/org_tensorflow/third_party/repo.bzl:122:14:
Warning: skipping import of repository 'pybind11_bazel' because it already exists.
DEBUG: /home/wsluser/.cache/bazel/_bazel_wsluser/27576fa3ed0872646a64b44d0d0f4d1d/external/org_tensorflow/third_party/repo.bzl:122:14:
Warning: skipping import of repository 'com_google_protobuf' because it already exists.
DEBUG: /home/wsluser/.cache/bazel/_bazel_wsluser/27576fa3ed0872646a64b44d0d0f4d1d/external/org_tensorflow/third_party/repo.bzl:122:14:
Warning: skipping import of repository 'com_google_googletest' because it already exists.
DEBUG: /home/wsluser/.cache/bazel/_bazel_wsluser/27576fa3ed0872646a64b44d0d0f4d1d/external/org_tensorflow/third_party/repo.bzl:122:14:
Warning: skipping import of repository 'com_github_gflags_gflags' because it already exists.
DEBUG: /home/wsluser/.cache/bazel/_bazel_wsluser/27576fa3ed0872646a64b44d0d0f4d1d/external/org_tensorflow/third_party/repo.bzl:122:14:
Warning: skipping import of repository 'build_bazel_rules_apple' because it already exists.
DEBUG: /home/wsluser/.cache/bazel/_bazel_wsluser/27576fa3ed0872646a64b44d0d0f4d1d/external/org_tensorflow/third_party/repo.bzl:122:14:
Warning: skipping import of repository 'build_bazel_rules_swift' because it already exists.
DEBUG: /home/wsluser/.cache/bazel/_bazel_wsluser/27576fa3ed0872646a64b44d0d0f4d1d/external/org_tensorflow/third_party/repo.bzl:122:14:
Warning: skipping import of repository 'build_bazel_apple_support' because it already exists.
DEBUG: /home/wsluser/.cache/bazel/_bazel_wsluser/27576fa3ed0872646a64b44d0d0f4d1d/external/org_tensorflow/third_party/repo.bzl:122:14:
Warning: skipping import of repository 'xctestrunner' because it already exists.
DEBUG: /home/wsluser/.cache/bazel/_bazel_wsluser/27576fa3ed0872646a64b44d0d0f4d1d/external/org_tensorflow/third_party/repo.bzl:122:14:
Warning: skipping import of repository 'pybind11' because it already exists.
INFO: SHA256 (https://github.com/bazelbuild/rules_cc/archive/main.zip) = 3839996049629e6377abdfd04681ddeeb0cc3db13b9d2ff81bf46700cb4529f7
DEBUG: Rule 'rules_cc' indicated that a canonical reproducible form can be obtained by modifying arguments sha256 = "3839996049629e6377abdfd04681ddeeb0cc3db13b9d2ff81bf46700cb4529f7"
DEBUG: Repository rules_cc instantiated at:
/mnt/c/Users/user1/Documents/Misc/Dev/custom-mediapipe-fork/WORKSPACE:36:13: in <toplevel>
Repository rule http_archive defined at:
/home/wsluser/.cache/bazel/_bazel_wsluser/27576fa3ed0872646a64b44d0d0f4d1d/external/bazel_tools/tools/build_defs/repo/http.bzl:336:31: in <toplevel>
WARNING: /mnt/c/Users/user1/Documents/Misc/Dev/custom-mediapipe-fork/mediapipe/framework/tool/BUILD:182:24: in cc_library rule //mediapipe/framework/tool:field_data_cc_proto: target '//mediapipe/framework/tool:field_data_cc_proto' depends on deprecated target '@com_google_protobuf//:cc_wkt_protos': Only for backward compatibility. Do not use.
WARNING: /mnt/c/Users/user1/Documents/Misc/Dev/custom-mediapipe-fork/mediapipe/framework/BUILD:54:24: in cc_library rule //mediapipe/framework:calculator_cc_proto: target '//mediapipe/framework:calculator_cc_proto' depends on deprecated target '@com_google_protobuf//:cc_wkt_protos': Only for backward compatibility. Do not use.
INFO: SHA256 (https://github.com/opencv/opencv/releases/download/3.4.3/opencv-3.4.3-android-sdk.zip) = cdb0e190c3734edd4052a3535d9e4310af912a9f70a421b1621711942a1028d5
DEBUG: Rule 'android_opencv' indicated that a canonical reproducible form can be obtained by modifying arguments sha256 = "cdb0e190c3734edd4052a3535d9e4310af912a9f70a421b1621711942a1028d5"
DEBUG: Repository android_opencv instantiated at:
/mnt/c/Users/user1/Documents/Misc/Dev/custom-mediapipe-fork/WORKSPACE:219:13: in <toplevel>
Repository rule http_archive defined at:
/home/wsluser/.cache/bazel/_bazel_wsluser/27576fa3ed0872646a64b44d0d0f4d1d/external/bazel_tools/tools/build_defs/repo/http.bzl:336:31: in <toplevel>
Analyzing: target //mediapipe/examples/android/src/java/com/hairlight/hairsegmentation/apps/hairsegmentationgpu:hairlight_application (171 packages loaded, 12182 targets configured)
FATAL: bazel crashed due to an internal error. Printing stack trace:
java.lang.RuntimeException: Unrecoverable error while evaluating node 'ConfiguredTargetKey{label=//mediapipe/examples/android/src/java/com/hairlight/hairsegmentation/apps/hairsegmentationgpu:hairlight_application_binary, config=BuildConfigurationValue.Key[24c3d57be09409ec5b3d2c93fae5528ea0f5267d0a2f463c174db0f2ce0825d8]}' (requested by nodes 'ConfiguredTargetKey{label=//mediapipe/examples/android/src/java/com/hairlight/hairsegmentation/apps/hairsegmentationgpu:hairlight_application_binary.apk, config=BuildConfigurationValue.Key[24c3d57be09409ec5b3d2c93fae5528ea0f5267d0a2f463c174db0f2ce0825d8]}')
at com.google.devtools.build.skyframe.AbstractParallelEvaluator$Evaluate.run(AbstractParallelEvaluator.java:563)
at com.google.devtools.build.lib.concurrent.AbstractQueueVisitor$WrappedRunnable.run(AbstractQueueVisitor.java:398)
at java.base/java.util.concurrent.ForkJoinTask$AdaptedRunnableAction.exec(Unknown Source)
at java.base/java.util.concurrent.ForkJoinTask.doExec(Unknown Source)
at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(Unknown Source)
at java.base/java.util.concurrent.ForkJoinPool.scan(Unknown Source)
at java.base/java.util.concurrent.ForkJoinPool.runWorker(Unknown Source)
at java.base/java.util.concurrent.ForkJoinWorkerThread.run(Unknown Source)
Caused by: java.lang.NullPointerException
at com.google.devtools.build.lib.rules.java.JavaCompilationHelper.usesAnnotationProcessing(JavaCompilationHelper.java:530)
at com.google.devtools.build.lib.rules.java.JavaCompilationHelper.createOutputs(JavaCompilationHelper.java:187)
at com.google.devtools.build.lib.rules.android.AndroidCommon.initJava(AndroidCommon.java:580)
at com.google.devtools.build.lib.rules.android.AndroidCommon.init(AndroidCommon.java:517)
at com.google.devtools.build.lib.rules.android.AndroidBinary.init(AndroidBinary.java:311)
at com.google.devtools.build.lib.rules.android.AndroidBinary.create(AndroidBinary.java:118)
at com.google.devtools.build.lib.rules.android.AndroidBinary.create(AndroidBinary.java:96)
at com.google.devtools.build.lib.analysis.ConfiguredTargetFactory.createRule(ConfiguredTargetFactory.java:385)
at com.google.devtools.build.lib.analysis.ConfiguredTargetFactory.createConfiguredTarget(ConfiguredTargetFactory.java:195)
at com.google.devtools.build.lib.skyframe.SkyframeBuildView.createConfiguredTarget(SkyframeBuildView.java:940)
at com.google.devtools.build.lib.skyframe.ConfiguredTargetFunction.createConfiguredTarget(ConfiguredTargetFunction.java:1031)
at com.google.devtools.build.lib.skyframe.ConfiguredTargetFunction.compute(ConfiguredTargetFunction.java:371)
at com.google.devtools.build.skyframe.AbstractParallelEvaluator$Evaluate.run(AbstractParallelEvaluator.java:477)
The text was updated successfully, but these errors were encountered:
The stack trace seems to point to native.android_binary's use of JavaCompilationHelper. (I'd consider it a Java issue if it was triggered from one of the native java_rules or calls into java_common).
Description of the problem / feature request:
Bazel doesn't support aab generation as it can be seen here: #11497 . Based on this comment #11497 (comment), I tried to modify my build process to generate aab. But I am getting NullPointerException and bazel is crashing due to an internal error. I dont know how to dig deeper to logs. However, I believe my issue is very similar to this issue #12225. Does any one know how to solve this issue?
It is not possible for me to provide an isolated example at the moment.
hairlight_android_application.bzl
BUILD
What operating system are you running Bazel on?
Windows 11 64 bit WSL
What's the output of
bazel info release
?release 4.2.1
What's the output of
git remote get-url origin ; git rev-parse master ; git rev-parse HEAD
?https://github.com/ya332/custom-mediapipe-fork.git
2b442aeeb64b99b866c227ffbdfa0a9a1806c1ba
d652b77bfcb9deb7f09baf623754a9ca548f39e5)
Have you found anything relevant by searching the web?
I found this issue #12225 which was due to this commit (81245fd)
Any other information, logs, or outputs that you want to share?
The text was updated successfully, but these errors were encountered: