Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[7.0.0] Fix Java runtime toolchain resolution in cross-compilation scenarios #19900

Merged
merged 3 commits into from
Oct 19, 2023

Conversation

iancha1992
Copy link
Member

The Java rules used toolchains of type @bazel_tools//tools/jdk:runtime_toolchain_type for two different purposes requiring different exec/target platform constraints, which led to issues when cross-compiling: The runtime added to the runfiles of an executable Java target has to have constraints on the target platform, whereas the runtime used to extract the bootclasspath should not have constraints on the target platform. As a result:

  1. @local_jdk did not define any target constraints for its runtime, which allowed the runtime to provide the bootclasspath required by Android rules, but could also end up building java_binary targets that wouldn't run on the target (see Bazel treats local_jdk as a "can run anywhere" toolchain #18265).
  2. Remote JDKs defined a target constraint for the their runtimes, which prevented them from being added to the runfiles of targets they couldn't run on, but broke Android compilation due to the failure to resolve a runtime for bootclasspath extraction (see Bazel 6 Remote JDK Toolchains Incorrect Configuration #17085).

This is resolved by adding a third toolchain type, bootstrap_runtime_toolchain_type, that is only used by the bootclasspath rule (realizes #17085 (comment)). Detailed explanations of the different types and their intended constraints have been added to @bazel_tools//tools/jdk:BUILD.

Addressing the cross-compilation errors then required the following changes:

  1. Direct dependencies on the Java runtime have been removed from rules that do not need them (e.g. android_library and java_binary with create_executable = False).
  2. Implicit dependencies on the Java runtime through the bootclasspath rule have been replaced with dependencies on the bootstrap runtime.
  3. {local,remote}_java_repository now use the user-provided exec constraints of the Java toolchain as the target constraints of the associated runtime.
  4. @local_jdk uses the auto-detected host constraints as exec constraints for the local Java toolchain.

Fixes #17085
Fixes #18265
Fixes bazelbuild/rules_java#64

RELNOTES[INC]: Java runtime toolchains created via local_java_repository from @bazel_tools//tools/jdk:local_java_repository.bzl, which includes local_jdk, now have target_compatible_with set to the auto-detected host constraints. This can result in errors about toolchain resolution failures for @bazel_tools//tools/jdk:runtime_toolchain_type, especially when cross-compiling. These failures can be fixed in the following ways (listed in decreasing order of preference):

  • Replace java_binary targets that aren't meant to be run with bazel run or as tools during the build with java_single_jar (available in @rules_java//java:java_single_jar.bzl). Such targets do not require a Java runtime for the target configuration.
  • Set --java_runtime_version=remotejdk_N for some Java version N to let Bazel choose and download an appropriate remote JDK for the current target platform. This setting defaults to local_jdk, which means that Bazel can only use the local JDK, which isn't compatible with any other platform.
  • Manually define and register a local_java_runtime with no value set for exec_compatible_with (defaults to []) and select it by setting --java_runtime_version to its name. This fully restores the previous behavior, but can result in incorrect results when cross-compiling (see Bazel treats local_jdk as a "can run anywhere" toolchain #18265).

Closes #18262.

Commit f79ca02

PiperOrigin-RevId: 574914446
Change-Id: I6cbfb7ffa2fbfd62e5f6fb49532b36be658dfa40

The Java rules used toolchains of type `@bazel_tools//tools/jdk:runtime_toolchain_type` for two different purposes requiring different exec/target platform constraints, which led to issues when cross-compiling: The runtime added to the runfiles of an executable Java target has to have constraints on the target platform, whereas the runtime used to extract the bootclasspath should not have constraints on the target platform. As a result:
1. `@local_jdk` did not define any target constraints for its runtime, which allowed the runtime to provide the bootclasspath required by Android rules, but could also end up building `java_binary` targets that wouldn't run on the target (see bazelbuild#18265).
2. Remote JDKs defined a target constraint for the their runtimes, which prevented them from being added to the runfiles of targets they couldn't run on, but broke Android compilation due to the failure to resolve a runtime for bootclasspath extraction (see bazelbuild#17085).

This is resolved by adding a third toolchain type, `bootstrap_runtime_toolchain_type`, that is only used by the `bootclasspath` rule (realizes bazelbuild#17085 (comment)). Detailed explanations of the different types and their intended constraints have been added to `@bazel_tools//tools/jdk:BUILD`.

Addressing the cross-compilation errors then required the following changes:
1. Direct dependencies on the Java runtime have been removed from rules that do not need them (e.g. `android_library` and `java_binary` with `create_executable = False`).
3. Implicit dependencies on the Java runtime through the `bootclasspath` rule have been replaced with dependencies on the bootstrap runtime.
4. `{local,remote}_java_repository` now use the user-provided exec constraints of the Java toolchain as the target constraints of the associated runtime.
5. `@local_jdk` uses the auto-detected host constraints as exec constraints for the local Java toolchain.

Fixes bazelbuild#17085
Fixes bazelbuild#18265
Fixes bazelbuild/rules_java#64

RELNOTES[INC]: Java runtime toolchains created via `local_java_repository` from `@bazel_tools//tools/jdk:local_java_repository.bzl`, which includes `local_jdk`, now have `target_compatible_with` set to the auto-detected host constraints. This can result in errors about toolchain resolution failures for `@bazel_tools//tools/jdk:runtime_toolchain_type`, especially when cross-compiling. These failures can be fixed in the following ways (listed in decreasing order of preference):
* Replace `java_binary` targets that aren't meant to be run with `bazel run` or as tools during the build with `java_single_jar` (available in `@rules_java//java:java_single_jar.bzl`). Such targets do not require a Java runtime for the target configuration.
* Set `--java_runtime_version=remotejdk_N` for some Java version `N` to let Bazel choose and download an appropriate remote JDK for the current target platform. This setting defaults to `local_jdk`, which means that Bazel can only use the local JDK, which isn't compatible with any other platform.
* Manually define and register a `local_java_runtime` with no value set for `exec_compatible_with` (defaults to `[]`) and select it by setting `--java_runtime_version` to its `name`. This fully restores the previous behavior, but can result in incorrect results when cross-compiling (see bazelbuild#18265).

Closes bazelbuild#18262.

PiperOrigin-RevId: 574914446
Change-Id: I6cbfb7ffa2fbfd62e5f6fb49532b36be658dfa40
@iancha1992 iancha1992 added team-Rules-Java Issues for Java rules awaiting-review PR is awaiting review from an assigned reviewer labels Oct 19, 2023
@iancha1992 iancha1992 requested a review from a team as a code owner October 19, 2023 17:36
@github-actions github-actions bot added the team-Android Issues for Android team label Oct 19, 2023
@iancha1992 iancha1992 removed the team-Android Issues for Android team label Oct 19, 2023
@iancha1992 iancha1992 enabled auto-merge (squash) October 19, 2023 17:37
@iancha1992
Copy link
Member Author

@hvadehra can you please take a look at the errors?

@hvadehra
Copy link
Member

I would assume we need to update the lock file again.

$ bazel mod deps --lockfile_mode=update

@iancha1992 iancha1992 merged commit 2cb3159 into bazelbuild:release-7.0.0 Oct 19, 2023
1 check passed
@github-actions github-actions bot removed the awaiting-review PR is awaiting review from an assigned reviewer label Oct 19, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
team-Rules-Java Issues for Java rules
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants