Regression with @@rules_java//toolchains:bootstrap_runtime_toolchain_type in 7.10.0 #214
Comments
|
Sorry for my ignorance. That was caused by moving bootstrap_runtime_toolchain_type into rules_java: #210 It was necessary to make rules_java compatible with Bazel 6, which doesn't have this toolchain type defined in @bazel_tools. If we don't have a version of rules_java compatible with Bazel 6, it's quite hard to migrate repos to use newly exposed bzl files. I see there are 2 public uses of the toolchain, rules_graalvm, rules_nixpkg (https://github.com/search?q=bootstrap_runtime_toolchain_type+%28path%3A*.BUILD+OR+path%3A*.bzl%29&type=Code&ref=advsearch&l=&l=). Options seem to be:
|
|
For what it's worth, we do not reference |
|
@protocol7 thanks for the info. Do you have a way to test if #215 fixes it? |
|
@comius I did some arguably ugly local patching and that did not resolve the issue. |
Can you provide a reproducer? |
|
Attaching a reproduction workspace, this is a minimisation of the actual repo in which we're seeing these issues. Note in particular .bazelrc as both of the settings there are required to reproduce. I suspect that the |
|
Argh, I see what's going on. You're missing the last line in: Without the last line, it used to work with rules_java 7.9, because bazel calls it for you from the embedded rules_java_prebuilt. However the embedded repository won't register toolchains for the new toolchain type. |
|
Thanks! That did indeed fix the issue in the repro example, however turns out I minimised it a bit too much. In our real repo, we also configure an extra toolchain. With this, we still get the same error, are we missing some further initialisation that were not previously required? |
|
Could this because of the conflict between toolchain(
name = "bootstrap_runtime_toolchain",
# These constraints are not required for correctness, but prevent fetches of remote JDK for
# different architectures. As every Java compilation toolchain depends on a bootstrap runtime in
# the same configuration, this constraint will not result in toolchain resolution failures.
exec_compatible_with = {target_compatible_with},
target_settings = [":version_or_prefix_version_setting"],
toolchain_type = "@bazel_tools//tools/jdk:bootstrap_runtime_toolchain_type",
toolchain = "{toolchain}",
)$ diff -u rules_java/toolchains/remote_java_repository.bzl rules_java_builtin/toolchains/remote_java_repository.bzl
--- rules_java/toolchains/remote_java_repository.bzl 2024-09-05 11:49:11
+++ rules_java_builtin/toolchains/remote_java_repository.bzl 2034-09-03 10:47:26
@@ -89,7 +89,7 @@
# the same configuration, this constraint will not result in toolchain resolution failures.
exec_compatible_with = {target_compatible_with},
target_settings = [":version_or_prefix_version_setting"],
- toolchain_type = "@rules_java//toolchains:bootstrap_runtime_toolchain_type",
+ toolchain_type = "@bazel_tools//tools/jdk:bootstrap_runtime_toolchain_type",
toolchain = "{toolchain}",
)
""".format(When using |
|
repro2 already fails on rules_java 7.9.1 - and I don't know precisely why. If there's a repro that works on 7.9.1 and fails on 7.10. I'll inspect it more closely. Generally yes, rules from rules_java need to be used and not from @bazel_tools, so that consistent toolchains are used. (I don't see this is the problem with repro2) |
Yeah this makes sense. While we can ensure in our repo we don't use One thing to note is that, we use |
|
The long-term way this will be fixed is by fully migrating to Bzlmod. The built-in rules_java repo is a hack and it's very hard to make changes to its structure. |
|
Is it fair to ask for a patch release to get #215 (it fixes local JDK toolchain), and upgrade rules_java_builtin in bazel to the same version afterwards? It might also be worth having something in the release notes mentioning this potential conflict when using |
mbland
added a commit
to EngFlow/example
that referenced
this issue
Sep 9, 2024
Looks like 7.10.0 has problems on non-macOS platforms: - bazelbuild/rules_java: Regression with @@rules_java//toolchains:bootstrap_runtime_toolchain_type in 7.10.0 #214 bazelbuild/rules_java#214 - bazelbuild/rules_java: Rules java upgrade to 7.10.0 causes issues on Windows #218 bazelbuild/rules_java#218 The latter issue seems to match what we're seeing in CI: - https://github.com/EngFlow/example/actions/runs/10778667925/job/29890522476?pr=339 ```txt ERROR: no such package '@@rules_java~//tools/jdk': BUILD file not found in directory 'tools/jdk' of external repository @@rules_java~. Add a BUILD file to a directory to mark it as a package. ERROR: /home/engflow/.cache/bazel/_bazel_engflow/9a27675a6850b9fc7b6f4bb1996879a4/external/rules_java~~toolchains~local_jdk/BUILD.bazel:18:10: no such package '@@rules_java~//tools/jdk': BUILD file not found in directory 'tools/jdk' of external repository @@rules_java~. Add a BUILD file to a directory to mark it as a package. and referenced by '@@rules_java~~toolchains~local_jdk//:bootstrap_runtime_toolchain_definition' ``` Signed-off-by: Mike Bland <[email protected]>
mbland
added a commit
to EngFlow/example
that referenced
this issue
Sep 9, 2024
rules_scala v6.6.0 updated all its repo names to include the version number in the suffix. - https://github.com/bazelbuild/rules_scala/releases/tag/v6.6.0 - bazelbuild/rules_scala: Use version-aware naming for artifact repositories bazelbuild/rules_scala#1573 Hence, now all the repos and toolchains are appended with the scala version suffix. My next PR will update the module extensions to enable setting our own Scala version. The intention is to eventually try to contribute these extensions (and other necessary changes) upstream. --------- Also, I had to pull rules_java back to 7.9.1 because 7.10.0 has problems on non-macOS platforms: - bazelbuild/rules_java: Regression with @@rules_java//toolchains:bootstrap_runtime_toolchain_type in 7.10.0 bazelbuild/rules_java#214 - bazelbuild/rules_java: Rules java upgrade to 7.10.0 causes issues on Windows bazelbuild/rules_java#218 The latter issue seems to match what we're seeing in CI: - https://github.com/EngFlow/example/actions/runs/10778667925/job/29890522476?pr=339 ```txt ERROR: no such package '@@rules_java~//tools/jdk': BUILD file not found in directory 'tools/jdk' of external repository @@rules_java~. Add a BUILD file to a directory to mark it as a package. ERROR: OUTPUT_BASE/external/rules_java~~toolchains~local_jdk/BUILD.bazel:18:10: no such package '@@rules_java~//tools/jdk': BUILD file not found in directory 'tools/jdk' of external repository @@rules_java~. Add a BUILD file to a directory to mark it as a package. and referenced by '@@rules_java~~toolchains~local_jdk//:bootstrap_runtime_toolchain_definition' ``` --------- Signed-off-by: Mike Bland <[email protected]>
|
May be fixed in 7.11.1? |
@honnix Could you please try this version and let us know if the issue persists? |
|
@hvadehra The original problem was solved by not using The local JDK toolchain issue is fixed by 7.11.1 that we can confirm. I think we can close this. Do you agree @protocol7 ? |
|
Yes, agreed. |
mbland
added a commit
to mbland/rules_scala
that referenced
this issue
Nov 30, 2024
Ensures we get all the versions of dependencies we want in `WORKSPACE`, while providing a new API to consumers. Part of bazelbuild#1652. Bumps several packages as high as they can go and still be compatible with Bazel 6 and 7: - `bazel_skylib`: 1.4.1 => 1.7.1 - `rules_cc`: 0.0.6 => 0.0.9 - `rules_python`: 0.36.0 => 0.38.0 - `rules_go`: 0.50.1 The following packages are at the maximum version to prevent breakages under Bazel 6.5.0. --- `abseil-cpp` and `protobuf` have to stay at 20220623.1 and v27.1, respectively, for Bazel 6 compatibility per bazelbuild#1647. `protobuf` up to v25.5 is compatible with Bazel 6 provided users set the compiler flags mentioned in that issue. --- `rules_python` 0.38.0 => 0.39.0 requires at least `rules_cc` 0.0.10, which introduced `cc/common/cc_info.bzl`: ```txt $ bazel build //{src,test,third_party,scala_proto}/... [ ...snip... ] ERROR: error loading package under directory 'test': error loading package 'test': at .../external/rules_python/python/defs.bzl:17:6: at .../external/rules_python/python/py_binary.bzl:18:6: at .../external/rules_python/python/private/py_binary_macro.bzl:16:6: at .../external/rules_python/python/private/common_bazel.bzl:19:6: Label '@rules_cc//cc/common:cc_info.bzl' is invalid because 'cc/common' is not a package; perhaps you meant to put the colon here: '@rules_cc//cc:common/cc_info.bzl'? ``` --- `rules_cc` 0.0.9 => 0.10.0 requires Bazel 7, which defines `CcSharedLibraryHintInfo`: ```txt $ bazel build //{src,test,third_party,scala_proto}/... ERROR: .../external/rules_cc/cc/private/rules_impl/native.bzl:40:33: name 'CcSharedLibraryHintInfo' is not defined (did you mean 'CcSharedLibraryInfo'?) [ ...snip... ] ERROR: error loading package under directory 'test': error loading package 'test': at .../external/rules_python/python/defs.bzl:17:6: at .../external/rules_python/python/py_binary.bzl:18:6: at .../external/rules_python/python/private/py_binary_macro.bzl:16:6: at .../external/rules_python/python/private/common_bazel.bzl:18:6: at .../external/rules_cc/cc/common/cc_common.bzl:17:6: compilation of module 'cc/private/rules_impl/native.bzl' failed ``` --- `rules_java` remains at 7.9.0 due to the more complicated situation it presents. The current version that could be compatible with Bazel 6 and 7 is 7.12.2. As it turns out, we could include `rules_java` versions up to and including 7.12.2 if we don't call the following in `WORKSPACE`: ```py load("@rules_java//java:repositories.bzl", "rules_java_dependencies", "rules_java_toolchains") rules_java_dependencies() rules_java_toolchains() ``` In fact, __we don't do that today__. When we do, Bazel 7.4.1 builds successfully, but Bazel 6.5.2 fails with: ```txt $ bazel build //{src,test,third_party,scala_proto}/... ERROR: .../external/rules_java/toolchains/java_toolchain_alias.bzl:83:34: Use of Starlark transition without allowlist attribute '_allowlist_function_transition'. See Starlark transitions documentation for details and usage: @rules_java//toolchains:java_toolchain_alias.bzl NORMAL ERROR: .../src/java/io/bazel/rulesscala/coverage/instrumenter/BUILD:3:12: While resolving toolchains for target //src/java/io/bazel/rulesscala/coverage/instrumenter:instrumenter: invalid registered toolchain '//toolchains:all': while parsing '//toolchains:all': error loading package '@rules_java//toolchains': initialization of module 'toolchains/java_toolchain_alias.bzl' failed ERROR: Analysis of target '//src/java/io/bazel/rulesscala/coverage/instrumenter:instrumenter' failed; build aborted: ``` When we bump to rules_java 7.10.0, which contains bazelbuild/rules_java#210, Bazel 7.4.1 fails with: ```txt $ bazel build //{src,test,third_party,scala_proto}/... ERROR: .../external/rules_java_builtin/toolchains/BUILD:254:14: While resolving toolchains for target @@rules_java_builtin//toolchains:platformclasspath (096dcc8): No matching toolchains found for types @@bazel_tools//tools/jdk:bootstrap_runtime_toolchain_type. To debug, rerun with --toolchain_resolution_debug='@@bazel_tools//tools/jdk:bootstrap_runtime_toolchain_type' If platforms or toolchains are a new concept for you, we'd encourage reading https://bazel.build/concepts/platforms-intro. ERROR: Analysis of target '//test/toolchains:java21_toolchain' failed; build aborted: Analysis failed ``` This is the error I described in bazelbuild#1619, under "Bump to rules_java 7.9.0 for Bazel 7 compatibility" in the message for commit cd22d88. Today I rediscovered: - bazelbuild/rules_java: Regression with @@rules_java//toolchains:bootstrap_runtime_toolchain_type in 7.10.0 bazelbuild/rules_java#214 But even worse, Bazel 6.5.0 fails with: ```txt $ bazel build //{src,test,third_party,scala_proto}/... [ ...snip... ] ERROR: .../test/src/main/resources/java_sources/BUILD:5:14: Building test/src/main/resources/java_sources/CompiledWithJava8_java.jar (1 source file) failed: Worker process did not return a WorkResponse: ---8<---8<--- Start of log, file at .../bazel-workers/multiplex-worker-18-Javac.log ---8<---8<--- Error thrown by worker thread, shutting down worker. java.lang.NoSuchMethodError: 'java.lang.Iterable com.google.devtools.build.buildjar.javac.BlazeJavacMain$ClassloaderMaskingFileManager.getJavaFileObjectsFromPaths(java.util.Collection)' at com.google.devtools.build.buildjar.javac.BlazeJavacMain.compile(BlazeJavacMain.java:142) at com.google.devtools.build.buildjar.ReducedClasspathJavaLibraryBuilder.compileSources(ReducedClasspathJavaLibraryBuilder.java:57) at com.google.devtools.build.buildjar.SimpleJavaLibraryBuilder.compileJavaLibrary(SimpleJavaLibraryBuilder.java:110) at com.google.devtools.build.buildjar.SimpleJavaLibraryBuilder.run(SimpleJavaLibraryBuilder.java:118) at com.google.devtools.build.buildjar.BazelJavaBuilder.build(BazelJavaBuilder.java:111) at com.google.devtools.build.buildjar.BazelJavaBuilder.parseAndBuild(BazelJavaBuilder.java:91) at com.google.devtools.build.buildjar.BazelJavaBuilder.lambda$main$0(BazelJavaBuilder.java:52) at com.google.devtools.build.lib.worker.WorkRequestHandler$WorkRequestCallback.apply(WorkRequestHandler.java:252) at com.google.devtools.build.lib.worker.WorkRequestHandler.respondToRequest(WorkRequestHandler.java:480) at com.google.devtools.build.lib.worker.WorkRequestHandler.lambda$startResponseThread$1(WorkRequestHandler.java:433) at java.base/java.lang.Thread.run(Thread.java:829) ---8<---8<--- End of log ---8<---8<--- ERROR: .../test/src/main/resources/java_sources/BUILD:11:14: Building test/src/main/resources/java_sources/CompiledWithJava11_java.jar (1 source file) failed: Worker process did not return a WorkResponse: ---8<---8<--- Start of log, file at .../bazel-workers/multiplex-worker-18-Javac.log ---8<---8<--- Error thrown by worker thread, shutting down worker. java.lang.NoSuchMethodError: 'java.lang.Iterable com.google.devtools.build.buildjar.javac.BlazeJavacMain$ClassloaderMaskingFileManager.getJavaFileObjectsFromPaths(java.util.Collection)' at com.google.devtools.build.buildjar.javac.BlazeJavacMain.compile(BlazeJavacMain.java:142) at com.google.devtools.build.buildjar.ReducedClasspathJavaLibraryBuilder.compileSources(ReducedClasspathJavaLibraryBuilder.java:57) at com.google.devtools.build.buildjar.SimpleJavaLibraryBuilder.compileJavaLibrary(SimpleJavaLibraryBuilder.java:110) at com.google.devtools.build.buildjar.SimpleJavaLibraryBuilder.run(SimpleJavaLibraryBuilder.java:118) at com.google.devtools.build.buildjar.BazelJavaBuilder.build(BazelJavaBuilder.java:111) at com.google.devtools.build.buildjar.BazelJavaBuilder.parseAndBuild(BazelJavaBuilder.java:91) at com.google.devtools.build.buildjar.BazelJavaBuilder.lambda$main$0(BazelJavaBuilder.java:52) at com.google.devtools.build.lib.worker.WorkRequestHandler$WorkRequestCallback.apply(WorkRequestHandler.java:252) at com.google.devtools.build.lib.worker.WorkRequestHandler.respondToRequest(WorkRequestHandler.java:480) at com.google.devtools.build.lib.worker.WorkRequestHandler.lambda$startResponseThread$1(WorkRequestHandler.java:433) at java.base/java.lang.Thread.run(Thread.java:829) ---8<---8<--- End of log ---8<---8<--- ``` In bazelbuild#1619, I thought we wanted to keep `@bazel_tools//tools/jdk:` as the canonical Java toolchains package, instead of `@rules_java//toolchains:`. But based on this comment from @fmeum, I'm inclined to believe switching to `@rules_java//toolchains:` actually is the preferred, futureproof approach: - bazelbuild/rules_java#214 (comment) --- Bazel 8 requires `rules_java` 8, per bazelbuild#1652. We can't update to `rules_java` 8 right now, unless we decided to do the following: - Tell Bazel 6 users to add C++ compiler flags to support the newer `abseil-cpp` versions required by newer `protobuf` versions. - Abandon Bazel 6 support in favor of supporting Bazel 7 at a minimum. - In addition to either of the above cases, update ScalaPB to 1.0.0-alpha.1 or higher to support `protobuf` v28. Here are the details explaining why `rules_java` 8 currently breaks. `rules_java` 8.0.0 requires Bazel >= 7.3.2, which provides the `subrule` API. Compatibility with Bazel >= 6.3.0 isn't restored until `rules_java` 8.3.2. ```txt $ bazel build //{src,test,third_party,scala_proto}/... ERROR: .../external/rules_java/java/common/rules/android_lint.bzl:142:24: name 'subrule' is not defined (did you mean 'rule'?) ERROR: Error computing the main repository mapping: at .../scala/private/extensions/dev_deps.bzl:8:6: at .../external/rules_java/java/repositories.bzl:20:6: at .../external/rules_java/toolchains/local_java_repository.bzl:17:6: at .../external/rules_java/java/defs.bzl:18:6: at .../external/rules_java/java/java_library.bzl:16:6: at .../external/rules_java/java/bazel/rules/bazel_java_library.bzl:21:6: compilation of module 'java/common/rules/android_lint.bzl' failed ``` `rules_java` 8.3.0 is broken, as it can't find its own `@compatibility_proxy` repo: ```txt $ bazel build //{src,test,third_party,scala_proto}/... ERROR: error loading package under directory 'src': error loading package 'src/protobuf/io/bazel/rules_scala': at .../external/rules_java/java/defs.bzl:22:6: at .../external/rules_java/java/java_test.bzl:16:6: Unable to find package for @compatibility_proxy//:proxy.bzl: The repository '@compatibility_proxy' could not be resolved: Repository '@compatibility_proxy' is not defined. ``` `rules_java` 8.3.1 seems to fix this, presumably by importing the `protobuf` repo as `com_google_protobuf`. However, it now requires at least `protobuf` v27.0, which adds `bazel/java/lite_proto_library.bzl`. Per bazelbuild#1647, we'd have to update to ScalaPB 1.0.0-alpha.1 to support `protobuf` v28, abandoning users of previous `protobuf` versions or forcing them to upgrade. ```txt $ bazel build //{src,test,third_party,scala_proto}/... [...snip...] ERROR: error loading package under directory 'src': error loading package 'src/java/io/bazel/rulesscala/worker': at .../external/rules_java/java/defs.bzl:16:6: Label '@com_google_protobuf//bazel:java_lite_proto_library.bzl' is invalid because 'bazel' is not a package; perhaps you meant to put the colon here: '@com_google_protobuf//:bazel/java_lite_proto_library.bzl'? ```
mbland
added a commit
to mbland/rules_scala
that referenced
this issue
Nov 30, 2024
Updates almost all dependencies on `@bazel_tools//tools/jdk:` toolchain targets and `.bzl` files to corresponding targets and files in `@rules_java//toolchains:`. Part of bazelbuild#1482 and bazelbuild#1652. All dependencies on `@bazel_tools//tools/jdk:toolchain_type` remain, however, as there is not yet a corresponding target in `@rules_java//toolchains:`. Adds the `WORKSPACE` stanza recommended in the `rules_java` release notes, and removes our own calls to instantiate `rules_java` repos and to register `rules_java` toolchains. These changes are required to avoid build breakages when updating to `rules_scala` 7.10.0 and higher. All targets build and all tests pass under Bazel 6.5.0 and 7.4.1. --- I was on the right track in my analysis from bazelbuild#1619 ("Bump to rules_java 7.9.0 for Bazel 7 compatibility" in the message for commit cd22d88). However, I thought we _shouldn't_ update any targets from `@bazel_tools//tools/jdk:` to `@rules_java//toolchains:`. That's why I thought we were stuck on `rules_java` 7.9.0. However, this comment from @fmeum made me think switching to `@rules_java//toolchains:` actually is the preferred approach: - bazelbuild/rules_java#214 (comment) So this is a potentially breaking change, but in the good kind of way, in that it requires an easy, future proof update.
mbland
added a commit
to mbland/rules_scala
that referenced
this issue
Dec 1, 2024
Ensures we get all the versions of dependencies we want in `WORKSPACE`, while providing a new API to consumers. Part of bazelbuild#1652. Bumps several packages as high as they can go and still be compatible with Bazel 6 and 7: - `bazel_skylib`: 1.4.1 => 1.7.1 - `rules_cc`: 0.0.6 => 0.0.9 - `rules_python`: 0.36.0 => 0.38.0 - `rules_go`: 0.50.1 The following packages are at the maximum version to prevent breakages under Bazel 6.5.0. --- `abseil-cpp` and `protobuf` have to stay at 20220623.1 and v27.1, respectively, for Bazel 6 compatibility per bazelbuild#1647. `protobuf` up to v25.5 is compatible with Bazel 6 provided users set the compiler flags mentioned in that issue. --- `rules_python` 0.38.0 => 0.39.0 requires at least `rules_cc` 0.0.10, which introduced `cc/common/cc_info.bzl`: ```txt $ bazel build //{src,test,third_party,scala_proto}/... [ ...snip... ] ERROR: error loading package under directory 'test': error loading package 'test': at .../external/rules_python/python/defs.bzl:17:6: at .../external/rules_python/python/py_binary.bzl:18:6: at .../external/rules_python/python/private/py_binary_macro.bzl:16:6: at .../external/rules_python/python/private/common_bazel.bzl:19:6: Label '@rules_cc//cc/common:cc_info.bzl' is invalid because 'cc/common' is not a package; perhaps you meant to put the colon here: '@rules_cc//cc:common/cc_info.bzl'? ``` --- `rules_cc` 0.0.9 => 0.10.0 requires Bazel 7, which defines `CcSharedLibraryHintInfo`: ```txt $ bazel build //{src,test,third_party,scala_proto}/... ERROR: .../external/rules_cc/cc/private/rules_impl/native.bzl:40:33: name 'CcSharedLibraryHintInfo' is not defined (did you mean 'CcSharedLibraryInfo'?) [ ...snip... ] ERROR: error loading package under directory 'test': error loading package 'test': at .../external/rules_python/python/defs.bzl:17:6: at .../external/rules_python/python/py_binary.bzl:18:6: at .../external/rules_python/python/private/py_binary_macro.bzl:16:6: at .../external/rules_python/python/private/common_bazel.bzl:18:6: at .../external/rules_cc/cc/common/cc_common.bzl:17:6: compilation of module 'cc/private/rules_impl/native.bzl' failed ``` --- `rules_java` remains at 7.9.0 due to the more complicated situation it presents. The current version that could be compatible with Bazel 6 and 7 is 7.12.2. As it turns out, we could include `rules_java` versions up to and including 7.12.2 if we don't call the following in `WORKSPACE`: ```py load("@rules_java//java:repositories.bzl", "rules_java_dependencies", "rules_java_toolchains") rules_java_dependencies() rules_java_toolchains() ``` In fact, __we don't do that today__. When we do, Bazel 7.4.1 builds successfully, but Bazel 6.5.2 fails with: ```txt $ bazel build //{src,test,third_party,scala_proto}/... ERROR: .../external/rules_java/toolchains/java_toolchain_alias.bzl:83:34: Use of Starlark transition without allowlist attribute '_allowlist_function_transition'. See Starlark transitions documentation for details and usage: @rules_java//toolchains:java_toolchain_alias.bzl NORMAL ERROR: .../src/java/io/bazel/rulesscala/coverage/instrumenter/BUILD:3:12: While resolving toolchains for target //src/java/io/bazel/rulesscala/coverage/instrumenter:instrumenter: invalid registered toolchain '//toolchains:all': while parsing '//toolchains:all': error loading package '@rules_java//toolchains': initialization of module 'toolchains/java_toolchain_alias.bzl' failed ERROR: Analysis of target '//src/java/io/bazel/rulesscala/coverage/instrumenter:instrumenter' failed; build aborted: ``` When we bump to rules_java 7.10.0, which contains bazelbuild/rules_java#210, Bazel 7.4.1 fails with: ```txt $ bazel build //{src,test,third_party,scala_proto}/... ERROR: .../external/rules_java_builtin/toolchains/BUILD:254:14: While resolving toolchains for target @@rules_java_builtin//toolchains:platformclasspath (096dcc8): No matching toolchains found for types @@bazel_tools//tools/jdk:bootstrap_runtime_toolchain_type. To debug, rerun with --toolchain_resolution_debug='@@bazel_tools//tools/jdk:bootstrap_runtime_toolchain_type' If platforms or toolchains are a new concept for you, we'd encourage reading https://bazel.build/concepts/platforms-intro. ERROR: Analysis of target '//test/toolchains:java21_toolchain' failed; build aborted: Analysis failed ``` This is the error I described in bazelbuild#1619, under "Bump to rules_java 7.9.0 for Bazel 7 compatibility" in the message for commit cd22d88. Today I rediscovered: - bazelbuild/rules_java: Regression with @@rules_java//toolchains:bootstrap_runtime_toolchain_type in 7.10.0 bazelbuild/rules_java#214 But even worse, Bazel 6.5.0 fails with: ```txt $ bazel build //{src,test,third_party,scala_proto}/... [ ...snip... ] ERROR: .../test/src/main/resources/java_sources/BUILD:5:14: Building test/src/main/resources/java_sources/CompiledWithJava8_java.jar (1 source file) failed: Worker process did not return a WorkResponse: ---8<---8<--- Start of log, file at .../bazel-workers/multiplex-worker-18-Javac.log ---8<---8<--- Error thrown by worker thread, shutting down worker. java.lang.NoSuchMethodError: 'java.lang.Iterable com.google.devtools.build.buildjar.javac.BlazeJavacMain$ClassloaderMaskingFileManager.getJavaFileObjectsFromPaths(java.util.Collection)' at com.google.devtools.build.buildjar.javac.BlazeJavacMain.compile(BlazeJavacMain.java:142) at com.google.devtools.build.buildjar.ReducedClasspathJavaLibraryBuilder.compileSources(ReducedClasspathJavaLibraryBuilder.java:57) at com.google.devtools.build.buildjar.SimpleJavaLibraryBuilder.compileJavaLibrary(SimpleJavaLibraryBuilder.java:110) at com.google.devtools.build.buildjar.SimpleJavaLibraryBuilder.run(SimpleJavaLibraryBuilder.java:118) at com.google.devtools.build.buildjar.BazelJavaBuilder.build(BazelJavaBuilder.java:111) at com.google.devtools.build.buildjar.BazelJavaBuilder.parseAndBuild(BazelJavaBuilder.java:91) at com.google.devtools.build.buildjar.BazelJavaBuilder.lambda$main$0(BazelJavaBuilder.java:52) at com.google.devtools.build.lib.worker.WorkRequestHandler$WorkRequestCallback.apply(WorkRequestHandler.java:252) at com.google.devtools.build.lib.worker.WorkRequestHandler.respondToRequest(WorkRequestHandler.java:480) at com.google.devtools.build.lib.worker.WorkRequestHandler.lambda$startResponseThread$1(WorkRequestHandler.java:433) at java.base/java.lang.Thread.run(Thread.java:829) ---8<---8<--- End of log ---8<---8<--- ERROR: .../test/src/main/resources/java_sources/BUILD:11:14: Building test/src/main/resources/java_sources/CompiledWithJava11_java.jar (1 source file) failed: Worker process did not return a WorkResponse: ---8<---8<--- Start of log, file at .../bazel-workers/multiplex-worker-18-Javac.log ---8<---8<--- Error thrown by worker thread, shutting down worker. java.lang.NoSuchMethodError: 'java.lang.Iterable com.google.devtools.build.buildjar.javac.BlazeJavacMain$ClassloaderMaskingFileManager.getJavaFileObjectsFromPaths(java.util.Collection)' at com.google.devtools.build.buildjar.javac.BlazeJavacMain.compile(BlazeJavacMain.java:142) at com.google.devtools.build.buildjar.ReducedClasspathJavaLibraryBuilder.compileSources(ReducedClasspathJavaLibraryBuilder.java:57) at com.google.devtools.build.buildjar.SimpleJavaLibraryBuilder.compileJavaLibrary(SimpleJavaLibraryBuilder.java:110) at com.google.devtools.build.buildjar.SimpleJavaLibraryBuilder.run(SimpleJavaLibraryBuilder.java:118) at com.google.devtools.build.buildjar.BazelJavaBuilder.build(BazelJavaBuilder.java:111) at com.google.devtools.build.buildjar.BazelJavaBuilder.parseAndBuild(BazelJavaBuilder.java:91) at com.google.devtools.build.buildjar.BazelJavaBuilder.lambda$main$0(BazelJavaBuilder.java:52) at com.google.devtools.build.lib.worker.WorkRequestHandler$WorkRequestCallback.apply(WorkRequestHandler.java:252) at com.google.devtools.build.lib.worker.WorkRequestHandler.respondToRequest(WorkRequestHandler.java:480) at com.google.devtools.build.lib.worker.WorkRequestHandler.lambda$startResponseThread$1(WorkRequestHandler.java:433) at java.base/java.lang.Thread.run(Thread.java:829) ---8<---8<--- End of log ---8<---8<--- ``` In bazelbuild#1619, I thought we wanted to keep `@bazel_tools//tools/jdk:` as the canonical Java toolchains package, instead of `@rules_java//toolchains:`. But based on this comment from @fmeum, I'm inclined to believe switching to `@rules_java//toolchains:` actually is the preferred, futureproof approach: - bazelbuild/rules_java#214 (comment) --- Bazel 8 requires `rules_java` 8, per bazelbuild#1652. We can't update to `rules_java` 8 right now, unless we decided to do the following: - Tell Bazel 6 users to add C++ compiler flags to support the newer `abseil-cpp` versions required by newer `protobuf` versions. - Abandon Bazel 6 support in favor of supporting Bazel 7 at a minimum. - In addition to either of the above cases, update ScalaPB to 1.0.0-alpha.1 or higher to support `protobuf` v28. Here are the details explaining why `rules_java` 8 currently breaks. `rules_java` 8.0.0 requires Bazel >= 7.3.2, which provides the `subrule` API. Compatibility with Bazel >= 6.3.0 isn't restored until `rules_java` 8.3.2. ```txt $ bazel build //{src,test,third_party,scala_proto}/... ERROR: .../external/rules_java/java/common/rules/android_lint.bzl:142:24: name 'subrule' is not defined (did you mean 'rule'?) ERROR: Error computing the main repository mapping: at .../scala/private/extensions/dev_deps.bzl:8:6: at .../external/rules_java/java/repositories.bzl:20:6: at .../external/rules_java/toolchains/local_java_repository.bzl:17:6: at .../external/rules_java/java/defs.bzl:18:6: at .../external/rules_java/java/java_library.bzl:16:6: at .../external/rules_java/java/bazel/rules/bazel_java_library.bzl:21:6: compilation of module 'java/common/rules/android_lint.bzl' failed ``` `rules_java` 8.3.0 is broken, as it can't find its own `@compatibility_proxy` repo: ```txt $ bazel build //{src,test,third_party,scala_proto}/... ERROR: error loading package under directory 'src': error loading package 'src/protobuf/io/bazel/rules_scala': at .../external/rules_java/java/defs.bzl:22:6: at .../external/rules_java/java/java_test.bzl:16:6: Unable to find package for @compatibility_proxy//:proxy.bzl: The repository '@compatibility_proxy' could not be resolved: Repository '@compatibility_proxy' is not defined. ``` `rules_java` 8.3.1 seems to fix this, presumably by importing the `protobuf` repo as `com_google_protobuf`. However, it now requires at least `protobuf` v27.0, which adds `bazel/java/lite_proto_library.bzl`. Per bazelbuild#1647, we'd have to update to ScalaPB 1.0.0-alpha.1 to support `protobuf` v28, abandoning users of previous `protobuf` versions or forcing them to upgrade. ```txt $ bazel build //{src,test,third_party,scala_proto}/... [...snip...] ERROR: error loading package under directory 'src': error loading package 'src/java/io/bazel/rulesscala/worker': at .../external/rules_java/java/defs.bzl:16:6: Label '@com_google_protobuf//bazel:java_lite_proto_library.bzl' is invalid because 'bazel' is not a package; perhaps you meant to put the colon here: '@com_google_protobuf//:bazel/java_lite_proto_library.bzl'? ```
mbland
added a commit
to mbland/rules_scala
that referenced
this issue
Dec 1, 2024
Updates almost all dependencies on `@bazel_tools//tools/jdk:` toolchain targets and `.bzl` files to corresponding targets and files in `@rules_java//toolchains:`. Part of bazelbuild#1482 and bazelbuild#1652. All dependencies on `@bazel_tools//tools/jdk:toolchain_type` remain, however, as there is not yet a corresponding target in `@rules_java//toolchains:`. Adds the `WORKSPACE` stanza recommended in the `rules_java` release notes, and removes our own calls to instantiate `rules_java` repos and to register `rules_java` toolchains. These changes are required to avoid build breakages when updating to `rules_scala` 7.10.0 and higher. All targets build and all tests pass under Bazel 6.5.0 and 7.4.1. --- I was on the right track in my analysis from bazelbuild#1619 ("Bump to rules_java 7.9.0 for Bazel 7 compatibility" in the message for commit cd22d88). However, I thought we _shouldn't_ update any targets from `@bazel_tools//tools/jdk:` to `@rules_java//toolchains:`. That's why I thought we were stuck on `rules_java` 7.9.0. However, this comment from @fmeum made me think switching to `@rules_java//toolchains:` actually is the preferred approach: - bazelbuild/rules_java#214 (comment) So this is a potentially breaking change, but in the good kind of way, in that it requires an easy, future proof update.
mbland
added a commit
to mbland/rules_scala
that referenced
this issue
Dec 2, 2024
Ensures we get all the versions of dependencies we want in `WORKSPACE`, while providing a new API to consumers. Part of bazelbuild#1652. Bumps several packages as high as they can go and still be compatible with Bazel 6 and 7: - `bazel_skylib`: 1.4.1 => 1.7.1 - `rules_cc`: 0.0.6 => 0.0.9 - `rules_python`: 0.36.0 => 0.38.0 - `rules_go`: 0.50.1 The following packages are at the maximum version to prevent breakages under Bazel 6.5.0. --- `abseil-cpp` and `protobuf` have to stay at 20220623.1 and v27.1, respectively, for Bazel 6 compatibility per bazelbuild#1647. `protobuf` up to v25.5 is compatible with Bazel 6 provided users set the compiler flags mentioned in that issue. --- `rules_python` 0.38.0 => 0.39.0 requires at least `rules_cc` 0.0.10, which introduced `cc/common/cc_info.bzl`: ```txt $ bazel build //{src,test,third_party,scala_proto}/... [ ...snip... ] ERROR: error loading package under directory 'test': error loading package 'test': at .../external/rules_python/python/defs.bzl:17:6: at .../external/rules_python/python/py_binary.bzl:18:6: at .../external/rules_python/python/private/py_binary_macro.bzl:16:6: at .../external/rules_python/python/private/common_bazel.bzl:19:6: Label '@rules_cc//cc/common:cc_info.bzl' is invalid because 'cc/common' is not a package; perhaps you meant to put the colon here: '@rules_cc//cc:common/cc_info.bzl'? ``` --- `rules_cc` 0.0.9 => 0.10.0 requires Bazel 7, which defines `CcSharedLibraryHintInfo`: ```txt $ bazel build //{src,test,third_party,scala_proto}/... ERROR: .../external/rules_cc/cc/private/rules_impl/native.bzl:40:33: name 'CcSharedLibraryHintInfo' is not defined (did you mean 'CcSharedLibraryInfo'?) [ ...snip... ] ERROR: error loading package under directory 'test': error loading package 'test': at .../external/rules_python/python/defs.bzl:17:6: at .../external/rules_python/python/py_binary.bzl:18:6: at .../external/rules_python/python/private/py_binary_macro.bzl:16:6: at .../external/rules_python/python/private/common_bazel.bzl:18:6: at .../external/rules_cc/cc/common/cc_common.bzl:17:6: compilation of module 'cc/private/rules_impl/native.bzl' failed ``` --- `rules_java` remains at 7.9.0 due to the more complicated situation it presents. The current version that could be compatible with Bazel 6 and 7 is 7.12.2. As it turns out, we could include `rules_java` versions up to and including 7.12.2 if we don't call the following in `WORKSPACE`: ```py load("@rules_java//java:repositories.bzl", "rules_java_dependencies", "rules_java_toolchains") rules_java_dependencies() rules_java_toolchains() ``` In fact, __we don't do that today__. When we do, Bazel 7.4.1 builds successfully, but Bazel 6.5.2 fails with: ```txt $ bazel build //{src,test,third_party,scala_proto}/... ERROR: .../external/rules_java/toolchains/java_toolchain_alias.bzl:83:34: Use of Starlark transition without allowlist attribute '_allowlist_function_transition'. See Starlark transitions documentation for details and usage: @rules_java//toolchains:java_toolchain_alias.bzl NORMAL ERROR: .../src/java/io/bazel/rulesscala/coverage/instrumenter/BUILD:3:12: While resolving toolchains for target //src/java/io/bazel/rulesscala/coverage/instrumenter:instrumenter: invalid registered toolchain '//toolchains:all': while parsing '//toolchains:all': error loading package '@rules_java//toolchains': initialization of module 'toolchains/java_toolchain_alias.bzl' failed ERROR: Analysis of target '//src/java/io/bazel/rulesscala/coverage/instrumenter:instrumenter' failed; build aborted: ``` When we bump to rules_java 7.10.0, which contains bazelbuild/rules_java#210, Bazel 7.4.1 fails with: ```txt $ bazel build //{src,test,third_party,scala_proto}/... ERROR: .../external/rules_java_builtin/toolchains/BUILD:254:14: While resolving toolchains for target @@rules_java_builtin//toolchains:platformclasspath (096dcc8): No matching toolchains found for types @@bazel_tools//tools/jdk:bootstrap_runtime_toolchain_type. To debug, rerun with --toolchain_resolution_debug='@@bazel_tools//tools/jdk:bootstrap_runtime_toolchain_type' If platforms or toolchains are a new concept for you, we'd encourage reading https://bazel.build/concepts/platforms-intro. ERROR: Analysis of target '//test/toolchains:java21_toolchain' failed; build aborted: Analysis failed ``` This is the error I described in bazelbuild#1619, under "Bump to rules_java 7.9.0 for Bazel 7 compatibility" in the message for commit cd22d88. Today I rediscovered: - bazelbuild/rules_java: Regression with @@rules_java//toolchains:bootstrap_runtime_toolchain_type in 7.10.0 bazelbuild/rules_java#214 But even worse, Bazel 6.5.0 fails with: ```txt $ bazel build //{src,test,third_party,scala_proto}/... [ ...snip... ] ERROR: .../test/src/main/resources/java_sources/BUILD:5:14: Building test/src/main/resources/java_sources/CompiledWithJava8_java.jar (1 source file) failed: Worker process did not return a WorkResponse: ---8<---8<--- Start of log, file at .../bazel-workers/multiplex-worker-18-Javac.log ---8<---8<--- Error thrown by worker thread, shutting down worker. java.lang.NoSuchMethodError: 'java.lang.Iterable com.google.devtools.build.buildjar.javac.BlazeJavacMain$ClassloaderMaskingFileManager.getJavaFileObjectsFromPaths(java.util.Collection)' at com.google.devtools.build.buildjar.javac.BlazeJavacMain.compile(BlazeJavacMain.java:142) at com.google.devtools.build.buildjar.ReducedClasspathJavaLibraryBuilder.compileSources(ReducedClasspathJavaLibraryBuilder.java:57) at com.google.devtools.build.buildjar.SimpleJavaLibraryBuilder.compileJavaLibrary(SimpleJavaLibraryBuilder.java:110) at com.google.devtools.build.buildjar.SimpleJavaLibraryBuilder.run(SimpleJavaLibraryBuilder.java:118) at com.google.devtools.build.buildjar.BazelJavaBuilder.build(BazelJavaBuilder.java:111) at com.google.devtools.build.buildjar.BazelJavaBuilder.parseAndBuild(BazelJavaBuilder.java:91) at com.google.devtools.build.buildjar.BazelJavaBuilder.lambda$main$0(BazelJavaBuilder.java:52) at com.google.devtools.build.lib.worker.WorkRequestHandler$WorkRequestCallback.apply(WorkRequestHandler.java:252) at com.google.devtools.build.lib.worker.WorkRequestHandler.respondToRequest(WorkRequestHandler.java:480) at com.google.devtools.build.lib.worker.WorkRequestHandler.lambda$startResponseThread$1(WorkRequestHandler.java:433) at java.base/java.lang.Thread.run(Thread.java:829) ---8<---8<--- End of log ---8<---8<--- ERROR: .../test/src/main/resources/java_sources/BUILD:11:14: Building test/src/main/resources/java_sources/CompiledWithJava11_java.jar (1 source file) failed: Worker process did not return a WorkResponse: ---8<---8<--- Start of log, file at .../bazel-workers/multiplex-worker-18-Javac.log ---8<---8<--- Error thrown by worker thread, shutting down worker. java.lang.NoSuchMethodError: 'java.lang.Iterable com.google.devtools.build.buildjar.javac.BlazeJavacMain$ClassloaderMaskingFileManager.getJavaFileObjectsFromPaths(java.util.Collection)' at com.google.devtools.build.buildjar.javac.BlazeJavacMain.compile(BlazeJavacMain.java:142) at com.google.devtools.build.buildjar.ReducedClasspathJavaLibraryBuilder.compileSources(ReducedClasspathJavaLibraryBuilder.java:57) at com.google.devtools.build.buildjar.SimpleJavaLibraryBuilder.compileJavaLibrary(SimpleJavaLibraryBuilder.java:110) at com.google.devtools.build.buildjar.SimpleJavaLibraryBuilder.run(SimpleJavaLibraryBuilder.java:118) at com.google.devtools.build.buildjar.BazelJavaBuilder.build(BazelJavaBuilder.java:111) at com.google.devtools.build.buildjar.BazelJavaBuilder.parseAndBuild(BazelJavaBuilder.java:91) at com.google.devtools.build.buildjar.BazelJavaBuilder.lambda$main$0(BazelJavaBuilder.java:52) at com.google.devtools.build.lib.worker.WorkRequestHandler$WorkRequestCallback.apply(WorkRequestHandler.java:252) at com.google.devtools.build.lib.worker.WorkRequestHandler.respondToRequest(WorkRequestHandler.java:480) at com.google.devtools.build.lib.worker.WorkRequestHandler.lambda$startResponseThread$1(WorkRequestHandler.java:433) at java.base/java.lang.Thread.run(Thread.java:829) ---8<---8<--- End of log ---8<---8<--- ``` In bazelbuild#1619, I thought we wanted to keep `@bazel_tools//tools/jdk:` as the canonical Java toolchains package, instead of `@rules_java//toolchains:`. But based on this comment from @fmeum, I'm inclined to believe switching to `@rules_java//toolchains:` actually is the preferred, futureproof approach: - bazelbuild/rules_java#214 (comment) --- Bazel 8 requires `rules_java` 8, per bazelbuild#1652. We can't update to `rules_java` 8 right now, unless we decided to do the following: - Tell Bazel 6 users to add C++ compiler flags to support the newer `abseil-cpp` versions required by newer `protobuf` versions. - Abandon Bazel 6 support in favor of supporting Bazel 7 at a minimum. - In addition to either of the above cases, update ScalaPB to 1.0.0-alpha.1 or higher to support `protobuf` v28. Here are the details explaining why `rules_java` 8 currently breaks. `rules_java` 8.0.0 requires Bazel >= 7.3.2, which provides the `subrule` API. Compatibility with Bazel >= 6.3.0 isn't restored until `rules_java` 8.3.2. ```txt $ bazel build //{src,test,third_party,scala_proto}/... ERROR: .../external/rules_java/java/common/rules/android_lint.bzl:142:24: name 'subrule' is not defined (did you mean 'rule'?) ERROR: Error computing the main repository mapping: at .../scala/private/extensions/dev_deps.bzl:8:6: at .../external/rules_java/java/repositories.bzl:20:6: at .../external/rules_java/toolchains/local_java_repository.bzl:17:6: at .../external/rules_java/java/defs.bzl:18:6: at .../external/rules_java/java/java_library.bzl:16:6: at .../external/rules_java/java/bazel/rules/bazel_java_library.bzl:21:6: compilation of module 'java/common/rules/android_lint.bzl' failed ``` `rules_java` 8.3.0 is broken, as it can't find its own `@compatibility_proxy` repo: ```txt $ bazel build //{src,test,third_party,scala_proto}/... ERROR: error loading package under directory 'src': error loading package 'src/protobuf/io/bazel/rules_scala': at .../external/rules_java/java/defs.bzl:22:6: at .../external/rules_java/java/java_test.bzl:16:6: Unable to find package for @compatibility_proxy//:proxy.bzl: The repository '@compatibility_proxy' could not be resolved: Repository '@compatibility_proxy' is not defined. ``` `rules_java` 8.3.1 seems to fix this, presumably by importing the `protobuf` repo as `com_google_protobuf`. However, it now requires at least `protobuf` v27.0, which adds `bazel/java/lite_proto_library.bzl`. Per bazelbuild#1647, we'd have to update to ScalaPB 1.0.0-alpha.1 to support `protobuf` v28, abandoning users of previous `protobuf` versions or forcing them to upgrade. ```txt $ bazel build //{src,test,third_party,scala_proto}/... [...snip...] ERROR: error loading package under directory 'src': error loading package 'src/java/io/bazel/rulesscala/worker': at .../external/rules_java/java/defs.bzl:16:6: Label '@com_google_protobuf//bazel:java_lite_proto_library.bzl' is invalid because 'bazel' is not a package; perhaps you meant to put the colon here: '@com_google_protobuf//:bazel/java_lite_proto_library.bzl'? ```
mbland
added a commit
to mbland/rules_scala
that referenced
this issue
Dec 2, 2024
Updates almost all dependencies on `@bazel_tools//tools/jdk:` toolchain targets and `.bzl` files to corresponding targets and files in `@rules_java//toolchains:`. Part of bazelbuild#1482 and bazelbuild#1652. All dependencies on `@bazel_tools//tools/jdk:toolchain_type` remain, however, as there is not yet a corresponding target in `@rules_java//toolchains:`. Adds the `WORKSPACE` stanza recommended in the `rules_java` release notes, and removes our own calls to instantiate `rules_java` repos and to register `rules_java` toolchains. These changes are required to avoid build breakages when updating to `rules_scala` 7.10.0 and higher. All targets build and all tests pass under Bazel 6.5.0 and 7.4.1. --- I was on the right track in my analysis from bazelbuild#1619 ("Bump to rules_java 7.9.0 for Bazel 7 compatibility" in the message for commit cd22d88). However, I thought we _shouldn't_ update any targets from `@bazel_tools//tools/jdk:` to `@rules_java//toolchains:`. That's why I thought we were stuck on `rules_java` 7.9.0. However, this comment from @fmeum made me think switching to `@rules_java//toolchains:` actually is the preferred approach: - bazelbuild/rules_java#214 (comment) So this is a potentially breaking change, but in the good kind of way, in that it requires an easy, future proof update.
mbland
added a commit
to mbland/rules_scala
that referenced
this issue
Dec 3, 2024
Ensures we get all the versions of dependencies we want in `WORKSPACE`, while providing a new API to consumers. Part of bazelbuild#1652. Bumps several packages as high as they can go and still be compatible with Bazel 6 and 7: - `bazel_skylib`: 1.4.1 => 1.7.1 - `rules_cc`: 0.0.6 => 0.0.9 - `rules_python`: 0.36.0 => 0.38.0 - `rules_go`: 0.50.1 The following packages are at the maximum version to prevent breakages under Bazel 6.5.0. --- `abseil-cpp` and `protobuf` have to stay at 20220623.1 and v27.1, respectively, for Bazel 6 compatibility per bazelbuild#1647. `protobuf` up to v25.5 is compatible with Bazel 6 provided users set the compiler flags mentioned in that issue. --- `rules_python` 0.38.0 => 0.39.0 requires at least `rules_cc` 0.0.10, which introduced `cc/common/cc_info.bzl`: ```txt $ bazel build //{src,test,third_party,scala_proto}/... [ ...snip... ] ERROR: error loading package under directory 'test': error loading package 'test': at .../external/rules_python/python/defs.bzl:17:6: at .../external/rules_python/python/py_binary.bzl:18:6: at .../external/rules_python/python/private/py_binary_macro.bzl:16:6: at .../external/rules_python/python/private/common_bazel.bzl:19:6: Label '@rules_cc//cc/common:cc_info.bzl' is invalid because 'cc/common' is not a package; perhaps you meant to put the colon here: '@rules_cc//cc:common/cc_info.bzl'? ``` --- `rules_cc` 0.0.9 => 0.0.10 requires Bazel 7, which defines `CcSharedLibraryHintInfo`: ```txt $ bazel build //{src,test,third_party,scala_proto}/... ERROR: .../external/rules_cc/cc/private/rules_impl/native.bzl:40:33: name 'CcSharedLibraryHintInfo' is not defined (did you mean 'CcSharedLibraryInfo'?) [ ...snip... ] ERROR: error loading package under directory 'test': error loading package 'test': at .../external/rules_python/python/defs.bzl:17:6: at .../external/rules_python/python/py_binary.bzl:18:6: at .../external/rules_python/python/private/py_binary_macro.bzl:16:6: at .../external/rules_python/python/private/common_bazel.bzl:18:6: at .../external/rules_cc/cc/common/cc_common.bzl:17:6: compilation of module 'cc/private/rules_impl/native.bzl' failed ``` --- `rules_java` remains at 7.9.0 due to the more complicated situation it presents. The current version that could be compatible with Bazel 6 and 7 is 7.12.2. As it turns out, we could include `rules_java` versions up to and including 7.12.2 if we don't call the following in `WORKSPACE`: ```py load("@rules_java//java:repositories.bzl", "rules_java_dependencies", "rules_java_toolchains") rules_java_dependencies() rules_java_toolchains() ``` In fact, __we don't do that today__. When we do, Bazel 7.4.1 builds successfully, but Bazel 6.5.2 fails with: ```txt $ bazel build //{src,test,third_party,scala_proto}/... ERROR: .../external/rules_java/toolchains/java_toolchain_alias.bzl:83:34: Use of Starlark transition without allowlist attribute '_allowlist_function_transition'. See Starlark transitions documentation for details and usage: @rules_java//toolchains:java_toolchain_alias.bzl NORMAL ERROR: .../src/java/io/bazel/rulesscala/coverage/instrumenter/BUILD:3:12: While resolving toolchains for target //src/java/io/bazel/rulesscala/coverage/instrumenter:instrumenter: invalid registered toolchain '//toolchains:all': while parsing '//toolchains:all': error loading package '@rules_java//toolchains': initialization of module 'toolchains/java_toolchain_alias.bzl' failed ERROR: Analysis of target '//src/java/io/bazel/rulesscala/coverage/instrumenter:instrumenter' failed; build aborted: ``` When we bump to rules_java 7.10.0, which contains bazelbuild/rules_java#210, Bazel 7.4.1 fails with: ```txt $ bazel build //{src,test,third_party,scala_proto}/... ERROR: .../external/rules_java_builtin/toolchains/BUILD:254:14: While resolving toolchains for target @@rules_java_builtin//toolchains:platformclasspath (096dcc8): No matching toolchains found for types @@bazel_tools//tools/jdk:bootstrap_runtime_toolchain_type. To debug, rerun with --toolchain_resolution_debug='@@bazel_tools//tools/jdk:bootstrap_runtime_toolchain_type' If platforms or toolchains are a new concept for you, we'd encourage reading https://bazel.build/concepts/platforms-intro. ERROR: Analysis of target '//test/toolchains:java21_toolchain' failed; build aborted: Analysis failed ``` This is the error I described in bazelbuild#1619, under "Bump to rules_java 7.9.0 for Bazel 7 compatibility" in the message for commit cd22d88. Today I rediscovered: - bazelbuild/rules_java: Regression with @@rules_java//toolchains:bootstrap_runtime_toolchain_type in 7.10.0 bazelbuild/rules_java#214 But even worse, Bazel 6.5.0 fails with: ```txt $ bazel build //{src,test,third_party,scala_proto}/... [ ...snip... ] ERROR: .../test/src/main/resources/java_sources/BUILD:5:14: Building test/src/main/resources/java_sources/CompiledWithJava8_java.jar (1 source file) failed: Worker process did not return a WorkResponse: ---8<---8<--- Start of log, file at .../bazel-workers/multiplex-worker-18-Javac.log ---8<---8<--- Error thrown by worker thread, shutting down worker. java.lang.NoSuchMethodError: 'java.lang.Iterable com.google.devtools.build.buildjar.javac.BlazeJavacMain$ClassloaderMaskingFileManager.getJavaFileObjectsFromPaths(java.util.Collection)' at com.google.devtools.build.buildjar.javac.BlazeJavacMain.compile(BlazeJavacMain.java:142) at com.google.devtools.build.buildjar.ReducedClasspathJavaLibraryBuilder.compileSources(ReducedClasspathJavaLibraryBuilder.java:57) at com.google.devtools.build.buildjar.SimpleJavaLibraryBuilder.compileJavaLibrary(SimpleJavaLibraryBuilder.java:110) at com.google.devtools.build.buildjar.SimpleJavaLibraryBuilder.run(SimpleJavaLibraryBuilder.java:118) at com.google.devtools.build.buildjar.BazelJavaBuilder.build(BazelJavaBuilder.java:111) at com.google.devtools.build.buildjar.BazelJavaBuilder.parseAndBuild(BazelJavaBuilder.java:91) at com.google.devtools.build.buildjar.BazelJavaBuilder.lambda$main$0(BazelJavaBuilder.java:52) at com.google.devtools.build.lib.worker.WorkRequestHandler$WorkRequestCallback.apply(WorkRequestHandler.java:252) at com.google.devtools.build.lib.worker.WorkRequestHandler.respondToRequest(WorkRequestHandler.java:480) at com.google.devtools.build.lib.worker.WorkRequestHandler.lambda$startResponseThread$1(WorkRequestHandler.java:433) at java.base/java.lang.Thread.run(Thread.java:829) ---8<---8<--- End of log ---8<---8<--- ERROR: .../test/src/main/resources/java_sources/BUILD:11:14: Building test/src/main/resources/java_sources/CompiledWithJava11_java.jar (1 source file) failed: Worker process did not return a WorkResponse: ---8<---8<--- Start of log, file at .../bazel-workers/multiplex-worker-18-Javac.log ---8<---8<--- Error thrown by worker thread, shutting down worker. java.lang.NoSuchMethodError: 'java.lang.Iterable com.google.devtools.build.buildjar.javac.BlazeJavacMain$ClassloaderMaskingFileManager.getJavaFileObjectsFromPaths(java.util.Collection)' at com.google.devtools.build.buildjar.javac.BlazeJavacMain.compile(BlazeJavacMain.java:142) at com.google.devtools.build.buildjar.ReducedClasspathJavaLibraryBuilder.compileSources(ReducedClasspathJavaLibraryBuilder.java:57) at com.google.devtools.build.buildjar.SimpleJavaLibraryBuilder.compileJavaLibrary(SimpleJavaLibraryBuilder.java:110) at com.google.devtools.build.buildjar.SimpleJavaLibraryBuilder.run(SimpleJavaLibraryBuilder.java:118) at com.google.devtools.build.buildjar.BazelJavaBuilder.build(BazelJavaBuilder.java:111) at com.google.devtools.build.buildjar.BazelJavaBuilder.parseAndBuild(BazelJavaBuilder.java:91) at com.google.devtools.build.buildjar.BazelJavaBuilder.lambda$main$0(BazelJavaBuilder.java:52) at com.google.devtools.build.lib.worker.WorkRequestHandler$WorkRequestCallback.apply(WorkRequestHandler.java:252) at com.google.devtools.build.lib.worker.WorkRequestHandler.respondToRequest(WorkRequestHandler.java:480) at com.google.devtools.build.lib.worker.WorkRequestHandler.lambda$startResponseThread$1(WorkRequestHandler.java:433) at java.base/java.lang.Thread.run(Thread.java:829) ---8<---8<--- End of log ---8<---8<--- ``` In bazelbuild#1619, I thought we wanted to keep `@bazel_tools//tools/jdk:` as the canonical Java toolchains package, instead of `@rules_java//toolchains:`. But based on this comment from @fmeum, I'm inclined to believe switching to `@rules_java//toolchains:` actually is the preferred, futureproof approach: - bazelbuild/rules_java#214 (comment) --- Bazel 8 requires `rules_java` 8, per bazelbuild#1652. We can't update to `rules_java` 8 right now, unless we decided to do the following: - Tell Bazel 6 users to add C++ compiler flags to support the newer `abseil-cpp` versions required by newer `protobuf` versions. - Abandon Bazel 6 support in favor of supporting Bazel 7 at a minimum. - In addition to either of the above cases, update ScalaPB to 1.0.0-alpha.1 or higher to support `protobuf` v28. Here are the details explaining why `rules_java` 8 currently breaks. `rules_java` 8.0.0 requires Bazel >= 7.3.2, which provides the `subrule` API. Compatibility with Bazel >= 6.3.0 isn't restored until `rules_java` 8.3.2. ```txt $ bazel build //{src,test,third_party,scala_proto}/... ERROR: .../external/rules_java/java/common/rules/android_lint.bzl:142:24: name 'subrule' is not defined (did you mean 'rule'?) ERROR: Error computing the main repository mapping: at .../scala/private/extensions/dev_deps.bzl:8:6: at .../external/rules_java/java/repositories.bzl:20:6: at .../external/rules_java/toolchains/local_java_repository.bzl:17:6: at .../external/rules_java/java/defs.bzl:18:6: at .../external/rules_java/java/java_library.bzl:16:6: at .../external/rules_java/java/bazel/rules/bazel_java_library.bzl:21:6: compilation of module 'java/common/rules/android_lint.bzl' failed ``` `rules_java` 8.3.0 is broken, as it can't find its own `@compatibility_proxy` repo: ```txt $ bazel build //{src,test,third_party,scala_proto}/... ERROR: error loading package under directory 'src': error loading package 'src/protobuf/io/bazel/rules_scala': at .../external/rules_java/java/defs.bzl:22:6: at .../external/rules_java/java/java_test.bzl:16:6: Unable to find package for @compatibility_proxy//:proxy.bzl: The repository '@compatibility_proxy' could not be resolved: Repository '@compatibility_proxy' is not defined. ``` `rules_java` 8.3.1 seems to fix this, presumably by importing the `protobuf` repo as `com_google_protobuf`. However, it now requires at least `protobuf` v27.0, which adds `bazel/java/lite_proto_library.bzl`. Per bazelbuild#1647, we'd have to update to ScalaPB 1.0.0-alpha.1 to support `protobuf` v28, abandoning users of previous `protobuf` versions or forcing them to upgrade. ```txt $ bazel build //{src,test,third_party,scala_proto}/... [...snip...] ERROR: error loading package under directory 'src': error loading package 'src/java/io/bazel/rulesscala/worker': at .../external/rules_java/java/defs.bzl:16:6: Label '@com_google_protobuf//bazel:java_lite_proto_library.bzl' is invalid because 'bazel' is not a package; perhaps you meant to put the colon here: '@com_google_protobuf//:bazel/java_lite_proto_library.bzl'? ```
mbland
added a commit
to mbland/rules_scala
that referenced
this issue
Dec 3, 2024
Updates almost all dependencies on `@bazel_tools//tools/jdk:` toolchain targets and `.bzl` files to corresponding targets and files in `@rules_java//toolchains:`. Part of bazelbuild#1482 and bazelbuild#1652. All dependencies on `@bazel_tools//tools/jdk:toolchain_type` remain, however, as there is not yet a corresponding target in `@rules_java//toolchains:`. Adds the `WORKSPACE` stanza recommended in the `rules_java` release notes, and removes our own calls to instantiate `rules_java` repos and to register `rules_java` toolchains. This commit also removes `test/toolchains/jdk.bzl` or `//test/toolchains:java21_toolchain` since we no longer need them. (I actually think we didn't need them with 7.9.0, either, but this seems like a good point at which to remove them.) These changes are required to avoid build breakages when updating to `rules_scala` 7.10.0 and higher. All targets build and all tests pass under Bazel 6.5.0 and 7.4.1. --- I was on the right track in my analysis from bazelbuild#1619 ("Bump to rules_java 7.9.0 for Bazel 7 compatibility" in the message for commit cd22d88). However, I thought we _shouldn't_ update any targets from `@bazel_tools//tools/jdk:` to `@rules_java//toolchains:`. That's why I thought we were stuck on `rules_java` 7.9.0. However, this comment from @fmeum made me think switching to `@rules_java//toolchains:` actually is the preferred approach: - bazelbuild/rules_java#214 (comment) So this is a potentially breaking change, but in the good kind of way, in that it requires an easy, future proof update.
mbland
added a commit
to mbland/rules_scala
that referenced
this issue
Dec 3, 2024
Ensures we get all the versions of dependencies we want in `WORKSPACE`, while providing a new API to consumers. Part of bazelbuild#1652. Bumps several packages as high as they can go and still be compatible with Bazel 6 and 7: - `bazel_skylib`: 1.4.1 => 1.7.1 - `rules_cc`: 0.0.6 => 0.0.9 - `rules_python`: 0.36.0 => 0.38.0 - `rules_go`: 0.50.1 The following packages are at the maximum version to prevent breakages under Bazel 6.5.0. --- `abseil-cpp` and `protobuf` have to stay at 20220623.1 and v27.1, respectively, for Bazel 6 compatibility per bazelbuild#1647. `protobuf` up to v25.5 is compatible with Bazel 6 provided users set the compiler flags mentioned in that issue. --- `rules_python` 0.38.0 => 0.39.0 requires at least `rules_cc` 0.0.10, which introduced `cc/common/cc_info.bzl`: ```txt $ bazel build //{src,test,third_party,scala_proto}/... [ ...snip... ] ERROR: error loading package under directory 'test': error loading package 'test': at .../external/rules_python/python/defs.bzl:17:6: at .../external/rules_python/python/py_binary.bzl:18:6: at .../external/rules_python/python/private/py_binary_macro.bzl:16:6: at .../external/rules_python/python/private/common_bazel.bzl:19:6: Label '@rules_cc//cc/common:cc_info.bzl' is invalid because 'cc/common' is not a package; perhaps you meant to put the colon here: '@rules_cc//cc:common/cc_info.bzl'? ``` --- `rules_cc` 0.0.9 => 0.0.10 requires Bazel 7, which defines `CcSharedLibraryHintInfo`: ```txt $ bazel build //{src,test,third_party,scala_proto}/... ERROR: .../external/rules_cc/cc/private/rules_impl/native.bzl:40:33: name 'CcSharedLibraryHintInfo' is not defined (did you mean 'CcSharedLibraryInfo'?) [ ...snip... ] ERROR: error loading package under directory 'test': error loading package 'test': at .../external/rules_python/python/defs.bzl:17:6: at .../external/rules_python/python/py_binary.bzl:18:6: at .../external/rules_python/python/private/py_binary_macro.bzl:16:6: at .../external/rules_python/python/private/common_bazel.bzl:18:6: at .../external/rules_cc/cc/common/cc_common.bzl:17:6: compilation of module 'cc/private/rules_impl/native.bzl' failed ``` --- `rules_java` remains at 7.9.0 due to the more complicated situation it presents. The current version that could be compatible with Bazel 6 and 7 is 7.12.2. As it turns out, we could include `rules_java` versions up to and including 7.12.2 if we don't call the following in `WORKSPACE`: ```py load("@rules_java//java:repositories.bzl", "rules_java_dependencies", "rules_java_toolchains") rules_java_dependencies() rules_java_toolchains() ``` In fact, __we don't do that today__. When we do, Bazel 7.4.1 builds successfully, but Bazel 6.5.2 fails with: ```txt $ bazel build //{src,test,third_party,scala_proto}/... ERROR: .../external/rules_java/toolchains/java_toolchain_alias.bzl:83:34: Use of Starlark transition without allowlist attribute '_allowlist_function_transition'. See Starlark transitions documentation for details and usage: @rules_java//toolchains:java_toolchain_alias.bzl NORMAL ERROR: .../src/java/io/bazel/rulesscala/coverage/instrumenter/BUILD:3:12: While resolving toolchains for target //src/java/io/bazel/rulesscala/coverage/instrumenter:instrumenter: invalid registered toolchain '//toolchains:all': while parsing '//toolchains:all': error loading package '@rules_java//toolchains': initialization of module 'toolchains/java_toolchain_alias.bzl' failed ERROR: Analysis of target '//src/java/io/bazel/rulesscala/coverage/instrumenter:instrumenter' failed; build aborted: ``` When we bump to rules_java 7.10.0, which contains bazelbuild/rules_java#210, Bazel 7.4.1 fails with: ```txt $ bazel build //{src,test,third_party,scala_proto}/... ERROR: .../external/rules_java_builtin/toolchains/BUILD:254:14: While resolving toolchains for target @@rules_java_builtin//toolchains:platformclasspath (096dcc8): No matching toolchains found for types @@bazel_tools//tools/jdk:bootstrap_runtime_toolchain_type. To debug, rerun with --toolchain_resolution_debug='@@bazel_tools//tools/jdk:bootstrap_runtime_toolchain_type' If platforms or toolchains are a new concept for you, we'd encourage reading https://bazel.build/concepts/platforms-intro. ERROR: Analysis of target '//test/toolchains:java21_toolchain' failed; build aborted: Analysis failed ``` This is the error I described in bazelbuild#1619, under "Bump to rules_java 7.9.0 for Bazel 7 compatibility" in the message for commit cd22d88. Today I rediscovered: - bazelbuild/rules_java: Regression with @@rules_java//toolchains:bootstrap_runtime_toolchain_type in 7.10.0 bazelbuild/rules_java#214 But even worse, Bazel 6.5.0 fails with: ```txt $ bazel build //{src,test,third_party,scala_proto}/... [ ...snip... ] ERROR: .../test/src/main/resources/java_sources/BUILD:5:14: Building test/src/main/resources/java_sources/CompiledWithJava8_java.jar (1 source file) failed: Worker process did not return a WorkResponse: ---8<---8<--- Start of log, file at .../bazel-workers/multiplex-worker-18-Javac.log ---8<---8<--- Error thrown by worker thread, shutting down worker. java.lang.NoSuchMethodError: 'java.lang.Iterable com.google.devtools.build.buildjar.javac.BlazeJavacMain$ClassloaderMaskingFileManager.getJavaFileObjectsFromPaths(java.util.Collection)' at com.google.devtools.build.buildjar.javac.BlazeJavacMain.compile(BlazeJavacMain.java:142) at com.google.devtools.build.buildjar.ReducedClasspathJavaLibraryBuilder.compileSources(ReducedClasspathJavaLibraryBuilder.java:57) at com.google.devtools.build.buildjar.SimpleJavaLibraryBuilder.compileJavaLibrary(SimpleJavaLibraryBuilder.java:110) at com.google.devtools.build.buildjar.SimpleJavaLibraryBuilder.run(SimpleJavaLibraryBuilder.java:118) at com.google.devtools.build.buildjar.BazelJavaBuilder.build(BazelJavaBuilder.java:111) at com.google.devtools.build.buildjar.BazelJavaBuilder.parseAndBuild(BazelJavaBuilder.java:91) at com.google.devtools.build.buildjar.BazelJavaBuilder.lambda$main$0(BazelJavaBuilder.java:52) at com.google.devtools.build.lib.worker.WorkRequestHandler$WorkRequestCallback.apply(WorkRequestHandler.java:252) at com.google.devtools.build.lib.worker.WorkRequestHandler.respondToRequest(WorkRequestHandler.java:480) at com.google.devtools.build.lib.worker.WorkRequestHandler.lambda$startResponseThread$1(WorkRequestHandler.java:433) at java.base/java.lang.Thread.run(Thread.java:829) ---8<---8<--- End of log ---8<---8<--- ERROR: .../test/src/main/resources/java_sources/BUILD:11:14: Building test/src/main/resources/java_sources/CompiledWithJava11_java.jar (1 source file) failed: Worker process did not return a WorkResponse: ---8<---8<--- Start of log, file at .../bazel-workers/multiplex-worker-18-Javac.log ---8<---8<--- Error thrown by worker thread, shutting down worker. java.lang.NoSuchMethodError: 'java.lang.Iterable com.google.devtools.build.buildjar.javac.BlazeJavacMain$ClassloaderMaskingFileManager.getJavaFileObjectsFromPaths(java.util.Collection)' at com.google.devtools.build.buildjar.javac.BlazeJavacMain.compile(BlazeJavacMain.java:142) at com.google.devtools.build.buildjar.ReducedClasspathJavaLibraryBuilder.compileSources(ReducedClasspathJavaLibraryBuilder.java:57) at com.google.devtools.build.buildjar.SimpleJavaLibraryBuilder.compileJavaLibrary(SimpleJavaLibraryBuilder.java:110) at com.google.devtools.build.buildjar.SimpleJavaLibraryBuilder.run(SimpleJavaLibraryBuilder.java:118) at com.google.devtools.build.buildjar.BazelJavaBuilder.build(BazelJavaBuilder.java:111) at com.google.devtools.build.buildjar.BazelJavaBuilder.parseAndBuild(BazelJavaBuilder.java:91) at com.google.devtools.build.buildjar.BazelJavaBuilder.lambda$main$0(BazelJavaBuilder.java:52) at com.google.devtools.build.lib.worker.WorkRequestHandler$WorkRequestCallback.apply(WorkRequestHandler.java:252) at com.google.devtools.build.lib.worker.WorkRequestHandler.respondToRequest(WorkRequestHandler.java:480) at com.google.devtools.build.lib.worker.WorkRequestHandler.lambda$startResponseThread$1(WorkRequestHandler.java:433) at java.base/java.lang.Thread.run(Thread.java:829) ---8<---8<--- End of log ---8<---8<--- ``` In bazelbuild#1619, I thought we wanted to keep `@bazel_tools//tools/jdk:` as the canonical Java toolchains package, instead of `@rules_java//toolchains:`. But based on this comment from @fmeum, I'm inclined to believe switching to `@rules_java//toolchains:` actually is the preferred, futureproof approach: - bazelbuild/rules_java#214 (comment) --- Bazel 8 requires `rules_java` 8, per bazelbuild#1652. We can't update to `rules_java` 8 right now, unless we decided to do the following: - Tell Bazel 6 users to add C++ compiler flags to support the newer `abseil-cpp` versions required by newer `protobuf` versions. - Abandon Bazel 6 support in favor of supporting Bazel 7 at a minimum. - In addition to either of the above cases, update ScalaPB to 1.0.0-alpha.1 or higher to support `protobuf` v28. Here are the details explaining why `rules_java` 8 currently breaks. `rules_java` 8.0.0 requires Bazel >= 7.3.2, which provides the `subrule` API. Compatibility with Bazel >= 6.3.0 isn't restored until `rules_java` 8.3.2. ```txt $ bazel build //{src,test,third_party,scala_proto}/... ERROR: .../external/rules_java/java/common/rules/android_lint.bzl:142:24: name 'subrule' is not defined (did you mean 'rule'?) ERROR: Error computing the main repository mapping: at .../scala/private/extensions/dev_deps.bzl:8:6: at .../external/rules_java/java/repositories.bzl:20:6: at .../external/rules_java/toolchains/local_java_repository.bzl:17:6: at .../external/rules_java/java/defs.bzl:18:6: at .../external/rules_java/java/java_library.bzl:16:6: at .../external/rules_java/java/bazel/rules/bazel_java_library.bzl:21:6: compilation of module 'java/common/rules/android_lint.bzl' failed ``` `rules_java` 8.3.0 is broken, as it can't find its own `@compatibility_proxy` repo: ```txt $ bazel build //{src,test,third_party,scala_proto}/... ERROR: error loading package under directory 'src': error loading package 'src/protobuf/io/bazel/rules_scala': at .../external/rules_java/java/defs.bzl:22:6: at .../external/rules_java/java/java_test.bzl:16:6: Unable to find package for @compatibility_proxy//:proxy.bzl: The repository '@compatibility_proxy' could not be resolved: Repository '@compatibility_proxy' is not defined. ``` `rules_java` 8.3.1 seems to fix this, presumably by importing the `protobuf` repo as `com_google_protobuf`. However, it now requires at least `protobuf` v27.0, which adds `bazel/java/lite_proto_library.bzl`. Per bazelbuild#1647, we'd have to update to ScalaPB 1.0.0-alpha.1 to support `protobuf` v28, abandoning users of previous `protobuf` versions or forcing them to upgrade. ```txt $ bazel build //{src,test,third_party,scala_proto}/... [...snip...] ERROR: error loading package under directory 'src': error loading package 'src/java/io/bazel/rulesscala/worker': at .../external/rules_java/java/defs.bzl:16:6: Label '@com_google_protobuf//bazel:java_lite_proto_library.bzl' is invalid because 'bazel' is not a package; perhaps you meant to put the colon here: '@com_google_protobuf//:bazel/java_lite_proto_library.bzl'? ```
mbland
added a commit
to mbland/rules_scala
that referenced
this issue
Dec 3, 2024
Ensures we get all the versions of dependencies we want in `WORKSPACE`, while providing a new API to consumers. Part of bazelbuild#1652. Bumps several packages as high as they can go and still be compatible with Bazel 6 and 7: - `bazel_skylib`: 1.4.1 => 1.7.1 - `rules_cc`: 0.0.6 => 0.0.9 - `rules_python`: 0.36.0 => 0.38.0 - `rules_go`: 0.50.1 The following packages are at the maximum version to prevent breakages under Bazel 6.5.0. --- `abseil-cpp` and `protobuf` have to stay at 20220623.1 and v27.1, respectively, for Bazel 6 compatibility per bazelbuild#1647. `protobuf` up to v25.5 is compatible with Bazel 6 provided users set the compiler flags mentioned in that issue. --- `rules_python` 0.38.0 => 0.39.0 requires at least `rules_cc` 0.0.10, which introduced `cc/common/cc_info.bzl`: ```txt $ bazel build //{src,test,third_party,scala_proto}/... [ ...snip... ] ERROR: error loading package under directory 'test': error loading package 'test': at .../external/rules_python/python/defs.bzl:17:6: at .../external/rules_python/python/py_binary.bzl:18:6: at .../external/rules_python/python/private/py_binary_macro.bzl:16:6: at .../external/rules_python/python/private/common_bazel.bzl:19:6: Label '@rules_cc//cc/common:cc_info.bzl' is invalid because 'cc/common' is not a package; perhaps you meant to put the colon here: '@rules_cc//cc:common/cc_info.bzl'? ``` --- `rules_cc` 0.0.9 => 0.0.10 requires Bazel 7, which defines `CcSharedLibraryHintInfo`: ```txt $ bazel build //{src,test,third_party,scala_proto}/... ERROR: .../external/rules_cc/cc/private/rules_impl/native.bzl:40:33: name 'CcSharedLibraryHintInfo' is not defined (did you mean 'CcSharedLibraryInfo'?) [ ...snip... ] ERROR: error loading package under directory 'test': error loading package 'test': at .../external/rules_python/python/defs.bzl:17:6: at .../external/rules_python/python/py_binary.bzl:18:6: at .../external/rules_python/python/private/py_binary_macro.bzl:16:6: at .../external/rules_python/python/private/common_bazel.bzl:18:6: at .../external/rules_cc/cc/common/cc_common.bzl:17:6: compilation of module 'cc/private/rules_impl/native.bzl' failed ``` --- `rules_java` remains at 7.9.0 due to the more complicated situation it presents. The current version that could be compatible with Bazel 6 and 7 is 7.12.2. As it turns out, we could include `rules_java` versions up to and including 7.12.2 if we don't call the following in `WORKSPACE`: ```py load("@rules_java//java:repositories.bzl", "rules_java_dependencies", "rules_java_toolchains") rules_java_dependencies() rules_java_toolchains() ``` In fact, __we don't do that today__. When we do, Bazel 7.4.1 builds successfully, but Bazel 6.5.2 fails with: ```txt $ bazel build //{src,test,third_party,scala_proto}/... ERROR: .../external/rules_java/toolchains/java_toolchain_alias.bzl:83:34: Use of Starlark transition without allowlist attribute '_allowlist_function_transition'. See Starlark transitions documentation for details and usage: @rules_java//toolchains:java_toolchain_alias.bzl NORMAL ERROR: .../src/java/io/bazel/rulesscala/coverage/instrumenter/BUILD:3:12: While resolving toolchains for target //src/java/io/bazel/rulesscala/coverage/instrumenter:instrumenter: invalid registered toolchain '//toolchains:all': while parsing '//toolchains:all': error loading package '@rules_java//toolchains': initialization of module 'toolchains/java_toolchain_alias.bzl' failed ERROR: Analysis of target '//src/java/io/bazel/rulesscala/coverage/instrumenter:instrumenter' failed; build aborted: ``` When we bump to rules_java 7.10.0, which contains bazelbuild/rules_java#210, Bazel 7.4.1 fails with: ```txt $ bazel build //{src,test,third_party,scala_proto}/... ERROR: .../external/rules_java_builtin/toolchains/BUILD:254:14: While resolving toolchains for target @@rules_java_builtin//toolchains:platformclasspath (096dcc8): No matching toolchains found for types @@bazel_tools//tools/jdk:bootstrap_runtime_toolchain_type. To debug, rerun with --toolchain_resolution_debug='@@bazel_tools//tools/jdk:bootstrap_runtime_toolchain_type' If platforms or toolchains are a new concept for you, we'd encourage reading https://bazel.build/concepts/platforms-intro. ERROR: Analysis of target '//test/toolchains:java21_toolchain' failed; build aborted: Analysis failed ``` This is the error I described in bazelbuild#1619, under "Bump to rules_java 7.9.0 for Bazel 7 compatibility" in the message for commit cd22d88. Today I rediscovered: - bazelbuild/rules_java: Regression with @@rules_java//toolchains:bootstrap_runtime_toolchain_type in 7.10.0 bazelbuild/rules_java#214 But even worse, Bazel 6.5.0 fails with: ```txt $ bazel build //{src,test,third_party,scala_proto}/... [ ...snip... ] ERROR: .../test/src/main/resources/java_sources/BUILD:5:14: Building test/src/main/resources/java_sources/CompiledWithJava8_java.jar (1 source file) failed: Worker process did not return a WorkResponse: ---8<---8<--- Start of log, file at .../bazel-workers/multiplex-worker-18-Javac.log ---8<---8<--- Error thrown by worker thread, shutting down worker. java.lang.NoSuchMethodError: 'java.lang.Iterable com.google.devtools.build.buildjar.javac.BlazeJavacMain$ClassloaderMaskingFileManager.getJavaFileObjectsFromPaths(java.util.Collection)' at com.google.devtools.build.buildjar.javac.BlazeJavacMain.compile(BlazeJavacMain.java:142) at com.google.devtools.build.buildjar.ReducedClasspathJavaLibraryBuilder.compileSources(ReducedClasspathJavaLibraryBuilder.java:57) at com.google.devtools.build.buildjar.SimpleJavaLibraryBuilder.compileJavaLibrary(SimpleJavaLibraryBuilder.java:110) at com.google.devtools.build.buildjar.SimpleJavaLibraryBuilder.run(SimpleJavaLibraryBuilder.java:118) at com.google.devtools.build.buildjar.BazelJavaBuilder.build(BazelJavaBuilder.java:111) at com.google.devtools.build.buildjar.BazelJavaBuilder.parseAndBuild(BazelJavaBuilder.java:91) at com.google.devtools.build.buildjar.BazelJavaBuilder.lambda$main$0(BazelJavaBuilder.java:52) at com.google.devtools.build.lib.worker.WorkRequestHandler$WorkRequestCallback.apply(WorkRequestHandler.java:252) at com.google.devtools.build.lib.worker.WorkRequestHandler.respondToRequest(WorkRequestHandler.java:480) at com.google.devtools.build.lib.worker.WorkRequestHandler.lambda$startResponseThread$1(WorkRequestHandler.java:433) at java.base/java.lang.Thread.run(Thread.java:829) ---8<---8<--- End of log ---8<---8<--- ERROR: .../test/src/main/resources/java_sources/BUILD:11:14: Building test/src/main/resources/java_sources/CompiledWithJava11_java.jar (1 source file) failed: Worker process did not return a WorkResponse: ---8<---8<--- Start of log, file at .../bazel-workers/multiplex-worker-18-Javac.log ---8<---8<--- Error thrown by worker thread, shutting down worker. java.lang.NoSuchMethodError: 'java.lang.Iterable com.google.devtools.build.buildjar.javac.BlazeJavacMain$ClassloaderMaskingFileManager.getJavaFileObjectsFromPaths(java.util.Collection)' at com.google.devtools.build.buildjar.javac.BlazeJavacMain.compile(BlazeJavacMain.java:142) at com.google.devtools.build.buildjar.ReducedClasspathJavaLibraryBuilder.compileSources(ReducedClasspathJavaLibraryBuilder.java:57) at com.google.devtools.build.buildjar.SimpleJavaLibraryBuilder.compileJavaLibrary(SimpleJavaLibraryBuilder.java:110) at com.google.devtools.build.buildjar.SimpleJavaLibraryBuilder.run(SimpleJavaLibraryBuilder.java:118) at com.google.devtools.build.buildjar.BazelJavaBuilder.build(BazelJavaBuilder.java:111) at com.google.devtools.build.buildjar.BazelJavaBuilder.parseAndBuild(BazelJavaBuilder.java:91) at com.google.devtools.build.buildjar.BazelJavaBuilder.lambda$main$0(BazelJavaBuilder.java:52) at com.google.devtools.build.lib.worker.WorkRequestHandler$WorkRequestCallback.apply(WorkRequestHandler.java:252) at com.google.devtools.build.lib.worker.WorkRequestHandler.respondToRequest(WorkRequestHandler.java:480) at com.google.devtools.build.lib.worker.WorkRequestHandler.lambda$startResponseThread$1(WorkRequestHandler.java:433) at java.base/java.lang.Thread.run(Thread.java:829) ---8<---8<--- End of log ---8<---8<--- ``` In bazelbuild#1619, I thought we wanted to keep `@bazel_tools//tools/jdk:` as the canonical Java toolchains package, instead of `@rules_java//toolchains:`. But based on this comment from @fmeum, I'm inclined to believe switching to `@rules_java//toolchains:` actually is the preferred, futureproof approach: - bazelbuild/rules_java#214 (comment) --- Bazel 8 requires `rules_java` 8, per bazelbuild#1652. We can't update to `rules_java` 8 right now, unless we decided to do the following: - Tell Bazel 6 users to add C++ compiler flags to support the newer `abseil-cpp` versions required by newer `protobuf` versions. - Abandon Bazel 6 support in favor of supporting Bazel 7 at a minimum. - In addition to either of the above cases, update ScalaPB to 1.0.0-alpha.1 or higher to support `protobuf` v28. Here are the details explaining why `rules_java` 8 currently breaks. `rules_java` 8.0.0 requires Bazel >= 7.3.2, which provides the `subrule` API. Compatibility with Bazel >= 6.3.0 isn't restored until `rules_java` 8.3.2. ```txt $ bazel build //{src,test,third_party,scala_proto}/... ERROR: .../external/rules_java/java/common/rules/android_lint.bzl:142:24: name 'subrule' is not defined (did you mean 'rule'?) ERROR: Error computing the main repository mapping: at .../scala/private/extensions/dev_deps.bzl:8:6: at .../external/rules_java/java/repositories.bzl:20:6: at .../external/rules_java/toolchains/local_java_repository.bzl:17:6: at .../external/rules_java/java/defs.bzl:18:6: at .../external/rules_java/java/java_library.bzl:16:6: at .../external/rules_java/java/bazel/rules/bazel_java_library.bzl:21:6: compilation of module 'java/common/rules/android_lint.bzl' failed ``` `rules_java` 8.3.0 is broken, as it can't find its own `@compatibility_proxy` repo: ```txt $ bazel build //{src,test,third_party,scala_proto}/... ERROR: error loading package under directory 'src': error loading package 'src/protobuf/io/bazel/rules_scala': at .../external/rules_java/java/defs.bzl:22:6: at .../external/rules_java/java/java_test.bzl:16:6: Unable to find package for @compatibility_proxy//:proxy.bzl: The repository '@compatibility_proxy' could not be resolved: Repository '@compatibility_proxy' is not defined. ``` `rules_java` 8.3.1 seems to fix this, presumably by importing the `protobuf` repo as `com_google_protobuf`. However, it now requires at least `protobuf` v27.0, which adds `bazel/java/lite_proto_library.bzl`. Per bazelbuild#1647, we'd have to update to ScalaPB 1.0.0-alpha.1 to support `protobuf` v28, abandoning users of previous `protobuf` versions or forcing them to upgrade. ```txt $ bazel build //{src,test,third_party,scala_proto}/... [...snip...] ERROR: error loading package under directory 'src': error loading package 'src/java/io/bazel/rulesscala/worker': at .../external/rules_java/java/defs.bzl:16:6: Label '@com_google_protobuf//bazel:java_lite_proto_library.bzl' is invalid because 'bazel' is not a package; perhaps you meant to put the colon here: '@com_google_protobuf//:bazel/java_lite_proto_library.bzl'? ```
mbland
added a commit
to mbland/rules_scala
that referenced
this issue
Dec 3, 2024
Updates almost all dependencies on `@bazel_tools//tools/jdk:` toolchain targets and `.bzl` files to corresponding targets and files in `@rules_java//toolchains:`. Part of bazelbuild#1482 and bazelbuild#1652. All dependencies on `@bazel_tools//tools/jdk:toolchain_type` remain, however, as there is not yet a corresponding target in `@rules_java//toolchains:`. Adds the `WORKSPACE` stanza recommended in the `rules_java` release notes, and removes our own calls to instantiate `rules_java` repos and to register `rules_java` toolchains. This commit also removes `test/toolchains/jdk.bzl` or `//test/toolchains:java21_toolchain` since we no longer need them. (I actually think we didn't need them with 7.9.0, either, but this seems like a good point at which to remove them.) These changes are required to avoid build breakages when updating to `rules_scala` 7.10.0 and higher. All targets build and all tests pass under Bazel 6.5.0 and 7.4.1. --- I was on the right track in my analysis from bazelbuild#1619 ("Bump to rules_java 7.9.0 for Bazel 7 compatibility" in the message for commit cd22d88). However, I thought we _shouldn't_ update any targets from `@bazel_tools//tools/jdk:` to `@rules_java//toolchains:`. That's why I thought we were stuck on `rules_java` 7.9.0. However, this comment from @fmeum made me think switching to `@rules_java//toolchains:` actually is the preferred approach: - bazelbuild/rules_java#214 (comment) So this is a potentially breaking change, but in the good kind of way, in that it requires an easy, future proof update.
mbland
added a commit
to mbland/rules_scala
that referenced
this issue
Dec 3, 2024
Ensures we get all the versions of dependencies we want in `WORKSPACE`, while providing a new API to consumers. Part of bazelbuild#1652. Bumps several packages as high as they can go and still be compatible with Bazel 6 and 7: - `bazel_skylib`: 1.4.1 => 1.7.1 - `rules_cc`: 0.0.6 => 0.0.9 - `rules_python`: 0.36.0 => 0.38.0 - `rules_go`: 0.50.1 The following packages are at the maximum version to prevent breakages under Bazel 6.5.0. --- `abseil-cpp` and `protobuf` have to stay at 20220623.1 and v27.1, respectively, for Bazel 6 compatibility per bazelbuild#1647. `protobuf` up to v25.5 is compatible with Bazel 6 provided users set the compiler flags mentioned in that issue. --- `rules_python` 0.38.0 => 0.39.0 requires at least `rules_cc` 0.0.10, which introduced `cc/common/cc_info.bzl`: ```txt $ bazel build //{src,test,third_party,scala_proto}/... [ ...snip... ] ERROR: error loading package under directory 'test': error loading package 'test': at .../external/rules_python/python/defs.bzl:17:6: at .../external/rules_python/python/py_binary.bzl:18:6: at .../external/rules_python/python/private/py_binary_macro.bzl:16:6: at .../external/rules_python/python/private/common_bazel.bzl:19:6: Label '@rules_cc//cc/common:cc_info.bzl' is invalid because 'cc/common' is not a package; perhaps you meant to put the colon here: '@rules_cc//cc:common/cc_info.bzl'? ``` --- `rules_cc` 0.0.9 => 0.0.10 requires Bazel 7, which defines `CcSharedLibraryHintInfo`: ```txt $ bazel build //{src,test,third_party,scala_proto}/... ERROR: .../external/rules_cc/cc/private/rules_impl/native.bzl:40:33: name 'CcSharedLibraryHintInfo' is not defined (did you mean 'CcSharedLibraryInfo'?) [ ...snip... ] ERROR: error loading package under directory 'test': error loading package 'test': at .../external/rules_python/python/defs.bzl:17:6: at .../external/rules_python/python/py_binary.bzl:18:6: at .../external/rules_python/python/private/py_binary_macro.bzl:16:6: at .../external/rules_python/python/private/common_bazel.bzl:18:6: at .../external/rules_cc/cc/common/cc_common.bzl:17:6: compilation of module 'cc/private/rules_impl/native.bzl' failed ``` --- `rules_java` remains at 7.9.0 due to the more complicated situation it presents. The current version that could be compatible with Bazel 6 and 7 is 7.12.2. As it turns out, we could include `rules_java` versions up to and including 7.12.2 if we don't call the following in `WORKSPACE`: ```py load("@rules_java//java:repositories.bzl", "rules_java_dependencies", "rules_java_toolchains") rules_java_dependencies() rules_java_toolchains() ``` In fact, __we don't do that today__. When we do, Bazel 7.4.1 builds successfully, but Bazel 6.5.2 fails with: ```txt $ bazel build //{src,test,third_party,scala_proto}/... ERROR: .../external/rules_java/toolchains/java_toolchain_alias.bzl:83:34: Use of Starlark transition without allowlist attribute '_allowlist_function_transition'. See Starlark transitions documentation for details and usage: @rules_java//toolchains:java_toolchain_alias.bzl NORMAL ERROR: .../src/java/io/bazel/rulesscala/coverage/instrumenter/BUILD:3:12: While resolving toolchains for target //src/java/io/bazel/rulesscala/coverage/instrumenter:instrumenter: invalid registered toolchain '//toolchains:all': while parsing '//toolchains:all': error loading package '@rules_java//toolchains': initialization of module 'toolchains/java_toolchain_alias.bzl' failed ERROR: Analysis of target '//src/java/io/bazel/rulesscala/coverage/instrumenter:instrumenter' failed; build aborted: ``` When we bump to rules_java 7.10.0, which contains bazelbuild/rules_java#210, Bazel 7.4.1 fails with: ```txt $ bazel build //{src,test,third_party,scala_proto}/... ERROR: .../external/rules_java_builtin/toolchains/BUILD:254:14: While resolving toolchains for target @@rules_java_builtin//toolchains:platformclasspath (096dcc8): No matching toolchains found for types @@bazel_tools//tools/jdk:bootstrap_runtime_toolchain_type. To debug, rerun with --toolchain_resolution_debug='@@bazel_tools//tools/jdk:bootstrap_runtime_toolchain_type' If platforms or toolchains are a new concept for you, we'd encourage reading https://bazel.build/concepts/platforms-intro. ERROR: Analysis of target '//test/toolchains:java21_toolchain' failed; build aborted: Analysis failed ``` This is the error I described in bazelbuild#1619, under "Bump to rules_java 7.9.0 for Bazel 7 compatibility" in the message for commit cd22d88. Today I rediscovered: - bazelbuild/rules_java: Regression with @@rules_java//toolchains:bootstrap_runtime_toolchain_type in 7.10.0 bazelbuild/rules_java#214 But even worse, Bazel 6.5.0 fails with: ```txt $ bazel build //{src,test,third_party,scala_proto}/... [ ...snip... ] ERROR: .../test/src/main/resources/java_sources/BUILD:5:14: Building test/src/main/resources/java_sources/CompiledWithJava8_java.jar (1 source file) failed: Worker process did not return a WorkResponse: ---8<---8<--- Start of log, file at .../bazel-workers/multiplex-worker-18-Javac.log ---8<---8<--- Error thrown by worker thread, shutting down worker. java.lang.NoSuchMethodError: 'java.lang.Iterable com.google.devtools.build.buildjar.javac.BlazeJavacMain$ClassloaderMaskingFileManager.getJavaFileObjectsFromPaths(java.util.Collection)' at com.google.devtools.build.buildjar.javac.BlazeJavacMain.compile(BlazeJavacMain.java:142) at com.google.devtools.build.buildjar.ReducedClasspathJavaLibraryBuilder.compileSources(ReducedClasspathJavaLibraryBuilder.java:57) at com.google.devtools.build.buildjar.SimpleJavaLibraryBuilder.compileJavaLibrary(SimpleJavaLibraryBuilder.java:110) at com.google.devtools.build.buildjar.SimpleJavaLibraryBuilder.run(SimpleJavaLibraryBuilder.java:118) at com.google.devtools.build.buildjar.BazelJavaBuilder.build(BazelJavaBuilder.java:111) at com.google.devtools.build.buildjar.BazelJavaBuilder.parseAndBuild(BazelJavaBuilder.java:91) at com.google.devtools.build.buildjar.BazelJavaBuilder.lambda$main$0(BazelJavaBuilder.java:52) at com.google.devtools.build.lib.worker.WorkRequestHandler$WorkRequestCallback.apply(WorkRequestHandler.java:252) at com.google.devtools.build.lib.worker.WorkRequestHandler.respondToRequest(WorkRequestHandler.java:480) at com.google.devtools.build.lib.worker.WorkRequestHandler.lambda$startResponseThread$1(WorkRequestHandler.java:433) at java.base/java.lang.Thread.run(Thread.java:829) ---8<---8<--- End of log ---8<---8<--- ERROR: .../test/src/main/resources/java_sources/BUILD:11:14: Building test/src/main/resources/java_sources/CompiledWithJava11_java.jar (1 source file) failed: Worker process did not return a WorkResponse: ---8<---8<--- Start of log, file at .../bazel-workers/multiplex-worker-18-Javac.log ---8<---8<--- Error thrown by worker thread, shutting down worker. java.lang.NoSuchMethodError: 'java.lang.Iterable com.google.devtools.build.buildjar.javac.BlazeJavacMain$ClassloaderMaskingFileManager.getJavaFileObjectsFromPaths(java.util.Collection)' at com.google.devtools.build.buildjar.javac.BlazeJavacMain.compile(BlazeJavacMain.java:142) at com.google.devtools.build.buildjar.ReducedClasspathJavaLibraryBuilder.compileSources(ReducedClasspathJavaLibraryBuilder.java:57) at com.google.devtools.build.buildjar.SimpleJavaLibraryBuilder.compileJavaLibrary(SimpleJavaLibraryBuilder.java:110) at com.google.devtools.build.buildjar.SimpleJavaLibraryBuilder.run(SimpleJavaLibraryBuilder.java:118) at com.google.devtools.build.buildjar.BazelJavaBuilder.build(BazelJavaBuilder.java:111) at com.google.devtools.build.buildjar.BazelJavaBuilder.parseAndBuild(BazelJavaBuilder.java:91) at com.google.devtools.build.buildjar.BazelJavaBuilder.lambda$main$0(BazelJavaBuilder.java:52) at com.google.devtools.build.lib.worker.WorkRequestHandler$WorkRequestCallback.apply(WorkRequestHandler.java:252) at com.google.devtools.build.lib.worker.WorkRequestHandler.respondToRequest(WorkRequestHandler.java:480) at com.google.devtools.build.lib.worker.WorkRequestHandler.lambda$startResponseThread$1(WorkRequestHandler.java:433) at java.base/java.lang.Thread.run(Thread.java:829) ---8<---8<--- End of log ---8<---8<--- ``` In bazelbuild#1619, I thought we wanted to keep `@bazel_tools//tools/jdk:` as the canonical Java toolchains package, instead of `@rules_java//toolchains:`. But based on this comment from @fmeum, I'm inclined to believe switching to `@rules_java//toolchains:` actually is the preferred, futureproof approach: - bazelbuild/rules_java#214 (comment) --- Bazel 8 requires `rules_java` 8, per bazelbuild#1652. We can't update to `rules_java` 8 right now, unless we decided to do the following: - Tell Bazel 6 users to add C++ compiler flags to support the newer `abseil-cpp` versions required by newer `protobuf` versions. - Abandon Bazel 6 support in favor of supporting Bazel 7 at a minimum. - In addition to either of the above cases, update ScalaPB to 1.0.0-alpha.1 or higher to support `protobuf` v28. Here are the details explaining why `rules_java` 8 currently breaks. `rules_java` 8.0.0 requires Bazel >= 7.3.2, which provides the `subrule` API. Compatibility with Bazel >= 6.3.0 isn't restored until `rules_java` 8.3.2. ```txt $ bazel build //{src,test,third_party,scala_proto}/... ERROR: .../external/rules_java/java/common/rules/android_lint.bzl:142:24: name 'subrule' is not defined (did you mean 'rule'?) ERROR: Error computing the main repository mapping: at .../scala/private/extensions/dev_deps.bzl:8:6: at .../external/rules_java/java/repositories.bzl:20:6: at .../external/rules_java/toolchains/local_java_repository.bzl:17:6: at .../external/rules_java/java/defs.bzl:18:6: at .../external/rules_java/java/java_library.bzl:16:6: at .../external/rules_java/java/bazel/rules/bazel_java_library.bzl:21:6: compilation of module 'java/common/rules/android_lint.bzl' failed ``` `rules_java` 8.3.0 is broken, as it can't find its own `@compatibility_proxy` repo: ```txt $ bazel build //{src,test,third_party,scala_proto}/... ERROR: error loading package under directory 'src': error loading package 'src/protobuf/io/bazel/rules_scala': at .../external/rules_java/java/defs.bzl:22:6: at .../external/rules_java/java/java_test.bzl:16:6: Unable to find package for @compatibility_proxy//:proxy.bzl: The repository '@compatibility_proxy' could not be resolved: Repository '@compatibility_proxy' is not defined. ``` `rules_java` 8.3.1 seems to fix this, presumably by importing the `protobuf` repo as `com_google_protobuf`. However, it now requires at least `protobuf` v27.0, which adds `bazel/java/lite_proto_library.bzl`. Per bazelbuild#1647, we'd have to update to ScalaPB 1.0.0-alpha.1 to support `protobuf` v28, abandoning users of previous `protobuf` versions or forcing them to upgrade. ```txt $ bazel build //{src,test,third_party,scala_proto}/... [...snip...] ERROR: error loading package under directory 'src': error loading package 'src/java/io/bazel/rulesscala/worker': at .../external/rules_java/java/defs.bzl:16:6: Label '@com_google_protobuf//bazel:java_lite_proto_library.bzl' is invalid because 'bazel' is not a package; perhaps you meant to put the colon here: '@com_google_protobuf//:bazel/java_lite_proto_library.bzl'? ```
mbland
added a commit
to mbland/rules_scala
that referenced
this issue
Dec 3, 2024
Updates almost all dependencies on `@bazel_tools//tools/jdk:` toolchain targets and `.bzl` files to corresponding targets and files in `@rules_java//toolchains:`. Part of bazelbuild#1482 and bazelbuild#1652. All dependencies on `@bazel_tools//tools/jdk:toolchain_type` remain, however, as there is not yet a corresponding target in `@rules_java//toolchains:`. Adds the `WORKSPACE` stanza recommended in the `rules_java` release notes, and removes our own calls to instantiate `rules_java` repos and to register `rules_java` toolchains. These changes are required to avoid build breakages when updating to `rules_scala` 7.10.0 and higher. All targets build and all tests pass under Bazel 6.5.0 and 7.4.1. --- I was on the right track in my analysis from bazelbuild#1619 ("Bump to rules_java 7.9.0 for Bazel 7 compatibility" in the message for commit cd22d88). However, I thought we _shouldn't_ update any targets from `@bazel_tools//tools/jdk:` to `@rules_java//toolchains:`. That's why I thought we were stuck on `rules_java` 7.9.0. However, this comment from @fmeum made me think switching to `@rules_java//toolchains:` actually is the preferred approach: - bazelbuild/rules_java#214 (comment) So this is a potentially breaking change, but in the good kind of way, in that it requires an easy, future proof update.
mbland
added a commit
to mbland/rules_scala
that referenced
this issue
Dec 5, 2024
Ensures we get all the versions of dependencies we want in `WORKSPACE`, while providing a new API to consumers. Part of bazelbuild#1652. Bumps several packages as high as they can go and still be compatible with Bazel 6 and 7: - `bazel_skylib`: 1.4.1 => 1.7.1 - `rules_cc`: 0.0.6 => 0.0.9 - `rules_python`: 0.36.0 => 0.38.0 - `rules_go`: 0.50.1 The following packages are at the maximum version to prevent breakages under Bazel 6.5.0. --- `abseil-cpp` and `protobuf` have to stay at 20220623.1 and v27.1, respectively, for Bazel 6 compatibility per bazelbuild#1647. `protobuf` up to v25.5 is compatible with Bazel 6 provided users set the compiler flags mentioned in that issue. --- `rules_python` 0.38.0 => 0.39.0 requires at least `rules_cc` 0.0.10, which introduced `cc/common/cc_info.bzl`: ```txt $ bazel build //{src,test,third_party,scala_proto}/... [ ...snip... ] ERROR: error loading package under directory 'test': error loading package 'test': at .../external/rules_python/python/defs.bzl:17:6: at .../external/rules_python/python/py_binary.bzl:18:6: at .../external/rules_python/python/private/py_binary_macro.bzl:16:6: at .../external/rules_python/python/private/common_bazel.bzl:19:6: Label '@rules_cc//cc/common:cc_info.bzl' is invalid because 'cc/common' is not a package; perhaps you meant to put the colon here: '@rules_cc//cc:common/cc_info.bzl'? ``` --- `rules_cc` 0.0.9 => 0.0.10 requires Bazel 7, which defines `CcSharedLibraryHintInfo`: ```txt $ bazel build //{src,test,third_party,scala_proto}/... ERROR: .../external/rules_cc/cc/private/rules_impl/native.bzl:40:33: name 'CcSharedLibraryHintInfo' is not defined (did you mean 'CcSharedLibraryInfo'?) [ ...snip... ] ERROR: error loading package under directory 'test': error loading package 'test': at .../external/rules_python/python/defs.bzl:17:6: at .../external/rules_python/python/py_binary.bzl:18:6: at .../external/rules_python/python/private/py_binary_macro.bzl:16:6: at .../external/rules_python/python/private/common_bazel.bzl:18:6: at .../external/rules_cc/cc/common/cc_common.bzl:17:6: compilation of module 'cc/private/rules_impl/native.bzl' failed ``` --- `rules_java` remains at 7.9.0 due to the more complicated situation it presents. The current version that could be compatible with Bazel 6 and 7 is 7.12.2. As it turns out, we could include `rules_java` versions up to and including 7.12.2 if we don't call the following in `WORKSPACE`: ```py load("@rules_java//java:repositories.bzl", "rules_java_dependencies", "rules_java_toolchains") rules_java_dependencies() rules_java_toolchains() ``` In fact, __we don't do that today__. When we do, Bazel 7.4.1 builds successfully, but Bazel 6.5.2 fails with: ```txt $ bazel build //{src,test,third_party,scala_proto}/... ERROR: .../external/rules_java/toolchains/java_toolchain_alias.bzl:83:34: Use of Starlark transition without allowlist attribute '_allowlist_function_transition'. See Starlark transitions documentation for details and usage: @rules_java//toolchains:java_toolchain_alias.bzl NORMAL ERROR: .../src/java/io/bazel/rulesscala/coverage/instrumenter/BUILD:3:12: While resolving toolchains for target //src/java/io/bazel/rulesscala/coverage/instrumenter:instrumenter: invalid registered toolchain '//toolchains:all': while parsing '//toolchains:all': error loading package '@rules_java//toolchains': initialization of module 'toolchains/java_toolchain_alias.bzl' failed ERROR: Analysis of target '//src/java/io/bazel/rulesscala/coverage/instrumenter:instrumenter' failed; build aborted: ``` When we bump to rules_java 7.10.0, which contains bazelbuild/rules_java#210, Bazel 7.4.1 fails with: ```txt $ bazel build //{src,test,third_party,scala_proto}/... ERROR: .../external/rules_java_builtin/toolchains/BUILD:254:14: While resolving toolchains for target @@rules_java_builtin//toolchains:platformclasspath (096dcc8): No matching toolchains found for types @@bazel_tools//tools/jdk:bootstrap_runtime_toolchain_type. To debug, rerun with --toolchain_resolution_debug='@@bazel_tools//tools/jdk:bootstrap_runtime_toolchain_type' If platforms or toolchains are a new concept for you, we'd encourage reading https://bazel.build/concepts/platforms-intro. ERROR: Analysis of target '//test/toolchains:java21_toolchain' failed; build aborted: Analysis failed ``` This is the error I described in bazelbuild#1619, under "Bump to rules_java 7.9.0 for Bazel 7 compatibility" in the message for commit cd22d88. Today I rediscovered: - bazelbuild/rules_java: Regression with @@rules_java//toolchains:bootstrap_runtime_toolchain_type in 7.10.0 bazelbuild/rules_java#214 But even worse, Bazel 6.5.0 fails with: ```txt $ bazel build //{src,test,third_party,scala_proto}/... [ ...snip... ] ERROR: .../test/src/main/resources/java_sources/BUILD:5:14: Building test/src/main/resources/java_sources/CompiledWithJava8_java.jar (1 source file) failed: Worker process did not return a WorkResponse: ---8<---8<--- Start of log, file at .../bazel-workers/multiplex-worker-18-Javac.log ---8<---8<--- Error thrown by worker thread, shutting down worker. java.lang.NoSuchMethodError: 'java.lang.Iterable com.google.devtools.build.buildjar.javac.BlazeJavacMain$ClassloaderMaskingFileManager.getJavaFileObjectsFromPaths(java.util.Collection)' at com.google.devtools.build.buildjar.javac.BlazeJavacMain.compile(BlazeJavacMain.java:142) at com.google.devtools.build.buildjar.ReducedClasspathJavaLibraryBuilder.compileSources(ReducedClasspathJavaLibraryBuilder.java:57) at com.google.devtools.build.buildjar.SimpleJavaLibraryBuilder.compileJavaLibrary(SimpleJavaLibraryBuilder.java:110) at com.google.devtools.build.buildjar.SimpleJavaLibraryBuilder.run(SimpleJavaLibraryBuilder.java:118) at com.google.devtools.build.buildjar.BazelJavaBuilder.build(BazelJavaBuilder.java:111) at com.google.devtools.build.buildjar.BazelJavaBuilder.parseAndBuild(BazelJavaBuilder.java:91) at com.google.devtools.build.buildjar.BazelJavaBuilder.lambda$main$0(BazelJavaBuilder.java:52) at com.google.devtools.build.lib.worker.WorkRequestHandler$WorkRequestCallback.apply(WorkRequestHandler.java:252) at com.google.devtools.build.lib.worker.WorkRequestHandler.respondToRequest(WorkRequestHandler.java:480) at com.google.devtools.build.lib.worker.WorkRequestHandler.lambda$startResponseThread$1(WorkRequestHandler.java:433) at java.base/java.lang.Thread.run(Thread.java:829) ---8<---8<--- End of log ---8<---8<--- ERROR: .../test/src/main/resources/java_sources/BUILD:11:14: Building test/src/main/resources/java_sources/CompiledWithJava11_java.jar (1 source file) failed: Worker process did not return a WorkResponse: ---8<---8<--- Start of log, file at .../bazel-workers/multiplex-worker-18-Javac.log ---8<---8<--- Error thrown by worker thread, shutting down worker. java.lang.NoSuchMethodError: 'java.lang.Iterable com.google.devtools.build.buildjar.javac.BlazeJavacMain$ClassloaderMaskingFileManager.getJavaFileObjectsFromPaths(java.util.Collection)' at com.google.devtools.build.buildjar.javac.BlazeJavacMain.compile(BlazeJavacMain.java:142) at com.google.devtools.build.buildjar.ReducedClasspathJavaLibraryBuilder.compileSources(ReducedClasspathJavaLibraryBuilder.java:57) at com.google.devtools.build.buildjar.SimpleJavaLibraryBuilder.compileJavaLibrary(SimpleJavaLibraryBuilder.java:110) at com.google.devtools.build.buildjar.SimpleJavaLibraryBuilder.run(SimpleJavaLibraryBuilder.java:118) at com.google.devtools.build.buildjar.BazelJavaBuilder.build(BazelJavaBuilder.java:111) at com.google.devtools.build.buildjar.BazelJavaBuilder.parseAndBuild(BazelJavaBuilder.java:91) at com.google.devtools.build.buildjar.BazelJavaBuilder.lambda$main$0(BazelJavaBuilder.java:52) at com.google.devtools.build.lib.worker.WorkRequestHandler$WorkRequestCallback.apply(WorkRequestHandler.java:252) at com.google.devtools.build.lib.worker.WorkRequestHandler.respondToRequest(WorkRequestHandler.java:480) at com.google.devtools.build.lib.worker.WorkRequestHandler.lambda$startResponseThread$1(WorkRequestHandler.java:433) at java.base/java.lang.Thread.run(Thread.java:829) ---8<---8<--- End of log ---8<---8<--- ``` In bazelbuild#1619, I thought we wanted to keep `@bazel_tools//tools/jdk:` as the canonical Java toolchains package, instead of `@rules_java//toolchains:`. But based on this comment from @fmeum, I'm inclined to believe switching to `@rules_java//toolchains:` actually is the preferred, futureproof approach: - bazelbuild/rules_java#214 (comment) --- Bazel 8 requires `rules_java` 8, per bazelbuild#1652. We can't update to `rules_java` 8 right now, unless we decided to do the following: - Tell Bazel 6 users to add C++ compiler flags to support the newer `abseil-cpp` versions required by newer `protobuf` versions. - Abandon Bazel 6 support in favor of supporting Bazel 7 at a minimum. - In addition to either of the above cases, update ScalaPB to 1.0.0-alpha.1 or higher to support `protobuf` v28. Here are the details explaining why `rules_java` 8 currently breaks. `rules_java` 8.0.0 requires Bazel >= 7.3.2, which provides the `subrule` API. Compatibility with Bazel >= 6.3.0 isn't restored until `rules_java` 8.3.2. ```txt $ bazel build //{src,test,third_party,scala_proto}/... ERROR: .../external/rules_java/java/common/rules/android_lint.bzl:142:24: name 'subrule' is not defined (did you mean 'rule'?) ERROR: Error computing the main repository mapping: at .../scala/private/extensions/dev_deps.bzl:8:6: at .../external/rules_java/java/repositories.bzl:20:6: at .../external/rules_java/toolchains/local_java_repository.bzl:17:6: at .../external/rules_java/java/defs.bzl:18:6: at .../external/rules_java/java/java_library.bzl:16:6: at .../external/rules_java/java/bazel/rules/bazel_java_library.bzl:21:6: compilation of module 'java/common/rules/android_lint.bzl' failed ``` `rules_java` 8.3.0 is broken, as it can't find its own `@compatibility_proxy` repo: ```txt $ bazel build //{src,test,third_party,scala_proto}/... ERROR: error loading package under directory 'src': error loading package 'src/protobuf/io/bazel/rules_scala': at .../external/rules_java/java/defs.bzl:22:6: at .../external/rules_java/java/java_test.bzl:16:6: Unable to find package for @compatibility_proxy//:proxy.bzl: The repository '@compatibility_proxy' could not be resolved: Repository '@compatibility_proxy' is not defined. ``` `rules_java` 8.3.1 seems to fix this, presumably by importing the `protobuf` repo as `com_google_protobuf`. However, it now requires at least `protobuf` v27.0, which adds `bazel/java/lite_proto_library.bzl`. Per bazelbuild#1647, we'd have to update to ScalaPB 1.0.0-alpha.1 to support `protobuf` v28, abandoning users of previous `protobuf` versions or forcing them to upgrade. ```txt $ bazel build //{src,test,third_party,scala_proto}/... [...snip...] ERROR: error loading package under directory 'src': error loading package 'src/java/io/bazel/rulesscala/worker': at .../external/rules_java/java/defs.bzl:16:6: Label '@com_google_protobuf//bazel:java_lite_proto_library.bzl' is invalid because 'bazel' is not a package; perhaps you meant to put the colon here: '@com_google_protobuf//:bazel/java_lite_proto_library.bzl'? ```
mbland
added a commit
to mbland/rules_scala
that referenced
this issue
Dec 8, 2024
Ensures we get all the versions of dependencies we want in `WORKSPACE`, while providing a new API to consumers. Part of bazelbuild#1652. Bumps several packages as high as they can go and still be compatible with Bazel 6 and 7: - `bazel_skylib`: 1.4.1 => 1.7.1 - `rules_cc`: 0.0.6 => 0.0.9 - `rules_python`: 0.36.0 => 0.38.0 - `rules_go`: 0.50.1 The following packages are at the maximum version to prevent breakages under Bazel 6.5.0. --- `abseil-cpp` and `protobuf` have to stay at 20220623.1 and v27.1, respectively, for Bazel 6 compatibility per bazelbuild#1647. `protobuf` up to v25.5 is compatible with Bazel 6 provided users set the compiler flags mentioned in that issue. --- `rules_python` 0.38.0 => 0.39.0 requires at least `rules_cc` 0.0.10, which introduced `cc/common/cc_info.bzl`: ```txt $ bazel build //{src,test,third_party,scala_proto}/... [ ...snip... ] ERROR: error loading package under directory 'test': error loading package 'test': at .../external/rules_python/python/defs.bzl:17:6: at .../external/rules_python/python/py_binary.bzl:18:6: at .../external/rules_python/python/private/py_binary_macro.bzl:16:6: at .../external/rules_python/python/private/common_bazel.bzl:19:6: Label '@rules_cc//cc/common:cc_info.bzl' is invalid because 'cc/common' is not a package; perhaps you meant to put the colon here: '@rules_cc//cc:common/cc_info.bzl'? ``` --- `rules_cc` 0.0.9 => 0.0.10 requires Bazel 7, which defines `CcSharedLibraryHintInfo`: ```txt $ bazel build //{src,test,third_party,scala_proto}/... ERROR: .../external/rules_cc/cc/private/rules_impl/native.bzl:40:33: name 'CcSharedLibraryHintInfo' is not defined (did you mean 'CcSharedLibraryInfo'?) [ ...snip... ] ERROR: error loading package under directory 'test': error loading package 'test': at .../external/rules_python/python/defs.bzl:17:6: at .../external/rules_python/python/py_binary.bzl:18:6: at .../external/rules_python/python/private/py_binary_macro.bzl:16:6: at .../external/rules_python/python/private/common_bazel.bzl:18:6: at .../external/rules_cc/cc/common/cc_common.bzl:17:6: compilation of module 'cc/private/rules_impl/native.bzl' failed ``` --- `rules_java` remains at 7.9.0 due to the more complicated situation it presents. The current version that could be compatible with Bazel 6 and 7 is 7.12.2. As it turns out, we could include `rules_java` versions up to and including 7.12.2 if we don't call the following in `WORKSPACE`: ```py load("@rules_java//java:repositories.bzl", "rules_java_dependencies", "rules_java_toolchains") rules_java_dependencies() rules_java_toolchains() ``` In fact, __we don't do that today__. When we do, Bazel 7.4.1 builds successfully, but Bazel 6.5.2 fails with: ```txt $ bazel build //{src,test,third_party,scala_proto}/... ERROR: .../external/rules_java/toolchains/java_toolchain_alias.bzl:83:34: Use of Starlark transition without allowlist attribute '_allowlist_function_transition'. See Starlark transitions documentation for details and usage: @rules_java//toolchains:java_toolchain_alias.bzl NORMAL ERROR: .../src/java/io/bazel/rulesscala/coverage/instrumenter/BUILD:3:12: While resolving toolchains for target //src/java/io/bazel/rulesscala/coverage/instrumenter:instrumenter: invalid registered toolchain '//toolchains:all': while parsing '//toolchains:all': error loading package '@rules_java//toolchains': initialization of module 'toolchains/java_toolchain_alias.bzl' failed ERROR: Analysis of target '//src/java/io/bazel/rulesscala/coverage/instrumenter:instrumenter' failed; build aborted: ``` When we bump to rules_java 7.10.0, which contains bazelbuild/rules_java#210, Bazel 7.4.1 fails with: ```txt $ bazel build //{src,test,third_party,scala_proto}/... ERROR: .../external/rules_java_builtin/toolchains/BUILD:254:14: While resolving toolchains for target @@rules_java_builtin//toolchains:platformclasspath (096dcc8): No matching toolchains found for types @@bazel_tools//tools/jdk:bootstrap_runtime_toolchain_type. To debug, rerun with --toolchain_resolution_debug='@@bazel_tools//tools/jdk:bootstrap_runtime_toolchain_type' If platforms or toolchains are a new concept for you, we'd encourage reading https://bazel.build/concepts/platforms-intro. ERROR: Analysis of target '//test/toolchains:java21_toolchain' failed; build aborted: Analysis failed ``` This is the error I described in bazelbuild#1619, under "Bump to rules_java 7.9.0 for Bazel 7 compatibility" in the message for commit cd22d88. Today I rediscovered: - bazelbuild/rules_java: Regression with @@rules_java//toolchains:bootstrap_runtime_toolchain_type in 7.10.0 bazelbuild/rules_java#214 But even worse, Bazel 6.5.0 fails with: ```txt $ bazel build //{src,test,third_party,scala_proto}/... [ ...snip... ] ERROR: .../test/src/main/resources/java_sources/BUILD:5:14: Building test/src/main/resources/java_sources/CompiledWithJava8_java.jar (1 source file) failed: Worker process did not return a WorkResponse: ---8<---8<--- Start of log, file at .../bazel-workers/multiplex-worker-18-Javac.log ---8<---8<--- Error thrown by worker thread, shutting down worker. java.lang.NoSuchMethodError: 'java.lang.Iterable com.google.devtools.build.buildjar.javac.BlazeJavacMain$ClassloaderMaskingFileManager.getJavaFileObjectsFromPaths(java.util.Collection)' at com.google.devtools.build.buildjar.javac.BlazeJavacMain.compile(BlazeJavacMain.java:142) at com.google.devtools.build.buildjar.ReducedClasspathJavaLibraryBuilder.compileSources(ReducedClasspathJavaLibraryBuilder.java:57) at com.google.devtools.build.buildjar.SimpleJavaLibraryBuilder.compileJavaLibrary(SimpleJavaLibraryBuilder.java:110) at com.google.devtools.build.buildjar.SimpleJavaLibraryBuilder.run(SimpleJavaLibraryBuilder.java:118) at com.google.devtools.build.buildjar.BazelJavaBuilder.build(BazelJavaBuilder.java:111) at com.google.devtools.build.buildjar.BazelJavaBuilder.parseAndBuild(BazelJavaBuilder.java:91) at com.google.devtools.build.buildjar.BazelJavaBuilder.lambda$main$0(BazelJavaBuilder.java:52) at com.google.devtools.build.lib.worker.WorkRequestHandler$WorkRequestCallback.apply(WorkRequestHandler.java:252) at com.google.devtools.build.lib.worker.WorkRequestHandler.respondToRequest(WorkRequestHandler.java:480) at com.google.devtools.build.lib.worker.WorkRequestHandler.lambda$startResponseThread$1(WorkRequestHandler.java:433) at java.base/java.lang.Thread.run(Thread.java:829) ---8<---8<--- End of log ---8<---8<--- ERROR: .../test/src/main/resources/java_sources/BUILD:11:14: Building test/src/main/resources/java_sources/CompiledWithJava11_java.jar (1 source file) failed: Worker process did not return a WorkResponse: ---8<---8<--- Start of log, file at .../bazel-workers/multiplex-worker-18-Javac.log ---8<---8<--- Error thrown by worker thread, shutting down worker. java.lang.NoSuchMethodError: 'java.lang.Iterable com.google.devtools.build.buildjar.javac.BlazeJavacMain$ClassloaderMaskingFileManager.getJavaFileObjectsFromPaths(java.util.Collection)' at com.google.devtools.build.buildjar.javac.BlazeJavacMain.compile(BlazeJavacMain.java:142) at com.google.devtools.build.buildjar.ReducedClasspathJavaLibraryBuilder.compileSources(ReducedClasspathJavaLibraryBuilder.java:57) at com.google.devtools.build.buildjar.SimpleJavaLibraryBuilder.compileJavaLibrary(SimpleJavaLibraryBuilder.java:110) at com.google.devtools.build.buildjar.SimpleJavaLibraryBuilder.run(SimpleJavaLibraryBuilder.java:118) at com.google.devtools.build.buildjar.BazelJavaBuilder.build(BazelJavaBuilder.java:111) at com.google.devtools.build.buildjar.BazelJavaBuilder.parseAndBuild(BazelJavaBuilder.java:91) at com.google.devtools.build.buildjar.BazelJavaBuilder.lambda$main$0(BazelJavaBuilder.java:52) at com.google.devtools.build.lib.worker.WorkRequestHandler$WorkRequestCallback.apply(WorkRequestHandler.java:252) at com.google.devtools.build.lib.worker.WorkRequestHandler.respondToRequest(WorkRequestHandler.java:480) at com.google.devtools.build.lib.worker.WorkRequestHandler.lambda$startResponseThread$1(WorkRequestHandler.java:433) at java.base/java.lang.Thread.run(Thread.java:829) ---8<---8<--- End of log ---8<---8<--- ``` In bazelbuild#1619, I thought we wanted to keep `@bazel_tools//tools/jdk:` as the canonical Java toolchains package, instead of `@rules_java//toolchains:`. But based on this comment from @fmeum, I'm inclined to believe switching to `@rules_java//toolchains:` actually is the preferred, futureproof approach: - bazelbuild/rules_java#214 (comment) --- Bazel 8 requires `rules_java` 8, per bazelbuild#1652. We can't update to `rules_java` 8 right now, unless we decided to do the following: - Tell Bazel 6 users to add C++ compiler flags to support the newer `abseil-cpp` versions required by newer `protobuf` versions. - Abandon Bazel 6 support in favor of supporting Bazel 7 at a minimum. - In addition to either of the above cases, update ScalaPB to 1.0.0-alpha.1 or higher to support `protobuf` v28. Here are the details explaining why `rules_java` 8 currently breaks. `rules_java` 8.0.0 requires Bazel >= 7.3.2, which provides the `subrule` API. Compatibility with Bazel >= 6.3.0 isn't restored until `rules_java` 8.3.2. ```txt $ bazel build //{src,test,third_party,scala_proto}/... ERROR: .../external/rules_java/java/common/rules/android_lint.bzl:142:24: name 'subrule' is not defined (did you mean 'rule'?) ERROR: Error computing the main repository mapping: at .../scala/private/extensions/dev_deps.bzl:8:6: at .../external/rules_java/java/repositories.bzl:20:6: at .../external/rules_java/toolchains/local_java_repository.bzl:17:6: at .../external/rules_java/java/defs.bzl:18:6: at .../external/rules_java/java/java_library.bzl:16:6: at .../external/rules_java/java/bazel/rules/bazel_java_library.bzl:21:6: compilation of module 'java/common/rules/android_lint.bzl' failed ``` `rules_java` 8.3.0 is broken, as it can't find its own `@compatibility_proxy` repo: ```txt $ bazel build //{src,test,third_party,scala_proto}/... ERROR: error loading package under directory 'src': error loading package 'src/protobuf/io/bazel/rules_scala': at .../external/rules_java/java/defs.bzl:22:6: at .../external/rules_java/java/java_test.bzl:16:6: Unable to find package for @compatibility_proxy//:proxy.bzl: The repository '@compatibility_proxy' could not be resolved: Repository '@compatibility_proxy' is not defined. ``` `rules_java` 8.3.1 seems to fix this, presumably by importing the `protobuf` repo as `com_google_protobuf`. However, it now requires at least `protobuf` v27.0, which adds `bazel/java/lite_proto_library.bzl`. Per bazelbuild#1647, we'd have to update to ScalaPB 1.0.0-alpha.1 to support `protobuf` v28, abandoning users of previous `protobuf` versions or forcing them to upgrade. ```txt $ bazel build //{src,test,third_party,scala_proto}/... [...snip...] ERROR: error loading package under directory 'src': error loading package 'src/java/io/bazel/rulesscala/worker': at .../external/rules_java/java/defs.bzl:16:6: Label '@com_google_protobuf//bazel:java_lite_proto_library.bzl' is invalid because 'bazel' is not a package; perhaps you meant to put the colon here: '@com_google_protobuf//:bazel/java_lite_proto_library.bzl'? ```
mbland
added a commit
to mbland/rules_scala
that referenced
this issue
Dec 8, 2024
Updates almost all dependencies on `@bazel_tools//tools/jdk:` toolchain targets and `.bzl` files to corresponding targets and files in `@rules_java//toolchains:`. Part of bazelbuild#1482 and bazelbuild#1652. All dependencies on `@bazel_tools//tools/jdk:toolchain_type` remain, however, as there is not yet a corresponding target in `@rules_java//toolchains:`. Adds the `WORKSPACE` stanza recommended in the `rules_java` release notes, and removes our own calls to instantiate `rules_java` repos and to register `rules_java` toolchains. These changes are required to avoid build breakages when updating to `rules_scala` 7.10.0 and higher. All targets build and all tests pass under Bazel 6.5.0 and 7.4.1. --- I was on the right track in my analysis from bazelbuild#1619 ("Bump to rules_java 7.9.0 for Bazel 7 compatibility" in the message for commit cd22d88). However, I thought we _shouldn't_ update any targets from `@bazel_tools//tools/jdk:` to `@rules_java//toolchains:`. That's why I thought we were stuck on `rules_java` 7.9.0. However, this comment from @fmeum made me think switching to `@rules_java//toolchains:` actually is the preferred approach: - bazelbuild/rules_java#214 (comment) So this is a potentially breaking change, but in the good kind of way, in that it requires an easy, future proof update.
mbland
added a commit
to mbland/rules_scala
that referenced
this issue
Dec 9, 2024
Moves `rules_scala_dependencies` to `scala/deps.bzl` and bumps several dependencies as high as they can go and still be compatible with Bazel 6.5.0 and 7.4.1. - `bazel_skylib`: 1.4.1 => 1.7.1 - `go`: 1.19.5 => 1.23.4 - `rules_cc`: 0.0.6 => 0.0.9 - `rules_go`: 0.39.1 => 0.50.1 - `rules_java`: 7.9.0 => 7.12.3 - `rules_python`: 0.36.0 => 0.38.0 The `rules_java` 7.12.13 bump precipitated the following changes: - Adds the `WORKSPACE` stanza for `rules_java` in every `WORKSPACE` file per https://github.com/bazelbuild/rules_java/releases/tag/7.12.3. This replaces previous calls to instantiate `rules_java` repos and to register `rules_java` toolchains. - Rearranges the other `WORKSPACE` setup macros for dependencies to come before the `rules_scala` setup macros. - Updates almost all dependencies on `@bazel_tools//tools/jdk:` toolchain targets and `.bzl` files to corresponding targets and files in `@rules_java//toolchains:`. - Removes several deprecated flags from the `scrooge_compile_with_jdk_11` test case from `test/shell/test_twitter_scrooge.sh`, and one obsolete flag that caused it to break. --- Moving `rules_scala_dependencies` to `scala/deps.bzl` ensures we get all the versions of dependencies we want in `WORKSPACE`, while providing a new API to consumers. It also prevents `WORKSPACE` from transitively loading any `.bzl` files that load `@rules_java`, ensuring Bazel 8 compatibility per bazelbuild#1652. Reasons for the other `rules_java` related changes include: - The `WORKSPACE` stanza for `rules_java` should've already been present while using the existing version 7.9.0. However, doing so would've broken Bazel 6 builds, as described in detail below. - Having the other `WORKSPACE` setup macros for dependencies come before the `rules_scala` setup macros helps ensure consistent, correct initialization before `rules_scala` initialization. - Updating the toolchain specifiers to use `@rules_java//toolchains` fixed `WORKSPACE` build breakages when updating to `rules_java` 7.10.0 and later. This is a potentially breaking change for consumers, but in the good kind of way, in that it requires an easy, futureproof update. (`@bazel_tools//tools/jdk:toolchain_type` dependencies remain, as there's not yet a corresponding `@rules_java//toolchains` target.) What follows are detailed notes on why some dependency versions are capped where they are, and on some breakages fixed by these changes. --- `abseil-cpp` and `protobuf` have to stay at 20220623.1 and v21.7, respectively, for Bazel 6 compatibility per bazelbuild#1647. `protobuf` up to v25.5 is compatible with Bazel 6 provided users set the compiler flags mentioned in that issue: ```txt build:linux --cxxopt=-std=c++17 build:linux --host_cxxopt=-std=c++17 build:macos --cxxopt=-std=c++17 build:macos --host_cxxopt=-std=c++17 build:windows --cxxopt=/std=c++17 build:windows --host_cxxopt=/std=c++17 ``` --- `rules_python` 0.38.0 => 0.39.0 requires at least `rules_cc` 0.0.10, which introduced `cc/common/cc_info.bzl`: ```txt $ bazel build //{src,test,third_party,scala_proto}/... [ ...snip... ] ERROR: error loading package under directory 'test': error loading package 'test': at .../external/rules_python/python/defs.bzl:17:6: at .../external/rules_python/python/py_binary.bzl:18:6: at .../external/rules_python/python/private/py_binary_macro.bzl:16:6: at .../external/rules_python/python/private/common_bazel.bzl:19:6: Label '@rules_cc//cc/common:cc_info.bzl' is invalid because 'cc/common' is not a package; perhaps you meant to put the colon here: '@rules_cc//cc:common/cc_info.bzl'? ``` However, `rules_cc` 0.0.9 => 0.0.10 requires Bazel 7, which defines `CcSharedLibraryHintInfo`: ```txt $ bazel build //{src,test,third_party,scala_proto}/... ERROR: .../external/rules_cc/cc/private/rules_impl/native.bzl:40:33: name 'CcSharedLibraryHintInfo' is not defined (did you mean 'CcSharedLibraryInfo'?) [ ...snip... ] ERROR: error loading package under directory 'test': error loading package 'test': at .../external/rules_python/python/defs.bzl:17:6: at .../external/rules_python/python/py_binary.bzl:18:6: at .../external/rules_python/python/private/py_binary_macro.bzl:16:6: at .../external/rules_python/python/private/common_bazel.bzl:18:6: at .../external/rules_cc/cc/common/cc_common.bzl:17:6: compilation of module 'cc/private/rules_impl/native.bzl' failed ``` --- This section provides a methodical description of the errors encountered and resolved during each step of the `rules_java` update. The proper `WORKSPACE` setup stanza for `rules_java`, which we didn't originally have in place, is: ```py load( "@rules_java//java:repositories.bzl", "rules_java_dependencies", "rules_java_toolchains", ) rules_java_dependencies() rules_java_toolchains() ``` Adding this stanza to `WORKSPACE` while still using `rules_java` 7.9.0 didn't break the Bazel 7.4.1 build, but it broke the Bazel 6.5.0 build. Note the `CompiledWithJava{8,11}_java.jar` references: ```txt $ bazel build //{src,test,third_party,scala_proto}/... [ ...snip... ] ERROR: .../test/src/main/resources/java_sources/BUILD:5:14: Building test/src/main/resources/java_sources/CompiledWithJava8_java.jar (1 source file) failed: Worker process did not return a WorkResponse: ---8<---8<--- Start of log, file at .../bazel-workers/multiplex-worker-18-Javac.log ---8<---8<--- Error thrown by worker thread, shutting down worker. java.lang.NoSuchMethodError: 'java.lang.Iterable com.google.devtools.build.buildjar.javac.BlazeJavacMain$ClassloaderMaskingFileManager.getJavaFileObjectsFromPaths(java.util.Collection)' at com.google.devtools.build.buildjar.javac.BlazeJavacMain.compile(BlazeJavacMain.java:142) [ ...snip... ] ---8<---8<--- End of log ---8<---8<--- ERROR: .../test/src/main/resources/java_sources/BUILD:11:14: Building test/src/main/resources/java_sources/CompiledWithJava11_java.jar (1 source file) failed: Worker process did not return a WorkResponse: ---8<---8<--- Start of log, file at .../bazel-workers/multiplex-worker-18-Javac.log ---8<---8<--- Error thrown by worker thread, shutting down worker. java.lang.NoSuchMethodError: 'java.lang.Iterable com.google.devtools.build.buildjar.javac.BlazeJavacMain$ClassloaderMaskingFileManager.getJavaFileObjectsFromPaths(java.util.Collection)' at com.google.devtools.build.buildjar.javac.BlazeJavacMain.compile(BlazeJavacMain.java:142) [ ...snip... ] ---8<---8<--- End of log ---8<---8<--- ``` Another variation of this I also saw was: ```txt $ bazel build //{src,jmh,test,third_party,scala_proto}/... [ ...snip... ] ERROR: test/src/main/resources/java_sources/BUILD:11:14: Building test/src/main/resources/java_sources/CompiledWithJava11_java.jar (1 source file) failed: Worker process did not return a WorkResponse: ---8<---8<--- Start of log, file at .../bazel-workers/multiplex-worker-33-Javac.log ---8<---8<--- Error thrown by worker thread, shutting down worker. java.lang.UnsupportedClassVersionError: com/google/errorprone/ErrorProneError has been compiled by a more recent version of the Java Runtime (class file version 61.0), this version of the Java Runtime only recognizes class file versions up to 55.0 at java.base/java.lang.ClassLoader.defineClass1(Native Method) [ ...snip... ] ERROR: test/src/main/resources/java_sources/BUILD:5:14: Building test/src/main/resources/java_sources/CompiledWithJava8_java.jar (1 source file) failed: Worker process did not return a WorkResponse: ---8<---8<--- Start of log, file at .../bazel-workers/multiplex-worker-33-Javac.log ---8<---8<--- Error thrown by worker thread, shutting down worker. java.lang.UnsupportedClassVersionError: com/google/errorprone/ErrorProneError has been compiled by a more recent version of the Java Runtime (class file version 61.0), this version of the Java Runtime only recognizes class file versions up to 55.0 at java.base/java.lang.ClassLoader.defineClass1(Native Method) [ ...snip... ] ``` Updating the toolchains from `@bazel_tools//tools/jdk:` to `@rules_java//toolchains:` for targets in `test/BUILD` and `test/src/main/resources/java_sources/BUILD` fixed this particular issue. (More on updating other `@bazel_tools//tools/jdk` toolchain targets below.) Bazel 6.5.0 then failed with the following expected issue, fixed by `rules_java` 7.10.0 via: - bazelbuild/rules_java#210 - bazelbuild/rules_java@30ecf3f ```txt $ bazel build //{src,test,third_party,scala_proto}/... ERROR: .../external/rules_java/toolchains/java_toolchain_alias.bzl:83:34: Use of Starlark transition without allowlist attribute '_allowlist_function_transition'. See Starlark transitions documentation for details and usage: @rules_java//toolchains:java_toolchain_alias.bzl NORMAL ERROR: .../src/java/io/bazel/rulesscala/coverage/instrumenter/BUILD:3:12: While resolving toolchains for target //src/java/io/bazel/rulesscala/coverage/instrumenter:instrumenter: invalid registered toolchain '//toolchains:all': while parsing '//toolchains:all': error loading package '@rules_java//toolchains': initialization of module 'toolchains/java_toolchain_alias.bzl' failed ERROR: Analysis of target '//src/java/io/bazel/rulesscala/coverage/instrumenter:instrumenter' failed; build aborted: ``` Updating to `rules_java` 7.10.0 fixed the Bazel 6.5.0 build, but caused Bazel 7.4.1 to fail with the error I originally described in bazelbuild#1619. For details, see "Bump to rules_java 7.9.0 for Bazel 7 compatibility" in the message for commit cd22d88. ```txt $ bazel build //{src,test,third_party,scala_proto}/... ERROR: .../external/rules_java_builtin/toolchains/BUILD:254:14: While resolving toolchains for target @@rules_java_builtin//toolchains:platformclasspath (096dcc8): No matching toolchains found for types @@bazel_tools//tools/jdk:bootstrap_runtime_toolchain_type. To debug, rerun with --toolchain_resolution_debug='@@bazel_tools//tools/jdk:bootstrap_runtime_toolchain_type' If platforms or toolchains are a new concept for you, we'd encourage reading https://bazel.build/concepts/platforms-intro. ERROR: Analysis of target '//test/toolchains:java21_toolchain' failed; build aborted: Analysis failed ``` My analysis from bazelbuild#1619 _was_ on the right track. The `rules_java` 7.9.0 built into Bazel's `WORKSPACE` preamble registered `@bazel_tools//tools/jdk:bootstrap_runtime_toolchain_type` toolchains that later versions of `rules_java` couldn't resolve. This was due to the very same change that fixed 6.5.0, bazelbuild/rules_java#210. However, I mistakenly thought we needed to keep `@bazel_tools//tools/jdk:` as the canonical Java toolchains package instead switching to `@rules_java//toolchains:`. That's why I originally thought we were stuck on `rules_java` 7.9.0. I eventually rediscovered the following issue, which surfaced this problem a month before bazelbuild#1619. The conversation helped me realize that switching to `@rules_java//toolchains:` actually is the preferred, futureproof approach (short of migrating to Bzlmod): - bazelbuild/rules_java: Regression with `@@rules_java//toolchains:bootstrap_runtime_toolchain_type` in 7.10.0 bazelbuild/rules_java#214 Switching all `@bazel_tools//tools/jdk:` toolchains to `@rules_java//toolchains:` resolved this issue, enabling Bazel 6.5.0 and 7.4.1 to successfully build using `rules_java` 7.12.2. I then updated `rules_java` to 7.12.3, which returns to registering some toolchains as `@bazel_tools//tools/jdk:bootstrap_runtime_toolchain_type`: - bazelbuild/rules_java#246 - bazelbuild/rules_java@b64eb7d @hvadehra requested that I try 7.12.3 to see if it resolved the issue. I was able to build successfully using this version in a branch without the toolchain updates from this commit. - bazelbuild#1652 (comment) However, the changes from this commit should remain futureproof when `@bazel_tools//tools/jdk:bootstrap_runtime_toolchain_type` moves to `@rules_java//toolchains:bootstrap_runtime_toolchain_type` again one day. --- Several of the flags removed from the `scrooge_compile_with_jdk_11` test case have been no-ops for years, so removing them eliminated their corresponding deprecation warnings. `--javacopt='--release 11'`, however, caused this breakage after originally bumping to `rules_java` 7.12.2: ```txt $ RULES_SCALA_TEST_ONLY="scrooge_compile_with_jdk_11" bash ./test/shell/test_twitter_scrooge.sh running test scrooge_compile_with_jdk_11 WARNING: Option 'javabase' is deprecated WARNING: Option 'host_javabase' is deprecated WARNING: Option 'host_java_toolchain' is deprecated WARNING: Option 'java_toolchain' is deprecated WARNING: Option 'javabase' is deprecated WARNING: Option 'host_javabase' is deprecated WARNING: Option 'host_java_toolchain' is deprecated WARNING: Option 'java_toolchain' is deprecated INFO: Analyzed 64 targets (0 packages loaded, 0 targets configured). INFO: Found 64 targets... ERROR: .../src/java/io/bazel/rulesscala/scalac/compileoptions/BUILD:3:13: Compiling Java headers src/java/io/bazel/rulesscala/scalac/compileoptions/libcompileoptions-hjar.jar (1 source file) failed: (Exit 1): turbine_direct_graal failed: error executing command (from target //src/java/io/bazel/rulesscala/scalac/compileoptions:compileoptions) external/remote_java_tools_darwin_arm64/java_tools/turbine_direct_graal --output bazel-out/darwin_arm64-fastbuild/bin/src/java/io/bazel/rulesscala/scalac/compileoptions/libcompileoptions-hjar.jar ... (remaining 32 arguments skipped) Use --sandbox_debug to see verbose messages from the sandbox and retain the sandbox build root for debugging java.lang.NullPointerException: attempted to use --release, but JAVA_HOME is not set at [email protected]/java.util.Objects.requireNonNull(Objects.java:259) at com.google.turbine.binder.CtSymClassBinder.bind(CtSymClassBinder.java:55) at com.google.turbine.main.Main.release(Main.java:318) at com.google.turbine.main.Main.bootclasspath(Main.java:304) at com.google.turbine.main.Main.compile(Main.java:142) at com.google.turbine.main.Main.compile(Main.java:133) at com.google.turbine.main.Main.main(Main.java:89) at [email protected]/java.lang.invoke.LambdaForm$DMH/sa346b79c.invokeStaticInit(LambdaForm$DMH) INFO: Elapsed time: 0.325s, Critical Path: 0.12s INFO: 11 processes: 10 internal, 1 worker. FAILED: Build did NOT complete successfully Test "scrooge_compile_with_jdk_11" failed (0 sec) ``` See also: - Bazel Blog: Bazel 5.0: https://blog.bazel.build/2022/01/19/bazel-5.0.html#java - bazelbuild/bazel: `incompatible_use_toolchain_resolution_for_java_rules`: use toolchain resolution for Java rules #7849 bazelbuild/bazel#7849 --- Bazel 8 requires `rules_java` 8, per bazelbuild#1652. Updating to `rules_java` 8 will likely happen in a new major release _after_ a release containing the Bzlmod changes. This is because: - The Bzlmod changes alone will work with Bazel 6 without requiring that users update `protobuf` beyond version 21.7. - Bazel 8 requires `protobuf` >= 29.0. Bazel 6 users would have to add the afforementioned C++ compiler flags to support the newer `abseil-cpp` versions required by newer `protobuf` versions. - `rules_scala` itself needs to update ScalaPB to 1.0.0-alpha.1 or higher to support `protobuf` >= v28.0, but this ScalaPB version will _not_ support `protobuf` < v28.0. The idea is that the next major release of `rules_scala` will help users migrate to Bazel 7 and Bzlmod, in either order. Then the next major release after that will enable them to migrate to Bazel 8, with all the requisite dependency upgrades. (Technically it will still support `WORKSPACE`, but hopefully they'll make the jump to Bzlmod first.) See: - bazelbuild#1482 (comment) --- Here are the details explaining how `rules_java` 8 currently breaks `rules_scala`, up until the point that upgrading to `protobuf` >= v29.0 would fix it. `rules_java` 8.0.0 requires Bazel >= 7.3.2, which provides the `subrule` API. Compatibility with Bazel >= 6.3.0 isn't restored until `rules_java` 8.3.2. ```txt $ bazel build //{src,test,third_party,scala_proto}/... ERROR: .../external/rules_java/java/common/rules/android_lint.bzl:142:24: name 'subrule' is not defined (did you mean 'rule'?) ERROR: Error computing the main repository mapping: at .../scala/private/extensions/dev_deps.bzl:8:6: at .../external/rules_java/java/repositories.bzl:20:6: at .../external/rules_java/toolchains/local_java_repository.bzl:17:6: at .../external/rules_java/java/defs.bzl:18:6: at .../external/rules_java/java/java_library.bzl:16:6: at .../external/rules_java/java/bazel/rules/bazel_java_library.bzl:21:6: compilation of module 'java/common/rules/android_lint.bzl' failed ``` `rules_java` 8.3.0 is broken, as it can't find its own `@compatibility_proxy` repo: ```txt $ bazel build //{src,test,third_party,scala_proto}/... ERROR: error loading package under directory 'src': error loading package 'src/protobuf/io/bazel/rules_scala': at .../external/rules_java/java/defs.bzl:22:6: at .../external/rules_java/java/java_test.bzl:16:6: Unable to find package for @compatibility_proxy//:proxy.bzl: The repository '@compatibility_proxy' could not be resolved: Repository '@compatibility_proxy' is not defined. ``` `rules_java` 8.3.1 seems to fix this, presumably by importing the `protobuf` repo as `com_google_protobuf`. However, it now requires at least `protobuf` v27.0, which adds `bazel/java/lite_proto_library.bzl`. Per bazelbuild#1647, we'd have to update to ScalaPB 1.0.0-alpha.1 to support `protobuf` v28, abandoning users of previous `protobuf` versions or forcing them to upgrade. ```txt $ bazel build //{src,test,third_party,scala_proto}/... [...snip...] ERROR: error loading package under directory 'src': error loading package 'src/java/io/bazel/rulesscala/worker': at .../external/rules_java/java/defs.bzl:16:6: Label '@com_google_protobuf//bazel:java_lite_proto_library.bzl' is invalid because 'bazel' is not a package; perhaps you meant to put the colon here: '@com_google_protobuf//:bazel/java_lite_proto_library.bzl'? ```
mbland
added a commit
to mbland/rules_scala
that referenced
this issue
Dec 10, 2024
Moves `rules_scala_dependencies` to `scala/deps.bzl` and bumps several dependencies as high as they can go and still be compatible with Bazel 6.5.0 and 7.4.1. - `bazel_skylib`: 1.4.1 => 1.7.1 - `go`: 1.19.5 => 1.23.4 - `rules_cc`: 0.0.6 => 0.0.9 - `rules_go`: 0.39.1 => 0.50.1 - `rules_java`: 7.9.0 => 7.12.3 - `rules_python`: 0.36.0 => 0.38.0 The `rules_java` 7.12.13 bump precipitated the following changes: - Adds the `WORKSPACE` stanza for `rules_java` in every `WORKSPACE` file per https://github.com/bazelbuild/rules_java/releases/tag/7.12.3. This replaces previous calls to instantiate `rules_java` repos and to register `rules_java` toolchains. - Rearranges the other `WORKSPACE` setup macros for dependencies to come before the `rules_scala` setup macros. - Updates almost all dependencies on `@bazel_tools//tools/jdk:` toolchain targets and `.bzl` files to corresponding targets and files in `@rules_java//toolchains:`. - Removes several deprecated flags from the `scrooge_compile_with_jdk_11` test case from `test/shell/test_twitter_scrooge.sh`, and one obsolete flag that caused it to break. --- Moving `rules_scala_dependencies` to `scala/deps.bzl` ensures we get all the versions of dependencies we want in `WORKSPACE`, while providing a new API to consumers. It also prevents `WORKSPACE` from transitively loading any `.bzl` files that load `@rules_java`, ensuring Bazel 8 compatibility per bazelbuild#1652. Reasons for the other `rules_java` related changes include: - The `WORKSPACE` stanza for `rules_java` should've already been present while using the existing version 7.9.0. However, doing so would've broken Bazel 6 builds, as described in detail below. - Having the other `WORKSPACE` setup macros for dependencies come before the `rules_scala` setup macros helps ensure consistent, correct initialization before `rules_scala` initialization. - Updating the toolchain specifiers to use `@rules_java//toolchains` fixed `WORKSPACE` build breakages when updating to `rules_java` 7.10.0 and later. This is a potentially breaking change for consumers, but in the good kind of way, in that it requires an easy, futureproof update. (`@bazel_tools//tools/jdk:toolchain_type` dependencies remain, as there's not yet a corresponding `@rules_java//toolchains` target.) What follows are detailed notes on why some dependency versions are capped where they are, and on some breakages fixed by these changes. --- `abseil-cpp` and `protobuf` have to stay at 20220623.1 and v21.7, respectively, for Bazel 6 compatibility per bazelbuild#1647. `protobuf` up to v25.5 is compatible with Bazel 6 provided users set the compiler flags mentioned in that issue: ```txt build:linux --cxxopt=-std=c++17 build:linux --host_cxxopt=-std=c++17 build:macos --cxxopt=-std=c++17 build:macos --host_cxxopt=-std=c++17 build:windows --cxxopt=/std=c++17 build:windows --host_cxxopt=/std=c++17 ``` --- `rules_python` 0.38.0 => 0.39.0 requires at least `rules_cc` 0.0.10, which introduced `cc/common/cc_info.bzl`: ```txt $ bazel build //{src,test,third_party,scala_proto}/... [ ...snip... ] ERROR: error loading package under directory 'test': error loading package 'test': at .../external/rules_python/python/defs.bzl:17:6: at .../external/rules_python/python/py_binary.bzl:18:6: at .../external/rules_python/python/private/py_binary_macro.bzl:16:6: at .../external/rules_python/python/private/common_bazel.bzl:19:6: Label '@rules_cc//cc/common:cc_info.bzl' is invalid because 'cc/common' is not a package; perhaps you meant to put the colon here: '@rules_cc//cc:common/cc_info.bzl'? ``` However, `rules_cc` 0.0.9 => 0.0.10 requires Bazel 7, which defines `CcSharedLibraryHintInfo`: ```txt $ bazel build //{src,test,third_party,scala_proto}/... ERROR: .../external/rules_cc/cc/private/rules_impl/native.bzl:40:33: name 'CcSharedLibraryHintInfo' is not defined (did you mean 'CcSharedLibraryInfo'?) [ ...snip... ] ERROR: error loading package under directory 'test': error loading package 'test': at .../external/rules_python/python/defs.bzl:17:6: at .../external/rules_python/python/py_binary.bzl:18:6: at .../external/rules_python/python/private/py_binary_macro.bzl:16:6: at .../external/rules_python/python/private/common_bazel.bzl:18:6: at .../external/rules_cc/cc/common/cc_common.bzl:17:6: compilation of module 'cc/private/rules_impl/native.bzl' failed ``` --- This section provides a methodical description of the errors encountered and resolved during each step of the `rules_java` update. The proper `WORKSPACE` setup stanza for `rules_java`, which we didn't originally have in place, is: ```py load( "@rules_java//java:repositories.bzl", "rules_java_dependencies", "rules_java_toolchains", ) rules_java_dependencies() rules_java_toolchains() ``` Adding this stanza to `WORKSPACE` while still using `rules_java` 7.9.0 didn't break the Bazel 7.4.1 build, but it broke the Bazel 6.5.0 build. Note the `CompiledWithJava{8,11}_java.jar` references: ```txt $ bazel build //{src,test,third_party,scala_proto}/... [ ...snip... ] ERROR: .../test/src/main/resources/java_sources/BUILD:5:14: Building test/src/main/resources/java_sources/CompiledWithJava8_java.jar (1 source file) failed: Worker process did not return a WorkResponse: ---8<---8<--- Start of log, file at .../bazel-workers/multiplex-worker-18-Javac.log ---8<---8<--- Error thrown by worker thread, shutting down worker. java.lang.NoSuchMethodError: 'java.lang.Iterable com.google.devtools.build.buildjar.javac.BlazeJavacMain$ClassloaderMaskingFileManager.getJavaFileObjectsFromPaths(java.util.Collection)' at com.google.devtools.build.buildjar.javac.BlazeJavacMain.compile(BlazeJavacMain.java:142) [ ...snip... ] ---8<---8<--- End of log ---8<---8<--- ERROR: .../test/src/main/resources/java_sources/BUILD:11:14: Building test/src/main/resources/java_sources/CompiledWithJava11_java.jar (1 source file) failed: Worker process did not return a WorkResponse: ---8<---8<--- Start of log, file at .../bazel-workers/multiplex-worker-18-Javac.log ---8<---8<--- Error thrown by worker thread, shutting down worker. java.lang.NoSuchMethodError: 'java.lang.Iterable com.google.devtools.build.buildjar.javac.BlazeJavacMain$ClassloaderMaskingFileManager.getJavaFileObjectsFromPaths(java.util.Collection)' at com.google.devtools.build.buildjar.javac.BlazeJavacMain.compile(BlazeJavacMain.java:142) [ ...snip... ] ---8<---8<--- End of log ---8<---8<--- ``` Another variation of this I also saw was: ```txt $ bazel build //{src,jmh,test,third_party,scala_proto}/... [ ...snip... ] ERROR: test/src/main/resources/java_sources/BUILD:11:14: Building test/src/main/resources/java_sources/CompiledWithJava11_java.jar (1 source file) failed: Worker process did not return a WorkResponse: ---8<---8<--- Start of log, file at .../bazel-workers/multiplex-worker-33-Javac.log ---8<---8<--- Error thrown by worker thread, shutting down worker. java.lang.UnsupportedClassVersionError: com/google/errorprone/ErrorProneError has been compiled by a more recent version of the Java Runtime (class file version 61.0), this version of the Java Runtime only recognizes class file versions up to 55.0 at java.base/java.lang.ClassLoader.defineClass1(Native Method) [ ...snip... ] ERROR: test/src/main/resources/java_sources/BUILD:5:14: Building test/src/main/resources/java_sources/CompiledWithJava8_java.jar (1 source file) failed: Worker process did not return a WorkResponse: ---8<---8<--- Start of log, file at .../bazel-workers/multiplex-worker-33-Javac.log ---8<---8<--- Error thrown by worker thread, shutting down worker. java.lang.UnsupportedClassVersionError: com/google/errorprone/ErrorProneError has been compiled by a more recent version of the Java Runtime (class file version 61.0), this version of the Java Runtime only recognizes class file versions up to 55.0 at java.base/java.lang.ClassLoader.defineClass1(Native Method) [ ...snip... ] ``` Updating the toolchains from `@bazel_tools//tools/jdk:` to `@rules_java//toolchains:` for targets in `test/BUILD` and `test/src/main/resources/java_sources/BUILD` fixed this particular issue. (More on updating other `@bazel_tools//tools/jdk` toolchain targets below.) Bazel 6.5.0 then failed with the following expected issue, fixed by `rules_java` 7.10.0 via: - bazelbuild/rules_java#210 - bazelbuild/rules_java@30ecf3f ```txt $ bazel build //{src,test,third_party,scala_proto}/... ERROR: .../external/rules_java/toolchains/java_toolchain_alias.bzl:83:34: Use of Starlark transition without allowlist attribute '_allowlist_function_transition'. See Starlark transitions documentation for details and usage: @rules_java//toolchains:java_toolchain_alias.bzl NORMAL ERROR: .../src/java/io/bazel/rulesscala/coverage/instrumenter/BUILD:3:12: While resolving toolchains for target //src/java/io/bazel/rulesscala/coverage/instrumenter:instrumenter: invalid registered toolchain '//toolchains:all': while parsing '//toolchains:all': error loading package '@rules_java//toolchains': initialization of module 'toolchains/java_toolchain_alias.bzl' failed ERROR: Analysis of target '//src/java/io/bazel/rulesscala/coverage/instrumenter:instrumenter' failed; build aborted: ``` Updating to `rules_java` 7.10.0 fixed the Bazel 6.5.0 build, but caused Bazel 7.4.1 to fail with the error I originally described in bazelbuild#1619. For details, see "Bump to rules_java 7.9.0 for Bazel 7 compatibility" in the message for commit cd22d88. ```txt $ bazel build //{src,test,third_party,scala_proto}/... ERROR: .../external/rules_java_builtin/toolchains/BUILD:254:14: While resolving toolchains for target @@rules_java_builtin//toolchains:platformclasspath (096dcc8): No matching toolchains found for types @@bazel_tools//tools/jdk:bootstrap_runtime_toolchain_type. To debug, rerun with --toolchain_resolution_debug='@@bazel_tools//tools/jdk:bootstrap_runtime_toolchain_type' If platforms or toolchains are a new concept for you, we'd encourage reading https://bazel.build/concepts/platforms-intro. ERROR: Analysis of target '//test/toolchains:java21_toolchain' failed; build aborted: Analysis failed ``` My analysis from bazelbuild#1619 _was_ on the right track. The `rules_java` 7.9.0 built into Bazel's `WORKSPACE` preamble registered `@bazel_tools//tools/jdk:bootstrap_runtime_toolchain_type` toolchains that later versions of `rules_java` couldn't resolve. This was due to the very same change that fixed 6.5.0, bazelbuild/rules_java#210. However, I mistakenly thought we needed to keep `@bazel_tools//tools/jdk:` as the canonical Java toolchains package instead switching to `@rules_java//toolchains:`. That's why I originally thought we were stuck on `rules_java` 7.9.0. I eventually rediscovered the following issue, which surfaced this problem a month before bazelbuild#1619. The conversation helped me realize that switching to `@rules_java//toolchains:` actually is the preferred, futureproof approach (short of migrating to Bzlmod): - bazelbuild/rules_java: Regression with `@@rules_java//toolchains:bootstrap_runtime_toolchain_type` in 7.10.0 bazelbuild/rules_java#214 Switching all `@bazel_tools//tools/jdk:` toolchains to `@rules_java//toolchains:` resolved this issue, enabling Bazel 6.5.0 and 7.4.1 to successfully build using `rules_java` 7.12.2. I then updated `rules_java` to 7.12.3, which returns to registering some toolchains as `@bazel_tools//tools/jdk:bootstrap_runtime_toolchain_type`: - bazelbuild/rules_java#246 - bazelbuild/rules_java@b64eb7d @hvadehra requested that I try 7.12.3 to see if it resolved the issue. I was able to build successfully using this version in a branch without the toolchain updates from this commit. - bazelbuild#1652 (comment) However, the changes from this commit should remain futureproof when `@bazel_tools//tools/jdk:bootstrap_runtime_toolchain_type` moves to `@rules_java//toolchains:bootstrap_runtime_toolchain_type` again one day. --- Several of the flags removed from the `scrooge_compile_with_jdk_11` test case have been no-ops for years, so removing them eliminated their corresponding deprecation warnings. `--javacopt='--release 11'`, however, caused this breakage after originally bumping to `rules_java` 7.12.2: ```txt $ RULES_SCALA_TEST_ONLY="scrooge_compile_with_jdk_11" bash ./test/shell/test_twitter_scrooge.sh running test scrooge_compile_with_jdk_11 WARNING: Option 'javabase' is deprecated WARNING: Option 'host_javabase' is deprecated WARNING: Option 'host_java_toolchain' is deprecated WARNING: Option 'java_toolchain' is deprecated WARNING: Option 'javabase' is deprecated WARNING: Option 'host_javabase' is deprecated WARNING: Option 'host_java_toolchain' is deprecated WARNING: Option 'java_toolchain' is deprecated INFO: Analyzed 64 targets (0 packages loaded, 0 targets configured). INFO: Found 64 targets... ERROR: .../src/java/io/bazel/rulesscala/scalac/compileoptions/BUILD:3:13: Compiling Java headers src/java/io/bazel/rulesscala/scalac/compileoptions/libcompileoptions-hjar.jar (1 source file) failed: (Exit 1): turbine_direct_graal failed: error executing command (from target //src/java/io/bazel/rulesscala/scalac/compileoptions:compileoptions) external/remote_java_tools_darwin_arm64/java_tools/turbine_direct_graal --output bazel-out/darwin_arm64-fastbuild/bin/src/java/io/bazel/rulesscala/scalac/compileoptions/libcompileoptions-hjar.jar ... (remaining 32 arguments skipped) Use --sandbox_debug to see verbose messages from the sandbox and retain the sandbox build root for debugging java.lang.NullPointerException: attempted to use --release, but JAVA_HOME is not set at [email protected]/java.util.Objects.requireNonNull(Objects.java:259) at com.google.turbine.binder.CtSymClassBinder.bind(CtSymClassBinder.java:55) at com.google.turbine.main.Main.release(Main.java:318) at com.google.turbine.main.Main.bootclasspath(Main.java:304) at com.google.turbine.main.Main.compile(Main.java:142) at com.google.turbine.main.Main.compile(Main.java:133) at com.google.turbine.main.Main.main(Main.java:89) at [email protected]/java.lang.invoke.LambdaForm$DMH/sa346b79c.invokeStaticInit(LambdaForm$DMH) INFO: Elapsed time: 0.325s, Critical Path: 0.12s INFO: 11 processes: 10 internal, 1 worker. FAILED: Build did NOT complete successfully Test "scrooge_compile_with_jdk_11" failed (0 sec) ``` See also: - Bazel Blog: Bazel 5.0: https://blog.bazel.build/2022/01/19/bazel-5.0.html#java - bazelbuild/bazel: `incompatible_use_toolchain_resolution_for_java_rules`: use toolchain resolution for Java rules #7849 bazelbuild/bazel#7849 --- Bazel 8 requires `rules_java` 8, per bazelbuild#1652. Updating to `rules_java` 8 will likely happen in a new major release _after_ a release containing the Bzlmod changes. This is because: - The Bzlmod changes alone will work with Bazel 6 without requiring that users update `protobuf` beyond version 21.7. - Bazel 8 requires `protobuf` >= 29.0. Bazel 6 users would have to add the afforementioned C++ compiler flags to support the newer `abseil-cpp` versions required by newer `protobuf` versions. - `rules_scala` itself needs to update ScalaPB to 1.0.0-alpha.1 or higher to support `protobuf` >= v28.0, but this ScalaPB version will _not_ support `protobuf` < v28.0. The idea is that the next major release of `rules_scala` will help users migrate to Bazel 7 and Bzlmod, in either order. Then the next major release after that will enable them to migrate to Bazel 8, with all the requisite dependency upgrades. (Technically it will still support `WORKSPACE`, but hopefully they'll make the jump to Bzlmod first.) See: - bazelbuild#1482 (comment) --- Here are the details explaining how `rules_java` 8 currently breaks `rules_scala`, up until the point that upgrading to `protobuf` >= v29.0 would fix it. `rules_java` 8.0.0 requires Bazel >= 7.3.2, which provides the `subrule` API. Compatibility with Bazel >= 6.3.0 isn't restored until `rules_java` 8.3.2. ```txt $ bazel build //{src,test,third_party,scala_proto}/... ERROR: .../external/rules_java/java/common/rules/android_lint.bzl:142:24: name 'subrule' is not defined (did you mean 'rule'?) ERROR: Error computing the main repository mapping: at .../scala/private/extensions/dev_deps.bzl:8:6: at .../external/rules_java/java/repositories.bzl:20:6: at .../external/rules_java/toolchains/local_java_repository.bzl:17:6: at .../external/rules_java/java/defs.bzl:18:6: at .../external/rules_java/java/java_library.bzl:16:6: at .../external/rules_java/java/bazel/rules/bazel_java_library.bzl:21:6: compilation of module 'java/common/rules/android_lint.bzl' failed ``` `rules_java` 8.3.0 is broken, as it can't find its own `@compatibility_proxy` repo: ```txt $ bazel build //{src,test,third_party,scala_proto}/... ERROR: error loading package under directory 'src': error loading package 'src/protobuf/io/bazel/rules_scala': at .../external/rules_java/java/defs.bzl:22:6: at .../external/rules_java/java/java_test.bzl:16:6: Unable to find package for @compatibility_proxy//:proxy.bzl: The repository '@compatibility_proxy' could not be resolved: Repository '@compatibility_proxy' is not defined. ``` `rules_java` 8.3.1 seems to fix this, presumably by importing the `protobuf` repo as `com_google_protobuf`. However, it now requires at least `protobuf` v27.0, which adds `bazel/java/lite_proto_library.bzl`. Per bazelbuild#1647, we'd have to update to ScalaPB 1.0.0-alpha.1 to support `protobuf` v28, abandoning users of previous `protobuf` versions or forcing them to upgrade. ```txt $ bazel build //{src,test,third_party,scala_proto}/... [...snip...] ERROR: error loading package under directory 'src': error loading package 'src/java/io/bazel/rulesscala/worker': at .../external/rules_java/java/defs.bzl:16:6: Label '@com_google_protobuf//bazel:java_lite_proto_library.bzl' is invalid because 'bazel' is not a package; perhaps you meant to put the colon here: '@com_google_protobuf//:bazel/java_lite_proto_library.bzl'? ```
mbland
added a commit
to mbland/rules_scala
that referenced
this issue
Dec 10, 2024
Moves `rules_scala_dependencies` to `scala/deps.bzl` and bumps several dependencies as high as they can go and still be compatible with Bazel 6.5.0 and 7.4.1. - `bazel_skylib`: 1.4.1 => 1.7.1 - `go`: 1.19.5 => 1.23.4 - `rules_cc`: 0.0.6 => 0.0.9 - `rules_go`: 0.39.1 => 0.50.1 - `rules_java`: 7.9.0 => 7.12.3 - `rules_python`: 0.36.0 => 0.38.0 The `rules_java` 7.12.13 bump precipitated the following changes: - Adds the `WORKSPACE` stanza for `rules_java` in every `WORKSPACE` file per https://github.com/bazelbuild/rules_java/releases/tag/7.12.3. This replaces previous calls to instantiate `rules_java` repos and to register `rules_java` toolchains. - Rearranges the other `WORKSPACE` setup macros for dependencies to come before the `rules_scala` setup macros. - Updates almost all dependencies on `@bazel_tools//tools/jdk:` toolchain targets and `.bzl` files to corresponding targets and files in `@rules_java//toolchains:`. - Removes several deprecated flags from the `scrooge_compile_with_jdk_11` test case from `test/shell/test_twitter_scrooge.sh`, and one obsolete flag that caused it to break. --- Moving `rules_scala_dependencies` to `scala/deps.bzl` ensures we get all the versions of dependencies we want in `WORKSPACE`, while providing a new API to consumers. It also prevents `WORKSPACE` from transitively loading any `.bzl` files that load `@rules_java`, ensuring Bazel 8 compatibility per bazelbuild#1652. Reasons for the other `rules_java` related changes include: - The `WORKSPACE` stanza for `rules_java` should've already been present while using the existing version 7.9.0. However, doing so would've broken Bazel 6 builds, as described in detail below. - Having the other `WORKSPACE` setup macros for dependencies come before the `rules_scala` setup macros helps ensure consistent, correct initialization before `rules_scala` initialization. - Updating the toolchain specifiers to use `@rules_java//toolchains` fixed `WORKSPACE` build breakages when updating to `rules_java` 7.10.0 and later. This is a potentially breaking change for consumers, but in the good kind of way, in that it requires an easy, futureproof update. (`@bazel_tools//tools/jdk:toolchain_type` dependencies remain, as there's not yet a corresponding `@rules_java//toolchains` target.) What follows are detailed notes on why some dependency versions are capped where they are, and on some breakages fixed by these changes. --- `abseil-cpp` and `protobuf` have to stay at 20220623.1 and v21.7, respectively, for Bazel 6 compatibility per bazelbuild#1647. `protobuf` up to v25.5 is compatible with Bazel 6 provided users set the compiler flags mentioned in that issue: ```txt build:linux --cxxopt=-std=c++17 build:linux --host_cxxopt=-std=c++17 build:macos --cxxopt=-std=c++17 build:macos --host_cxxopt=-std=c++17 build:windows --cxxopt=/std=c++17 build:windows --host_cxxopt=/std=c++17 ``` --- `rules_python` 0.38.0 => 0.39.0 requires at least `rules_cc` 0.0.10, which introduced `cc/common/cc_info.bzl`: ```txt $ bazel build //{src,test,third_party,scala_proto}/... [ ...snip... ] ERROR: error loading package under directory 'test': error loading package 'test': at .../external/rules_python/python/defs.bzl:17:6: at .../external/rules_python/python/py_binary.bzl:18:6: at .../external/rules_python/python/private/py_binary_macro.bzl:16:6: at .../external/rules_python/python/private/common_bazel.bzl:19:6: Label '@rules_cc//cc/common:cc_info.bzl' is invalid because 'cc/common' is not a package; perhaps you meant to put the colon here: '@rules_cc//cc:common/cc_info.bzl'? ``` However, `rules_cc` 0.0.9 => 0.0.10 requires Bazel 7, which defines `CcSharedLibraryHintInfo`: ```txt $ bazel build //{src,test,third_party,scala_proto}/... ERROR: .../external/rules_cc/cc/private/rules_impl/native.bzl:40:33: name 'CcSharedLibraryHintInfo' is not defined (did you mean 'CcSharedLibraryInfo'?) [ ...snip... ] ERROR: error loading package under directory 'test': error loading package 'test': at .../external/rules_python/python/defs.bzl:17:6: at .../external/rules_python/python/py_binary.bzl:18:6: at .../external/rules_python/python/private/py_binary_macro.bzl:16:6: at .../external/rules_python/python/private/common_bazel.bzl:18:6: at .../external/rules_cc/cc/common/cc_common.bzl:17:6: compilation of module 'cc/private/rules_impl/native.bzl' failed ``` --- This section provides a methodical description of the errors encountered and resolved during each step of the `rules_java` update. The proper `WORKSPACE` setup stanza for `rules_java`, which we didn't originally have in place, is: ```py load( "@rules_java//java:repositories.bzl", "rules_java_dependencies", "rules_java_toolchains", ) rules_java_dependencies() rules_java_toolchains() ``` Adding this stanza to `WORKSPACE` while still using `rules_java` 7.9.0 didn't break the Bazel 7.4.1 build, but it broke the Bazel 6.5.0 build. Note the `CompiledWithJava{8,11}_java.jar` references: ```txt $ bazel build //{src,test,third_party,scala_proto}/... [ ...snip... ] ERROR: .../test/src/main/resources/java_sources/BUILD:5:14: Building test/src/main/resources/java_sources/CompiledWithJava8_java.jar (1 source file) failed: Worker process did not return a WorkResponse: ---8<---8<--- Start of log, file at .../bazel-workers/multiplex-worker-18-Javac.log ---8<---8<--- Error thrown by worker thread, shutting down worker. java.lang.NoSuchMethodError: 'java.lang.Iterable com.google.devtools.build.buildjar.javac.BlazeJavacMain$ClassloaderMaskingFileManager.getJavaFileObjectsFromPaths(java.util.Collection)' at com.google.devtools.build.buildjar.javac.BlazeJavacMain.compile(BlazeJavacMain.java:142) [ ...snip... ] ---8<---8<--- End of log ---8<---8<--- ERROR: .../test/src/main/resources/java_sources/BUILD:11:14: Building test/src/main/resources/java_sources/CompiledWithJava11_java.jar (1 source file) failed: Worker process did not return a WorkResponse: ---8<---8<--- Start of log, file at .../bazel-workers/multiplex-worker-18-Javac.log ---8<---8<--- Error thrown by worker thread, shutting down worker. java.lang.NoSuchMethodError: 'java.lang.Iterable com.google.devtools.build.buildjar.javac.BlazeJavacMain$ClassloaderMaskingFileManager.getJavaFileObjectsFromPaths(java.util.Collection)' at com.google.devtools.build.buildjar.javac.BlazeJavacMain.compile(BlazeJavacMain.java:142) [ ...snip... ] ---8<---8<--- End of log ---8<---8<--- ``` Another variation of this I also saw was: ```txt $ bazel build //{src,jmh,test,third_party,scala_proto}/... [ ...snip... ] ERROR: test/src/main/resources/java_sources/BUILD:11:14: Building test/src/main/resources/java_sources/CompiledWithJava11_java.jar (1 source file) failed: Worker process did not return a WorkResponse: ---8<---8<--- Start of log, file at .../bazel-workers/multiplex-worker-33-Javac.log ---8<---8<--- Error thrown by worker thread, shutting down worker. java.lang.UnsupportedClassVersionError: com/google/errorprone/ErrorProneError has been compiled by a more recent version of the Java Runtime (class file version 61.0), this version of the Java Runtime only recognizes class file versions up to 55.0 at java.base/java.lang.ClassLoader.defineClass1(Native Method) [ ...snip... ] ERROR: test/src/main/resources/java_sources/BUILD:5:14: Building test/src/main/resources/java_sources/CompiledWithJava8_java.jar (1 source file) failed: Worker process did not return a WorkResponse: ---8<---8<--- Start of log, file at .../bazel-workers/multiplex-worker-33-Javac.log ---8<---8<--- Error thrown by worker thread, shutting down worker. java.lang.UnsupportedClassVersionError: com/google/errorprone/ErrorProneError has been compiled by a more recent version of the Java Runtime (class file version 61.0), this version of the Java Runtime only recognizes class file versions up to 55.0 at java.base/java.lang.ClassLoader.defineClass1(Native Method) [ ...snip... ] ``` Updating the toolchains from `@bazel_tools//tools/jdk:` to `@rules_java//toolchains:` for targets in `test/BUILD` and `test/src/main/resources/java_sources/BUILD` fixed this particular issue. (More on updating other `@bazel_tools//tools/jdk` toolchain targets below.) Bazel 6.5.0 then failed with the following expected issue, fixed by `rules_java` 7.10.0 via: - bazelbuild/rules_java#210 - bazelbuild/rules_java@30ecf3f ```txt $ bazel build //{src,test,third_party,scala_proto}/... ERROR: .../external/rules_java/toolchains/java_toolchain_alias.bzl:83:34: Use of Starlark transition without allowlist attribute '_allowlist_function_transition'. See Starlark transitions documentation for details and usage: @rules_java//toolchains:java_toolchain_alias.bzl NORMAL ERROR: .../src/java/io/bazel/rulesscala/coverage/instrumenter/BUILD:3:12: While resolving toolchains for target //src/java/io/bazel/rulesscala/coverage/instrumenter:instrumenter: invalid registered toolchain '//toolchains:all': while parsing '//toolchains:all': error loading package '@rules_java//toolchains': initialization of module 'toolchains/java_toolchain_alias.bzl' failed ERROR: Analysis of target '//src/java/io/bazel/rulesscala/coverage/instrumenter:instrumenter' failed; build aborted: ``` Updating to `rules_java` 7.10.0 fixed the Bazel 6.5.0 build, but caused Bazel 7.4.1 to fail with the error I originally described in bazelbuild#1619. For details, see "Bump to rules_java 7.9.0 for Bazel 7 compatibility" in the message for commit cd22d88. ```txt $ bazel build //{src,test,third_party,scala_proto}/... ERROR: .../external/rules_java_builtin/toolchains/BUILD:254:14: While resolving toolchains for target @@rules_java_builtin//toolchains:platformclasspath (096dcc8): No matching toolchains found for types @@bazel_tools//tools/jdk:bootstrap_runtime_toolchain_type. To debug, rerun with --toolchain_resolution_debug='@@bazel_tools//tools/jdk:bootstrap_runtime_toolchain_type' If platforms or toolchains are a new concept for you, we'd encourage reading https://bazel.build/concepts/platforms-intro. ERROR: Analysis of target '//test/toolchains:java21_toolchain' failed; build aborted: Analysis failed ``` My analysis from bazelbuild#1619 _was_ on the right track. The `rules_java` 7.9.0 built into Bazel's `WORKSPACE` preamble registered `@bazel_tools//tools/jdk:bootstrap_runtime_toolchain_type` toolchains that later versions of `rules_java` couldn't resolve. This was due to the very same change that fixed 6.5.0, bazelbuild/rules_java#210. However, I mistakenly thought we needed to keep `@bazel_tools//tools/jdk:` as the canonical Java toolchains package instead switching to `@rules_java//toolchains:`. That's why I originally thought we were stuck on `rules_java` 7.9.0. I eventually rediscovered the following issue, which surfaced this problem a month before bazelbuild#1619. The conversation helped me realize that switching to `@rules_java//toolchains:` actually is the preferred, futureproof approach (short of migrating to Bzlmod): - bazelbuild/rules_java: Regression with `@@rules_java//toolchains:bootstrap_runtime_toolchain_type` in 7.10.0 bazelbuild/rules_java#214 Switching all `@bazel_tools//tools/jdk:` toolchains to `@rules_java//toolchains:` resolved this issue, enabling Bazel 6.5.0 and 7.4.1 to successfully build using `rules_java` 7.12.2. I then updated `rules_java` to 7.12.3, which returns to registering some toolchains as `@bazel_tools//tools/jdk:bootstrap_runtime_toolchain_type`: - bazelbuild/rules_java#246 - bazelbuild/rules_java@b64eb7d @hvadehra requested that I try 7.12.3 to see if it resolved the issue. I was able to build successfully using this version in a branch without the toolchain updates from this commit. - bazelbuild#1652 (comment) However, the changes from this commit should remain futureproof when `@bazel_tools//tools/jdk:bootstrap_runtime_toolchain_type` moves to `@rules_java//toolchains:bootstrap_runtime_toolchain_type` again one day. --- Several of the flags removed from the `scrooge_compile_with_jdk_11` test case have been no-ops for years, so removing them eliminated their corresponding deprecation warnings. `--javacopt='--release 11'`, however, caused this breakage after originally bumping to `rules_java` 7.12.2: ```txt $ RULES_SCALA_TEST_ONLY="scrooge_compile_with_jdk_11" bash ./test/shell/test_twitter_scrooge.sh running test scrooge_compile_with_jdk_11 WARNING: Option 'javabase' is deprecated WARNING: Option 'host_javabase' is deprecated WARNING: Option 'host_java_toolchain' is deprecated WARNING: Option 'java_toolchain' is deprecated WARNING: Option 'javabase' is deprecated WARNING: Option 'host_javabase' is deprecated WARNING: Option 'host_java_toolchain' is deprecated WARNING: Option 'java_toolchain' is deprecated INFO: Analyzed 64 targets (0 packages loaded, 0 targets configured). INFO: Found 64 targets... ERROR: .../src/java/io/bazel/rulesscala/scalac/compileoptions/BUILD:3:13: Compiling Java headers src/java/io/bazel/rulesscala/scalac/compileoptions/libcompileoptions-hjar.jar (1 source file) failed: (Exit 1): turbine_direct_graal failed: error executing command (from target //src/java/io/bazel/rulesscala/scalac/compileoptions:compileoptions) external/remote_java_tools_darwin_arm64/java_tools/turbine_direct_graal --output bazel-out/darwin_arm64-fastbuild/bin/src/java/io/bazel/rulesscala/scalac/compileoptions/libcompileoptions-hjar.jar ... (remaining 32 arguments skipped) Use --sandbox_debug to see verbose messages from the sandbox and retain the sandbox build root for debugging java.lang.NullPointerException: attempted to use --release, but JAVA_HOME is not set at [email protected]/java.util.Objects.requireNonNull(Objects.java:259) at com.google.turbine.binder.CtSymClassBinder.bind(CtSymClassBinder.java:55) at com.google.turbine.main.Main.release(Main.java:318) at com.google.turbine.main.Main.bootclasspath(Main.java:304) at com.google.turbine.main.Main.compile(Main.java:142) at com.google.turbine.main.Main.compile(Main.java:133) at com.google.turbine.main.Main.main(Main.java:89) at [email protected]/java.lang.invoke.LambdaForm$DMH/sa346b79c.invokeStaticInit(LambdaForm$DMH) INFO: Elapsed time: 0.325s, Critical Path: 0.12s INFO: 11 processes: 10 internal, 1 worker. FAILED: Build did NOT complete successfully Test "scrooge_compile_with_jdk_11" failed (0 sec) ``` See also: - Bazel Blog: Bazel 5.0: https://blog.bazel.build/2022/01/19/bazel-5.0.html#java - bazelbuild/bazel: `incompatible_use_toolchain_resolution_for_java_rules`: use toolchain resolution for Java rules #7849 bazelbuild/bazel#7849 --- Bazel 8 requires `rules_java` 8, per bazelbuild#1652. Updating to `rules_java` 8 will likely happen in a new major release _after_ a release containing the Bzlmod changes. This is because: - The Bzlmod changes alone will work with Bazel 6 without requiring that users update `protobuf` beyond version 21.7. - Bazel 8 requires `protobuf` >= 29.0. Bazel 6 users would have to add the afforementioned C++ compiler flags to support the newer `abseil-cpp` versions required by newer `protobuf` versions. - `rules_scala` itself needs to update ScalaPB to 1.0.0-alpha.1 or higher to support `protobuf` >= v28.0, but this ScalaPB version will _not_ support `protobuf` < v28.0. The idea is that the next major release of `rules_scala` will help users migrate to Bazel 7 and Bzlmod, in either order. Then the next major release after that will enable them to migrate to Bazel 8, with all the requisite dependency upgrades. (Technically it will still support `WORKSPACE`, but hopefully they'll make the jump to Bzlmod first.) See: - bazelbuild#1482 (comment) --- Here are the details explaining how `rules_java` 8 currently breaks `rules_scala`, up until the point that upgrading to `protobuf` >= v29.0 would fix it. `rules_java` 8.0.0 requires Bazel >= 7.3.2, which provides the `subrule` API. Compatibility with Bazel >= 6.3.0 isn't restored until `rules_java` 8.3.2. ```txt $ bazel build //{src,test,third_party,scala_proto}/... ERROR: .../external/rules_java/java/common/rules/android_lint.bzl:142:24: name 'subrule' is not defined (did you mean 'rule'?) ERROR: Error computing the main repository mapping: at .../scala/private/extensions/dev_deps.bzl:8:6: at .../external/rules_java/java/repositories.bzl:20:6: at .../external/rules_java/toolchains/local_java_repository.bzl:17:6: at .../external/rules_java/java/defs.bzl:18:6: at .../external/rules_java/java/java_library.bzl:16:6: at .../external/rules_java/java/bazel/rules/bazel_java_library.bzl:21:6: compilation of module 'java/common/rules/android_lint.bzl' failed ``` `rules_java` 8.3.0 is broken, as it can't find its own `@compatibility_proxy` repo: ```txt $ bazel build //{src,test,third_party,scala_proto}/... ERROR: error loading package under directory 'src': error loading package 'src/protobuf/io/bazel/rules_scala': at .../external/rules_java/java/defs.bzl:22:6: at .../external/rules_java/java/java_test.bzl:16:6: Unable to find package for @compatibility_proxy//:proxy.bzl: The repository '@compatibility_proxy' could not be resolved: Repository '@compatibility_proxy' is not defined. ``` `rules_java` 8.3.1 seems to fix this, presumably by importing the `protobuf` repo as `com_google_protobuf`. However, it now requires at least `protobuf` v27.0, which adds `bazel/java/lite_proto_library.bzl`. Per bazelbuild#1647, we'd have to update to ScalaPB 1.0.0-alpha.1 to support `protobuf` v28, abandoning users of previous `protobuf` versions or forcing them to upgrade. ```txt $ bazel build //{src,test,third_party,scala_proto}/... [...snip...] ERROR: error loading package under directory 'src': error loading package 'src/java/io/bazel/rulesscala/worker': at .../external/rules_java/java/defs.bzl:16:6: Label '@com_google_protobuf//bazel:java_lite_proto_library.bzl' is invalid because 'bazel' is not a package; perhaps you meant to put the colon here: '@com_google_protobuf//:bazel/java_lite_proto_library.bzl'? ```
mbland
added a commit
to mbland/rules_scala
that referenced
this issue
Dec 10, 2024
Moves `rules_scala_dependencies` to `scala/deps.bzl` and bumps several dependencies as high as they can go and still be compatible with Bazel 6.5.0 and 7.4.1. - `bazel_skylib`: 1.4.1 => 1.7.1 - `go`: 1.19.5 => 1.23.4 - `rules_cc`: 0.0.6 => 0.0.9 - `rules_go`: 0.39.1 => 0.50.1 - `rules_java`: 7.9.0 => 7.12.3 - `rules_python`: 0.36.0 => 0.38.0 The `rules_java` 7.12.13 bump precipitated the following changes: - Adds the `WORKSPACE` stanza for `rules_java` in every `WORKSPACE` file per https://github.com/bazelbuild/rules_java/releases/tag/7.12.3. This replaces previous calls to instantiate `rules_java` repos and to register `rules_java` toolchains. - Rearranges the other `WORKSPACE` setup macros for dependencies to come before the `rules_scala` setup macros. - Updates almost all dependencies on `@bazel_tools//tools/jdk:` toolchain targets and `.bzl` files to corresponding targets and files in `@rules_java//toolchains:`. - Removes several deprecated flags from the `scrooge_compile_with_jdk_11` test case from `test/shell/test_twitter_scrooge.sh`, and one obsolete flag that caused it to break. --- Moving `rules_scala_dependencies` to `scala/deps.bzl` ensures we get all the versions of dependencies we want in `WORKSPACE`, while providing a new API to consumers. It also prevents `WORKSPACE` from transitively loading any `.bzl` files that load `@rules_java`, ensuring Bazel 8 compatibility per bazelbuild#1652. Reasons for the other `rules_java` related changes include: - The `WORKSPACE` stanza for `rules_java` should've already been present while using the existing version 7.9.0. However, doing so would've broken Bazel 6 builds, as described in detail below. - Having the other `WORKSPACE` setup macros for dependencies come before the `rules_scala` setup macros helps ensure consistent, correct initialization before `rules_scala` initialization. - Updating the toolchain specifiers to use `@rules_java//toolchains` fixed `WORKSPACE` build breakages when updating to `rules_java` 7.10.0 and later. This is a potentially breaking change for consumers, but in the good kind of way, in that it requires an easy, futureproof update. (`@bazel_tools//tools/jdk:toolchain_type` dependencies remain, as there's not yet a corresponding `@rules_java//toolchains` target.) What follows are detailed notes on why some dependency versions are capped where they are, and on some breakages fixed by these changes. --- `abseil-cpp` and `protobuf` have to stay at 20220623.1 and v21.7, respectively, for Bazel 6 compatibility per bazelbuild#1647. `protobuf` up to v25.5 is compatible with Bazel 6 provided users set the compiler flags mentioned in that issue: ```txt build:linux --cxxopt=-std=c++17 build:linux --host_cxxopt=-std=c++17 build:macos --cxxopt=-std=c++17 build:macos --host_cxxopt=-std=c++17 build:windows --cxxopt=/std=c++17 build:windows --host_cxxopt=/std=c++17 ``` --- `rules_python` 0.38.0 => 0.39.0 requires at least `rules_cc` 0.0.10, which introduced `cc/common/cc_info.bzl`: ```txt $ bazel build //{src,test,third_party,scala_proto}/... [ ...snip... ] ERROR: error loading package under directory 'test': error loading package 'test': at .../external/rules_python/python/defs.bzl:17:6: at .../external/rules_python/python/py_binary.bzl:18:6: at .../external/rules_python/python/private/py_binary_macro.bzl:16:6: at .../external/rules_python/python/private/common_bazel.bzl:19:6: Label '@rules_cc//cc/common:cc_info.bzl' is invalid because 'cc/common' is not a package; perhaps you meant to put the colon here: '@rules_cc//cc:common/cc_info.bzl'? ``` However, `rules_cc` 0.0.9 => 0.0.10 requires Bazel 7, which defines `CcSharedLibraryHintInfo`: ```txt $ bazel build //{src,test,third_party,scala_proto}/... ERROR: .../external/rules_cc/cc/private/rules_impl/native.bzl:40:33: name 'CcSharedLibraryHintInfo' is not defined (did you mean 'CcSharedLibraryInfo'?) [ ...snip... ] ERROR: error loading package under directory 'test': error loading package 'test': at .../external/rules_python/python/defs.bzl:17:6: at .../external/rules_python/python/py_binary.bzl:18:6: at .../external/rules_python/python/private/py_binary_macro.bzl:16:6: at .../external/rules_python/python/private/common_bazel.bzl:18:6: at .../external/rules_cc/cc/common/cc_common.bzl:17:6: compilation of module 'cc/private/rules_impl/native.bzl' failed ``` --- This section provides a methodical description of the errors encountered and resolved during each step of the `rules_java` update. The proper `WORKSPACE` setup stanza for `rules_java`, which we didn't originally have in place, is: ```py load( "@rules_java//java:repositories.bzl", "rules_java_dependencies", "rules_java_toolchains", ) rules_java_dependencies() rules_java_toolchains() ``` Adding this stanza to `WORKSPACE` while still using `rules_java` 7.9.0 didn't break the Bazel 7.4.1 build, but it broke the Bazel 6.5.0 build. Note the `CompiledWithJava{8,11}_java.jar` references: ```txt $ bazel build //{src,test,third_party,scala_proto}/... [ ...snip... ] ERROR: .../test/src/main/resources/java_sources/BUILD:5:14: Building test/src/main/resources/java_sources/CompiledWithJava8_java.jar (1 source file) failed: Worker process did not return a WorkResponse: ---8<---8<--- Start of log, file at .../bazel-workers/multiplex-worker-18-Javac.log ---8<---8<--- Error thrown by worker thread, shutting down worker. java.lang.NoSuchMethodError: 'java.lang.Iterable com.google.devtools.build.buildjar.javac.BlazeJavacMain$ClassloaderMaskingFileManager.getJavaFileObjectsFromPaths(java.util.Collection)' at com.google.devtools.build.buildjar.javac.BlazeJavacMain.compile(BlazeJavacMain.java:142) [ ...snip... ] ---8<---8<--- End of log ---8<---8<--- ERROR: .../test/src/main/resources/java_sources/BUILD:11:14: Building test/src/main/resources/java_sources/CompiledWithJava11_java.jar (1 source file) failed: Worker process did not return a WorkResponse: ---8<---8<--- Start of log, file at .../bazel-workers/multiplex-worker-18-Javac.log ---8<---8<--- Error thrown by worker thread, shutting down worker. java.lang.NoSuchMethodError: 'java.lang.Iterable com.google.devtools.build.buildjar.javac.BlazeJavacMain$ClassloaderMaskingFileManager.getJavaFileObjectsFromPaths(java.util.Collection)' at com.google.devtools.build.buildjar.javac.BlazeJavacMain.compile(BlazeJavacMain.java:142) [ ...snip... ] ---8<---8<--- End of log ---8<---8<--- ``` Another variation of this I also saw was: ```txt $ bazel build //{src,jmh,test,third_party,scala_proto}/... [ ...snip... ] ERROR: test/src/main/resources/java_sources/BUILD:11:14: Building test/src/main/resources/java_sources/CompiledWithJava11_java.jar (1 source file) failed: Worker process did not return a WorkResponse: ---8<---8<--- Start of log, file at .../bazel-workers/multiplex-worker-33-Javac.log ---8<---8<--- Error thrown by worker thread, shutting down worker. java.lang.UnsupportedClassVersionError: com/google/errorprone/ErrorProneError has been compiled by a more recent version of the Java Runtime (class file version 61.0), this version of the Java Runtime only recognizes class file versions up to 55.0 at java.base/java.lang.ClassLoader.defineClass1(Native Method) [ ...snip... ] ERROR: test/src/main/resources/java_sources/BUILD:5:14: Building test/src/main/resources/java_sources/CompiledWithJava8_java.jar (1 source file) failed: Worker process did not return a WorkResponse: ---8<---8<--- Start of log, file at .../bazel-workers/multiplex-worker-33-Javac.log ---8<---8<--- Error thrown by worker thread, shutting down worker. java.lang.UnsupportedClassVersionError: com/google/errorprone/ErrorProneError has been compiled by a more recent version of the Java Runtime (class file version 61.0), this version of the Java Runtime only recognizes class file versions up to 55.0 at java.base/java.lang.ClassLoader.defineClass1(Native Method) [ ...snip... ] ``` Updating the toolchains from `@bazel_tools//tools/jdk:` to `@rules_java//toolchains:` for targets in `test/BUILD` and `test/src/main/resources/java_sources/BUILD` fixed this particular issue. (More on updating other `@bazel_tools//tools/jdk` toolchain targets below.) Bazel 6.5.0 then failed with the following expected issue, fixed by `rules_java` 7.10.0 via: - bazelbuild/rules_java#210 - bazelbuild/rules_java@30ecf3f ```txt $ bazel build //{src,test,third_party,scala_proto}/... ERROR: .../external/rules_java/toolchains/java_toolchain_alias.bzl:83:34: Use of Starlark transition without allowlist attribute '_allowlist_function_transition'. See Starlark transitions documentation for details and usage: @rules_java//toolchains:java_toolchain_alias.bzl NORMAL ERROR: .../src/java/io/bazel/rulesscala/coverage/instrumenter/BUILD:3:12: While resolving toolchains for target //src/java/io/bazel/rulesscala/coverage/instrumenter:instrumenter: invalid registered toolchain '//toolchains:all': while parsing '//toolchains:all': error loading package '@rules_java//toolchains': initialization of module 'toolchains/java_toolchain_alias.bzl' failed ERROR: Analysis of target '//src/java/io/bazel/rulesscala/coverage/instrumenter:instrumenter' failed; build aborted: ``` Updating to `rules_java` 7.10.0 fixed the Bazel 6.5.0 build, but caused Bazel 7.4.1 to fail with the error I originally described in bazelbuild#1619. For details, see "Bump to rules_java 7.9.0 for Bazel 7 compatibility" in the message for commit cd22d88. ```txt $ bazel build //{src,test,third_party,scala_proto}/... ERROR: .../external/rules_java_builtin/toolchains/BUILD:254:14: While resolving toolchains for target @@rules_java_builtin//toolchains:platformclasspath (096dcc8): No matching toolchains found for types @@bazel_tools//tools/jdk:bootstrap_runtime_toolchain_type. To debug, rerun with --toolchain_resolution_debug='@@bazel_tools//tools/jdk:bootstrap_runtime_toolchain_type' If platforms or toolchains are a new concept for you, we'd encourage reading https://bazel.build/concepts/platforms-intro. ERROR: Analysis of target '//test/toolchains:java21_toolchain' failed; build aborted: Analysis failed ``` My analysis from bazelbuild#1619 _was_ on the right track. The `rules_java` 7.9.0 built into Bazel's `WORKSPACE` preamble registered `@bazel_tools//tools/jdk:bootstrap_runtime_toolchain_type` toolchains that later versions of `rules_java` couldn't resolve. This was due to the very same change that fixed 6.5.0, bazelbuild/rules_java#210. However, I mistakenly thought we needed to keep `@bazel_tools//tools/jdk:` as the canonical Java toolchains package instead switching to `@rules_java//toolchains:`. That's why I originally thought we were stuck on `rules_java` 7.9.0. I eventually rediscovered the following issue, which surfaced this problem a month before bazelbuild#1619. The conversation helped me realize that switching to `@rules_java//toolchains:` actually is the preferred, futureproof approach (short of migrating to Bzlmod): - bazelbuild/rules_java: Regression with `@@rules_java//toolchains:bootstrap_runtime_toolchain_type` in 7.10.0 bazelbuild/rules_java#214 Switching all `@bazel_tools//tools/jdk:` toolchains to `@rules_java//toolchains:` resolved this issue, enabling Bazel 6.5.0 and 7.4.1 to successfully build using `rules_java` 7.12.2. I then updated `rules_java` to 7.12.3, which returns to registering some toolchains as `@bazel_tools//tools/jdk:bootstrap_runtime_toolchain_type`: - bazelbuild/rules_java#246 - bazelbuild/rules_java@b64eb7d @hvadehra requested that I try 7.12.3 to see if it resolved the issue. I was able to build successfully using this version in a branch without the toolchain updates from this commit. - bazelbuild#1652 (comment) However, the changes from this commit should remain futureproof when `@bazel_tools//tools/jdk:bootstrap_runtime_toolchain_type` moves to `@rules_java//toolchains:bootstrap_runtime_toolchain_type` again one day. --- Several of the flags removed from the `scrooge_compile_with_jdk_11` test case have been no-ops for years, so removing them eliminated their corresponding deprecation warnings. `--javacopt='--release 11'`, however, caused this breakage after originally bumping to `rules_java` 7.12.2: ```txt $ RULES_SCALA_TEST_ONLY="scrooge_compile_with_jdk_11" bash ./test/shell/test_twitter_scrooge.sh running test scrooge_compile_with_jdk_11 WARNING: Option 'javabase' is deprecated WARNING: Option 'host_javabase' is deprecated WARNING: Option 'host_java_toolchain' is deprecated WARNING: Option 'java_toolchain' is deprecated WARNING: Option 'javabase' is deprecated WARNING: Option 'host_javabase' is deprecated WARNING: Option 'host_java_toolchain' is deprecated WARNING: Option 'java_toolchain' is deprecated INFO: Analyzed 64 targets (0 packages loaded, 0 targets configured). INFO: Found 64 targets... ERROR: .../src/java/io/bazel/rulesscala/scalac/compileoptions/BUILD:3:13: Compiling Java headers src/java/io/bazel/rulesscala/scalac/compileoptions/libcompileoptions-hjar.jar (1 source file) failed: (Exit 1): turbine_direct_graal failed: error executing command (from target //src/java/io/bazel/rulesscala/scalac/compileoptions:compileoptions) external/remote_java_tools_darwin_arm64/java_tools/turbine_direct_graal --output bazel-out/darwin_arm64-fastbuild/bin/src/java/io/bazel/rulesscala/scalac/compileoptions/libcompileoptions-hjar.jar ... (remaining 32 arguments skipped) Use --sandbox_debug to see verbose messages from the sandbox and retain the sandbox build root for debugging java.lang.NullPointerException: attempted to use --release, but JAVA_HOME is not set at [email protected]/java.util.Objects.requireNonNull(Objects.java:259) at com.google.turbine.binder.CtSymClassBinder.bind(CtSymClassBinder.java:55) at com.google.turbine.main.Main.release(Main.java:318) at com.google.turbine.main.Main.bootclasspath(Main.java:304) at com.google.turbine.main.Main.compile(Main.java:142) at com.google.turbine.main.Main.compile(Main.java:133) at com.google.turbine.main.Main.main(Main.java:89) at [email protected]/java.lang.invoke.LambdaForm$DMH/sa346b79c.invokeStaticInit(LambdaForm$DMH) INFO: Elapsed time: 0.325s, Critical Path: 0.12s INFO: 11 processes: 10 internal, 1 worker. FAILED: Build did NOT complete successfully Test "scrooge_compile_with_jdk_11" failed (0 sec) ``` See also: - Bazel Blog: Bazel 5.0: https://blog.bazel.build/2022/01/19/bazel-5.0.html#java - bazelbuild/bazel: `incompatible_use_toolchain_resolution_for_java_rules`: use toolchain resolution for Java rules #7849 bazelbuild/bazel#7849 --- Bazel 8 requires `rules_java` 8, per bazelbuild#1652. Updating to `rules_java` 8 will likely happen in a new major release _after_ a release containing the Bzlmod changes. This is because: - The Bzlmod changes alone will work with Bazel 6 without requiring that users update `protobuf` beyond version 21.7. - Bazel 8 requires `protobuf` >= 29.0. Bazel 6 users would have to add the afforementioned C++ compiler flags to support the newer `abseil-cpp` versions required by newer `protobuf` versions. - `rules_scala` itself needs to update ScalaPB to 1.0.0-alpha.1 or higher to support `protobuf` >= v28.0, but this ScalaPB version will _not_ support `protobuf` < v28.0. The idea is that the next major release of `rules_scala` will help users migrate to Bazel 7 and Bzlmod, in either order. Then the next major release after that will enable them to migrate to Bazel 8, with all the requisite dependency upgrades. (Technically it will still support `WORKSPACE`, but hopefully they'll make the jump to Bzlmod first.) See: - bazelbuild#1482 (comment) --- Here are the details explaining how `rules_java` 8 currently breaks `rules_scala`, up until the point that upgrading to `protobuf` >= v29.0 would fix it. `rules_java` 8.0.0 requires Bazel >= 7.3.2, which provides the `subrule` API. Compatibility with Bazel >= 6.3.0 isn't restored until `rules_java` 8.3.2. ```txt $ bazel build //{src,test,third_party,scala_proto}/... ERROR: .../external/rules_java/java/common/rules/android_lint.bzl:142:24: name 'subrule' is not defined (did you mean 'rule'?) ERROR: Error computing the main repository mapping: at .../scala/private/extensions/dev_deps.bzl:8:6: at .../external/rules_java/java/repositories.bzl:20:6: at .../external/rules_java/toolchains/local_java_repository.bzl:17:6: at .../external/rules_java/java/defs.bzl:18:6: at .../external/rules_java/java/java_library.bzl:16:6: at .../external/rules_java/java/bazel/rules/bazel_java_library.bzl:21:6: compilation of module 'java/common/rules/android_lint.bzl' failed ``` `rules_java` 8.3.0 is broken, as it can't find its own `@compatibility_proxy` repo: ```txt $ bazel build //{src,test,third_party,scala_proto}/... ERROR: error loading package under directory 'src': error loading package 'src/protobuf/io/bazel/rules_scala': at .../external/rules_java/java/defs.bzl:22:6: at .../external/rules_java/java/java_test.bzl:16:6: Unable to find package for @compatibility_proxy//:proxy.bzl: The repository '@compatibility_proxy' could not be resolved: Repository '@compatibility_proxy' is not defined. ``` `rules_java` 8.3.1 seems to fix this, presumably by importing the `protobuf` repo as `com_google_protobuf`. However, it now requires at least `protobuf` v27.0, which adds `bazel/java/lite_proto_library.bzl`. Per bazelbuild#1647, we'd have to update to ScalaPB 1.0.0-alpha.1 to support `protobuf` v28, abandoning users of previous `protobuf` versions or forcing them to upgrade. ```txt $ bazel build //{src,test,third_party,scala_proto}/... [...snip...] ERROR: error loading package under directory 'src': error loading package 'src/java/io/bazel/rulesscala/worker': at .../external/rules_java/java/defs.bzl:16:6: Label '@com_google_protobuf//bazel:java_lite_proto_library.bzl' is invalid because 'bazel' is not a package; perhaps you meant to put the colon here: '@com_google_protobuf//:bazel/java_lite_proto_library.bzl'? ```
mbland
added a commit
to mbland/rules_scala
that referenced
this issue
Dec 10, 2024
Moves `rules_scala_dependencies` to `scala/deps.bzl` and bumps several dependencies as high as they can go and still be compatible with Bazel 6.5.0 and 7.4.1. - `bazel_skylib`: 1.4.1 => 1.7.1 - `go`: 1.19.5 => 1.23.4 - `rules_cc`: 0.0.6 => 0.0.9 - `rules_go`: 0.39.1 => 0.50.1 - `rules_java`: 7.9.0 => 7.12.3 - `rules_python`: 0.36.0 => 0.38.0 The `rules_java` 7.12.13 bump precipitated the following changes: - Adds the `WORKSPACE` stanza for `rules_java` in every `WORKSPACE` file per https://github.com/bazelbuild/rules_java/releases/tag/7.12.3. This replaces previous calls to instantiate `rules_java` repos and to register `rules_java` toolchains. - Rearranges the other `WORKSPACE` setup macros for dependencies to come before the `rules_scala` setup macros. - Updates almost all dependencies on `@bazel_tools//tools/jdk:` toolchain targets and `.bzl` files to corresponding targets and files in `@rules_java//toolchains:`. - Removes several deprecated flags from the `scrooge_compile_with_jdk_11` test case from `test/shell/test_twitter_scrooge.sh`, and one obsolete flag that caused it to break. --- Moving `rules_scala_dependencies` to `scala/deps.bzl` ensures we get all the versions of dependencies we want in `WORKSPACE`, while providing a new API to consumers. It also prevents `WORKSPACE` from transitively loading any `.bzl` files that load `@rules_java`, ensuring Bazel 8 compatibility per bazelbuild#1652. Reasons for the other `rules_java` related changes include: - The `WORKSPACE` stanza for `rules_java` should've already been present while using the existing version 7.9.0. However, doing so would've broken Bazel 6 builds, as described in detail below. - The `rules_java_toolchains()` call follows the `protobuf_deps()` call instead of immediately following `rules_java_dependencies()` because upgrading to `rules_java` >= 8.5.0 will require this. It has no adverse impact to do it now, amidst the other `WORKSPACE` changes, and will make the eventual `rules_java` >= 8.5.0 diff smaller. - Having the other `WORKSPACE` setup macros for dependencies come before the `rules_scala` setup macros helps ensure consistent, correct initialization before `rules_scala` initialization. - Updating the toolchain specifiers to use `@rules_java//toolchains` fixed `WORKSPACE` build breakages when updating to `rules_java` 7.10.0 and later. This is a potentially breaking change for consumers, but in the good kind of way, in that it requires an easy, futureproof update. (`@bazel_tools//tools/jdk:toolchain_type` dependencies remain, as there's not yet a corresponding `@rules_java//toolchains` target.) What follows are detailed notes on why some dependency versions are capped where they are, and on some breakages fixed by these changes. --- `abseil-cpp` and `protobuf` have to stay at 20220623.1 and v21.7, respectively, for Bazel 6 compatibility per bazelbuild#1647. `protobuf` up to v25.5 is compatible with Bazel 6 provided users set the compiler flags mentioned in that issue: ```txt build:linux --cxxopt=-std=c++17 build:linux --host_cxxopt=-std=c++17 build:macos --cxxopt=-std=c++17 build:macos --host_cxxopt=-std=c++17 build:windows --cxxopt=/std=c++17 build:windows --host_cxxopt=/std=c++17 ``` --- `rules_python` 0.38.0 => 0.39.0 requires at least `rules_cc` 0.0.10, which introduced `cc/common/cc_info.bzl`: ```txt $ bazel build //{src,test,third_party,scala_proto}/... [ ...snip... ] ERROR: error loading package under directory 'test': error loading package 'test': at .../external/rules_python/python/defs.bzl:17:6: at .../external/rules_python/python/py_binary.bzl:18:6: at .../external/rules_python/python/private/py_binary_macro.bzl:16:6: at .../external/rules_python/python/private/common_bazel.bzl:19:6: Label '@rules_cc//cc/common:cc_info.bzl' is invalid because 'cc/common' is not a package; perhaps you meant to put the colon here: '@rules_cc//cc:common/cc_info.bzl'? ``` However, `rules_cc` 0.0.9 => 0.0.10 requires Bazel 7, which defines `CcSharedLibraryHintInfo`: ```txt $ bazel build //{src,test,third_party,scala_proto}/... ERROR: .../external/rules_cc/cc/private/rules_impl/native.bzl:40:33: name 'CcSharedLibraryHintInfo' is not defined (did you mean 'CcSharedLibraryInfo'?) [ ...snip... ] ERROR: error loading package under directory 'test': error loading package 'test': at .../external/rules_python/python/defs.bzl:17:6: at .../external/rules_python/python/py_binary.bzl:18:6: at .../external/rules_python/python/private/py_binary_macro.bzl:16:6: at .../external/rules_python/python/private/common_bazel.bzl:18:6: at .../external/rules_cc/cc/common/cc_common.bzl:17:6: compilation of module 'cc/private/rules_impl/native.bzl' failed ``` --- This section provides a methodical description of the errors encountered and resolved during each step of the `rules_java` update. The proper `WORKSPACE` setup stanza for `rules_java`, which we didn't originally have in place, is: ```py load( "@rules_java//java:repositories.bzl", "rules_java_dependencies", "rules_java_toolchains", ) rules_java_dependencies() rules_java_toolchains() ``` Adding this stanza to `WORKSPACE` while still using `rules_java` 7.9.0 didn't break the Bazel 7.4.1 build, but it broke the Bazel 6.5.0 build. Note the `CompiledWithJava{8,11}_java.jar` references: ```txt $ bazel build //{src,test,third_party,scala_proto}/... [ ...snip... ] ERROR: .../test/src/main/resources/java_sources/BUILD:5:14: Building test/src/main/resources/java_sources/CompiledWithJava8_java.jar (1 source file) failed: Worker process did not return a WorkResponse: ---8<---8<--- Start of log, file at .../bazel-workers/multiplex-worker-18-Javac.log ---8<---8<--- Error thrown by worker thread, shutting down worker. java.lang.NoSuchMethodError: 'java.lang.Iterable com.google.devtools.build.buildjar.javac.BlazeJavacMain$ClassloaderMaskingFileManager.getJavaFileObjectsFromPaths(java.util.Collection)' at com.google.devtools.build.buildjar.javac.BlazeJavacMain.compile(BlazeJavacMain.java:142) [ ...snip... ] ---8<---8<--- End of log ---8<---8<--- ERROR: .../test/src/main/resources/java_sources/BUILD:11:14: Building test/src/main/resources/java_sources/CompiledWithJava11_java.jar (1 source file) failed: Worker process did not return a WorkResponse: ---8<---8<--- Start of log, file at .../bazel-workers/multiplex-worker-18-Javac.log ---8<---8<--- Error thrown by worker thread, shutting down worker. java.lang.NoSuchMethodError: 'java.lang.Iterable com.google.devtools.build.buildjar.javac.BlazeJavacMain$ClassloaderMaskingFileManager.getJavaFileObjectsFromPaths(java.util.Collection)' at com.google.devtools.build.buildjar.javac.BlazeJavacMain.compile(BlazeJavacMain.java:142) [ ...snip... ] ---8<---8<--- End of log ---8<---8<--- ``` Another variation of this I also saw was: ```txt $ bazel build //{src,jmh,test,third_party,scala_proto}/... [ ...snip... ] ERROR: test/src/main/resources/java_sources/BUILD:11:14: Building test/src/main/resources/java_sources/CompiledWithJava11_java.jar (1 source file) failed: Worker process did not return a WorkResponse: ---8<---8<--- Start of log, file at .../bazel-workers/multiplex-worker-33-Javac.log ---8<---8<--- Error thrown by worker thread, shutting down worker. java.lang.UnsupportedClassVersionError: com/google/errorprone/ErrorProneError has been compiled by a more recent version of the Java Runtime (class file version 61.0), this version of the Java Runtime only recognizes class file versions up to 55.0 at java.base/java.lang.ClassLoader.defineClass1(Native Method) [ ...snip... ] ERROR: test/src/main/resources/java_sources/BUILD:5:14: Building test/src/main/resources/java_sources/CompiledWithJava8_java.jar (1 source file) failed: Worker process did not return a WorkResponse: ---8<---8<--- Start of log, file at .../bazel-workers/multiplex-worker-33-Javac.log ---8<---8<--- Error thrown by worker thread, shutting down worker. java.lang.UnsupportedClassVersionError: com/google/errorprone/ErrorProneError has been compiled by a more recent version of the Java Runtime (class file version 61.0), this version of the Java Runtime only recognizes class file versions up to 55.0 at java.base/java.lang.ClassLoader.defineClass1(Native Method) [ ...snip... ] ``` Updating the toolchains from `@bazel_tools//tools/jdk:` to `@rules_java//toolchains:` for targets in `test/BUILD` and `test/src/main/resources/java_sources/BUILD` fixed this particular issue. (More on updating other `@bazel_tools//tools/jdk` toolchain targets below.) Bazel 6.5.0 then failed with the following expected issue, fixed by `rules_java` 7.10.0 via: - bazelbuild/rules_java#210 - bazelbuild/rules_java@30ecf3f ```txt $ bazel build //{src,test,third_party,scala_proto}/... ERROR: .../external/rules_java/toolchains/java_toolchain_alias.bzl:83:34: Use of Starlark transition without allowlist attribute '_allowlist_function_transition'. See Starlark transitions documentation for details and usage: @rules_java//toolchains:java_toolchain_alias.bzl NORMAL ERROR: .../src/java/io/bazel/rulesscala/coverage/instrumenter/BUILD:3:12: While resolving toolchains for target //src/java/io/bazel/rulesscala/coverage/instrumenter:instrumenter: invalid registered toolchain '//toolchains:all': while parsing '//toolchains:all': error loading package '@rules_java//toolchains': initialization of module 'toolchains/java_toolchain_alias.bzl' failed ERROR: Analysis of target '//src/java/io/bazel/rulesscala/coverage/instrumenter:instrumenter' failed; build aborted: ``` Updating to `rules_java` 7.10.0 fixed the Bazel 6.5.0 build, but caused Bazel 7.4.1 to fail with the error I originally described in bazelbuild#1619. For details, see "Bump to rules_java 7.9.0 for Bazel 7 compatibility" in the message for commit cd22d88. ```txt $ bazel build //{src,test,third_party,scala_proto}/... ERROR: .../external/rules_java_builtin/toolchains/BUILD:254:14: While resolving toolchains for target @@rules_java_builtin//toolchains:platformclasspath (096dcc8): No matching toolchains found for types @@bazel_tools//tools/jdk:bootstrap_runtime_toolchain_type. To debug, rerun with --toolchain_resolution_debug='@@bazel_tools//tools/jdk:bootstrap_runtime_toolchain_type' If platforms or toolchains are a new concept for you, we'd encourage reading https://bazel.build/concepts/platforms-intro. ERROR: Analysis of target '//test/toolchains:java21_toolchain' failed; build aborted: Analysis failed ``` My analysis from bazelbuild#1619 _was_ on the right track. The `rules_java` 7.9.0 built into Bazel's `WORKSPACE` preamble registered `@bazel_tools//tools/jdk:bootstrap_runtime_toolchain_type` toolchains that later versions of `rules_java` couldn't resolve. This was due to the very same change that fixed 6.5.0, bazelbuild/rules_java#210. However, I mistakenly thought we needed to keep `@bazel_tools//tools/jdk:` as the canonical Java toolchains package instead switching to `@rules_java//toolchains:`. That's why I originally thought we were stuck on `rules_java` 7.9.0. I eventually rediscovered the following issue, which surfaced this problem a month before bazelbuild#1619. The conversation helped me realize that switching to `@rules_java//toolchains:` actually is the preferred, futureproof approach (short of migrating to Bzlmod): - bazelbuild/rules_java: Regression with `@@rules_java//toolchains:bootstrap_runtime_toolchain_type` in 7.10.0 bazelbuild/rules_java#214 Switching all `@bazel_tools//tools/jdk:` toolchains to `@rules_java//toolchains:` resolved this issue, enabling Bazel 6.5.0 and 7.4.1 to successfully build using `rules_java` 7.12.2. I then updated `rules_java` to 7.12.3, which returns to registering some toolchains as `@bazel_tools//tools/jdk:bootstrap_runtime_toolchain_type`: - bazelbuild/rules_java#246 - bazelbuild/rules_java@b64eb7d @hvadehra requested that I try 7.12.3 to see if it resolved the issue. I was able to build successfully using this version in a branch without the toolchain updates from this commit. - bazelbuild#1652 (comment) However, the changes from this commit should remain futureproof when `@bazel_tools//tools/jdk:bootstrap_runtime_toolchain_type` moves to `@rules_java//toolchains:bootstrap_runtime_toolchain_type` again one day. --- Several of the flags removed from the `scrooge_compile_with_jdk_11` test case have been no-ops for years, so removing them eliminated their corresponding deprecation warnings. `--javacopt='--release 11'`, however, caused this breakage after originally bumping to `rules_java` 7.12.2: ```txt $ RULES_SCALA_TEST_ONLY="scrooge_compile_with_jdk_11" bash ./test/shell/test_twitter_scrooge.sh running test scrooge_compile_with_jdk_11 WARNING: Option 'javabase' is deprecated WARNING: Option 'host_javabase' is deprecated WARNING: Option 'host_java_toolchain' is deprecated WARNING: Option 'java_toolchain' is deprecated WARNING: Option 'javabase' is deprecated WARNING: Option 'host_javabase' is deprecated WARNING: Option 'host_java_toolchain' is deprecated WARNING: Option 'java_toolchain' is deprecated INFO: Analyzed 64 targets (0 packages loaded, 0 targets configured). INFO: Found 64 targets... ERROR: .../src/java/io/bazel/rulesscala/scalac/compileoptions/BUILD:3:13: Compiling Java headers src/java/io/bazel/rulesscala/scalac/compileoptions/libcompileoptions-hjar.jar (1 source file) failed: (Exit 1): turbine_direct_graal failed: error executing command (from target //src/java/io/bazel/rulesscala/scalac/compileoptions:compileoptions) external/remote_java_tools_darwin_arm64/java_tools/turbine_direct_graal --output bazel-out/darwin_arm64-fastbuild/bin/src/java/io/bazel/rulesscala/scalac/compileoptions/libcompileoptions-hjar.jar ... (remaining 32 arguments skipped) Use --sandbox_debug to see verbose messages from the sandbox and retain the sandbox build root for debugging java.lang.NullPointerException: attempted to use --release, but JAVA_HOME is not set at [email protected]/java.util.Objects.requireNonNull(Objects.java:259) at com.google.turbine.binder.CtSymClassBinder.bind(CtSymClassBinder.java:55) at com.google.turbine.main.Main.release(Main.java:318) at com.google.turbine.main.Main.bootclasspath(Main.java:304) at com.google.turbine.main.Main.compile(Main.java:142) at com.google.turbine.main.Main.compile(Main.java:133) at com.google.turbine.main.Main.main(Main.java:89) at [email protected]/java.lang.invoke.LambdaForm$DMH/sa346b79c.invokeStaticInit(LambdaForm$DMH) INFO: Elapsed time: 0.325s, Critical Path: 0.12s INFO: 11 processes: 10 internal, 1 worker. FAILED: Build did NOT complete successfully Test "scrooge_compile_with_jdk_11" failed (0 sec) ``` See also: - Bazel Blog: Bazel 5.0: https://blog.bazel.build/2022/01/19/bazel-5.0.html#java - bazelbuild/bazel: `incompatible_use_toolchain_resolution_for_java_rules`: use toolchain resolution for Java rules #7849 bazelbuild/bazel#7849 --- Bazel 8 requires `rules_java` 8, per bazelbuild#1652. Updating to `rules_java` 8 will likely happen in a new major release _after_ a release containing the Bzlmod changes. This is because: - The Bzlmod changes alone will work with Bazel 6 without requiring that users update `protobuf` beyond version 21.7. - Bazel 8 requires `protobuf` >= 29.0. Bazel 6 users would have to add the afforementioned C++ compiler flags to support the newer `abseil-cpp` versions required by newer `protobuf` versions. - `rules_scala` itself needs to update ScalaPB to 1.0.0-alpha.1 or higher to support `protobuf` >= v28.0, but this ScalaPB version will _not_ support `protobuf` < v28.0. The idea is that the next major release of `rules_scala` will help users migrate to Bazel 7 and Bzlmod, in either order. Then the next major release after that will enable them to migrate to Bazel 8, with all the requisite dependency upgrades. (Technically it will still support `WORKSPACE`, but hopefully they'll make the jump to Bzlmod first.) See: - bazelbuild#1482 (comment) --- Here are the details explaining how `rules_java` 8 currently breaks `rules_scala`, up until the point that upgrading to `protobuf` >= v29.0 would fix it. `rules_java` 8.0.0 requires Bazel >= 7.3.2, which provides the `subrule` API. Compatibility with Bazel >= 6.3.0 isn't restored until `rules_java` 8.3.2. ```txt $ bazel build //{src,test,third_party,scala_proto}/... ERROR: .../external/rules_java/java/common/rules/android_lint.bzl:142:24: name 'subrule' is not defined (did you mean 'rule'?) ERROR: Error computing the main repository mapping: at .../scala/private/extensions/dev_deps.bzl:8:6: at .../external/rules_java/java/repositories.bzl:20:6: at .../external/rules_java/toolchains/local_java_repository.bzl:17:6: at .../external/rules_java/java/defs.bzl:18:6: at .../external/rules_java/java/java_library.bzl:16:6: at .../external/rules_java/java/bazel/rules/bazel_java_library.bzl:21:6: compilation of module 'java/common/rules/android_lint.bzl' failed ``` `rules_java` 8.3.0 is broken, as it can't find its own `@compatibility_proxy` repo: ```txt $ bazel build //{src,test,third_party,scala_proto}/... ERROR: error loading package under directory 'src': error loading package 'src/protobuf/io/bazel/rules_scala': at .../external/rules_java/java/defs.bzl:22:6: at .../external/rules_java/java/java_test.bzl:16:6: Unable to find package for @compatibility_proxy//:proxy.bzl: The repository '@compatibility_proxy' could not be resolved: Repository '@compatibility_proxy' is not defined. ``` `rules_java` 8.3.1 seems to fix this, presumably by importing the `protobuf` repo as `com_google_protobuf`. However, it now requires at least `protobuf` v27.0, which adds `bazel/java/lite_proto_library.bzl`. Per bazelbuild#1647, we'd have to update to ScalaPB 1.0.0-alpha.1 to support `protobuf` v28, abandoning users of previous `protobuf` versions or forcing them to upgrade. ```txt $ bazel build //{src,test,third_party,scala_proto}/... [...snip...] ERROR: error loading package under directory 'src': error loading package 'src/java/io/bazel/rulesscala/worker': at .../external/rules_java/java/defs.bzl:16:6: Label '@com_google_protobuf//bazel:java_lite_proto_library.bzl' is invalid because 'bazel' is not a package; perhaps you meant to put the colon here: '@com_google_protobuf//:bazel/java_lite_proto_library.bzl'? ```
mbland
added a commit
to mbland/rules_scala
that referenced
this issue
Dec 10, 2024
Moves `rules_scala_dependencies` to `scala/deps.bzl` and bumps several dependencies as high as they can go and still be compatible with Bazel 6.5.0 and 7.4.1. - `bazel_skylib`: 1.4.1 => 1.7.1 - `go`: 1.19.5 => 1.23.4 - `rules_cc`: 0.0.6 => 0.0.9 - `rules_go`: 0.39.1 => 0.50.1 - `rules_java`: 7.9.0 => 7.12.3 - `rules_python`: 0.36.0 => 0.38.0 The `rules_java` 7.12.13 bump precipitated the following changes: - Adds the `WORKSPACE` stanza for `rules_java` in every `WORKSPACE` file per https://github.com/bazelbuild/rules_java/releases/tag/7.12.3. This replaces previous calls to instantiate `rules_java` repos and to register `rules_java` toolchains. - Rearranges the other `WORKSPACE` setup macros for dependencies to come before the `rules_scala` setup macros. - Updates almost all dependencies on `@bazel_tools//tools/jdk:` toolchain targets and `.bzl` files to corresponding targets and files in `@rules_java//toolchains:`. - Removes several deprecated flags from the `scrooge_compile_with_jdk_11` test case from `test/shell/test_twitter_scrooge.sh`, and one obsolete flag that caused it to break. --- Moving `rules_scala_dependencies` to `scala/deps.bzl` ensures we get all the versions of dependencies we want in `WORKSPACE`, while providing a new API to consumers. It also prevents `WORKSPACE` from transitively loading any `.bzl` files that load `@rules_java`, ensuring Bazel 8 compatibility per bazelbuild#1652. Reasons for the other `rules_java` related changes include: - The `WORKSPACE` stanza for `rules_java` should've already been present while using the existing version 7.9.0. However, doing so would've broken Bazel 6 builds, as described in detail below. - The `rules_java_toolchains()` call follows the `protobuf_deps()` call instead of immediately following `rules_java_dependencies()` because upgrading to `rules_java` >= 8.5.0 will require this. It has no adverse impact to do it now, amidst the other `WORKSPACE` changes, and will make the eventual `rules_java` >= 8.5.0 diff smaller. - Having the other `WORKSPACE` setup macros for dependencies come before the `rules_scala` setup macros helps ensure consistent, correct initialization before `rules_scala` initialization. - Updating the toolchain specifiers to use `@rules_java//toolchains` fixed `WORKSPACE` build breakages when updating to `rules_java` 7.10.0 and later. This is a potentially breaking change for consumers, but in the good kind of way, in that it requires an easy, futureproof update. (`@bazel_tools//tools/jdk:toolchain_type` dependencies remain, as there's not yet a corresponding `@rules_java//toolchains` target.) What follows are detailed notes on why some dependency versions are capped where they are, and on some breakages fixed by these changes. --- `abseil-cpp` and `protobuf` have to stay at 20220623.1 and v21.7, respectively, for Bazel 6 compatibility per bazelbuild#1647. `protobuf` up to v25.5 is compatible with Bazel 6 provided users set the compiler flags mentioned in that issue: ```txt build:linux --cxxopt=-std=c++17 build:linux --host_cxxopt=-std=c++17 build:macos --cxxopt=-std=c++17 build:macos --host_cxxopt=-std=c++17 build:windows --cxxopt=/std=c++17 build:windows --host_cxxopt=/std=c++17 ``` --- `rules_python` 0.38.0 => 0.39.0 requires at least `rules_cc` 0.0.10, which introduced `cc/common/cc_info.bzl`: ```txt $ bazel build //{src,test,third_party,scala_proto}/... [ ...snip... ] ERROR: error loading package under directory 'test': error loading package 'test': at .../external/rules_python/python/defs.bzl:17:6: at .../external/rules_python/python/py_binary.bzl:18:6: at .../external/rules_python/python/private/py_binary_macro.bzl:16:6: at .../external/rules_python/python/private/common_bazel.bzl:19:6: Label '@rules_cc//cc/common:cc_info.bzl' is invalid because 'cc/common' is not a package; perhaps you meant to put the colon here: '@rules_cc//cc:common/cc_info.bzl'? ``` However, `rules_cc` 0.0.9 => 0.0.10 requires Bazel 7, which defines `CcSharedLibraryHintInfo`: ```txt $ bazel build //{src,test,third_party,scala_proto}/... ERROR: .../external/rules_cc/cc/private/rules_impl/native.bzl:40:33: name 'CcSharedLibraryHintInfo' is not defined (did you mean 'CcSharedLibraryInfo'?) [ ...snip... ] ERROR: error loading package under directory 'test': error loading package 'test': at .../external/rules_python/python/defs.bzl:17:6: at .../external/rules_python/python/py_binary.bzl:18:6: at .../external/rules_python/python/private/py_binary_macro.bzl:16:6: at .../external/rules_python/python/private/common_bazel.bzl:18:6: at .../external/rules_cc/cc/common/cc_common.bzl:17:6: compilation of module 'cc/private/rules_impl/native.bzl' failed ``` --- This section provides a methodical description of the errors encountered and resolved during each step of the `rules_java` update. The proper `WORKSPACE` setup stanza for `rules_java`, which we didn't originally have in place, is: ```py load( "@rules_java//java:repositories.bzl", "rules_java_dependencies", "rules_java_toolchains", ) rules_java_dependencies() rules_java_toolchains() ``` Adding this stanza to `WORKSPACE` while still using `rules_java` 7.9.0 didn't break the Bazel 7.4.1 build, but it broke the Bazel 6.5.0 build. Note the `CompiledWithJava{8,11}_java.jar` references: ```txt $ bazel build //{src,test,third_party,scala_proto}/... [ ...snip... ] ERROR: .../test/src/main/resources/java_sources/BUILD:5:14: Building test/src/main/resources/java_sources/CompiledWithJava8_java.jar (1 source file) failed: Worker process did not return a WorkResponse: ---8<---8<--- Start of log, file at .../bazel-workers/multiplex-worker-18-Javac.log ---8<---8<--- Error thrown by worker thread, shutting down worker. java.lang.NoSuchMethodError: 'java.lang.Iterable com.google.devtools.build.buildjar.javac.BlazeJavacMain$ClassloaderMaskingFileManager.getJavaFileObjectsFromPaths(java.util.Collection)' at com.google.devtools.build.buildjar.javac.BlazeJavacMain.compile(BlazeJavacMain.java:142) [ ...snip... ] ---8<---8<--- End of log ---8<---8<--- ERROR: .../test/src/main/resources/java_sources/BUILD:11:14: Building test/src/main/resources/java_sources/CompiledWithJava11_java.jar (1 source file) failed: Worker process did not return a WorkResponse: ---8<---8<--- Start of log, file at .../bazel-workers/multiplex-worker-18-Javac.log ---8<---8<--- Error thrown by worker thread, shutting down worker. java.lang.NoSuchMethodError: 'java.lang.Iterable com.google.devtools.build.buildjar.javac.BlazeJavacMain$ClassloaderMaskingFileManager.getJavaFileObjectsFromPaths(java.util.Collection)' at com.google.devtools.build.buildjar.javac.BlazeJavacMain.compile(BlazeJavacMain.java:142) [ ...snip... ] ---8<---8<--- End of log ---8<---8<--- ``` Another variation of this I also saw was: ```txt $ bazel build //{src,jmh,test,third_party,scala_proto}/... [ ...snip... ] ERROR: test/src/main/resources/java_sources/BUILD:11:14: Building test/src/main/resources/java_sources/CompiledWithJava11_java.jar (1 source file) failed: Worker process did not return a WorkResponse: ---8<---8<--- Start of log, file at .../bazel-workers/multiplex-worker-33-Javac.log ---8<---8<--- Error thrown by worker thread, shutting down worker. java.lang.UnsupportedClassVersionError: com/google/errorprone/ErrorProneError has been compiled by a more recent version of the Java Runtime (class file version 61.0), this version of the Java Runtime only recognizes class file versions up to 55.0 at java.base/java.lang.ClassLoader.defineClass1(Native Method) [ ...snip... ] ERROR: test/src/main/resources/java_sources/BUILD:5:14: Building test/src/main/resources/java_sources/CompiledWithJava8_java.jar (1 source file) failed: Worker process did not return a WorkResponse: ---8<---8<--- Start of log, file at .../bazel-workers/multiplex-worker-33-Javac.log ---8<---8<--- Error thrown by worker thread, shutting down worker. java.lang.UnsupportedClassVersionError: com/google/errorprone/ErrorProneError has been compiled by a more recent version of the Java Runtime (class file version 61.0), this version of the Java Runtime only recognizes class file versions up to 55.0 at java.base/java.lang.ClassLoader.defineClass1(Native Method) [ ...snip... ] ``` Updating the toolchains from `@bazel_tools//tools/jdk:` to `@rules_java//toolchains:` for targets in `test/BUILD` and `test/src/main/resources/java_sources/BUILD` fixed this particular issue. (More on updating other `@bazel_tools//tools/jdk` toolchain targets below.) Bazel 6.5.0 then failed with the following expected issue, fixed by `rules_java` 7.10.0 via: - bazelbuild/rules_java#210 - bazelbuild/rules_java@30ecf3f ```txt $ bazel build //{src,test,third_party,scala_proto}/... ERROR: .../external/rules_java/toolchains/java_toolchain_alias.bzl:83:34: Use of Starlark transition without allowlist attribute '_allowlist_function_transition'. See Starlark transitions documentation for details and usage: @rules_java//toolchains:java_toolchain_alias.bzl NORMAL ERROR: .../src/java/io/bazel/rulesscala/coverage/instrumenter/BUILD:3:12: While resolving toolchains for target //src/java/io/bazel/rulesscala/coverage/instrumenter:instrumenter: invalid registered toolchain '//toolchains:all': while parsing '//toolchains:all': error loading package '@rules_java//toolchains': initialization of module 'toolchains/java_toolchain_alias.bzl' failed ERROR: Analysis of target '//src/java/io/bazel/rulesscala/coverage/instrumenter:instrumenter' failed; build aborted: ``` Updating to `rules_java` 7.10.0 fixed the Bazel 6.5.0 build, but caused Bazel 7.4.1 to fail with the error I originally described in bazelbuild#1619. For details, see "Bump to rules_java 7.9.0 for Bazel 7 compatibility" in the message for commit cd22d88. ```txt $ bazel build //{src,test,third_party,scala_proto}/... ERROR: .../external/rules_java_builtin/toolchains/BUILD:254:14: While resolving toolchains for target @@rules_java_builtin//toolchains:platformclasspath (096dcc8): No matching toolchains found for types @@bazel_tools//tools/jdk:bootstrap_runtime_toolchain_type. To debug, rerun with --toolchain_resolution_debug='@@bazel_tools//tools/jdk:bootstrap_runtime_toolchain_type' If platforms or toolchains are a new concept for you, we'd encourage reading https://bazel.build/concepts/platforms-intro. ERROR: Analysis of target '//test/toolchains:java21_toolchain' failed; build aborted: Analysis failed ``` My analysis from bazelbuild#1619 _was_ on the right track. The `rules_java` 7.9.0 built into Bazel's `WORKSPACE` preamble registered `@bazel_tools//tools/jdk:bootstrap_runtime_toolchain_type` toolchains that later versions of `rules_java` couldn't resolve. This was due to the very same change that fixed 6.5.0, bazelbuild/rules_java#210. However, I mistakenly thought we needed to keep `@bazel_tools//tools/jdk:` as the canonical Java toolchains package instead switching to `@rules_java//toolchains:`. That's why I originally thought we were stuck on `rules_java` 7.9.0. I eventually rediscovered the following issue, which surfaced this problem a month before bazelbuild#1619. The conversation helped me realize that switching to `@rules_java//toolchains:` actually is the preferred, futureproof approach (short of migrating to Bzlmod): - bazelbuild/rules_java: Regression with `@@rules_java//toolchains:bootstrap_runtime_toolchain_type` in 7.10.0 bazelbuild/rules_java#214 Switching all `@bazel_tools//tools/jdk:` toolchains to `@rules_java//toolchains:` resolved this issue, enabling Bazel 6.5.0 and 7.4.1 to successfully build using `rules_java` 7.12.2. I then updated `rules_java` to 7.12.3, which returns to registering some toolchains as `@bazel_tools//tools/jdk:bootstrap_runtime_toolchain_type`: - bazelbuild/rules_java#246 - bazelbuild/rules_java@b64eb7d @hvadehra requested that I try 7.12.3 to see if it resolved the issue. I was able to build successfully using this version in a branch without the toolchain updates from this commit. - bazelbuild#1652 (comment) However, the changes from this commit should remain futureproof when `@bazel_tools//tools/jdk:bootstrap_runtime_toolchain_type` moves to `@rules_java//toolchains:bootstrap_runtime_toolchain_type` again one day. --- Several of the flags removed from the `scrooge_compile_with_jdk_11` test case have been no-ops for years, so removing them eliminated their corresponding deprecation warnings. `--javacopt='--release 11'`, however, caused this breakage after originally bumping to `rules_java` 7.12.2: ```txt $ RULES_SCALA_TEST_ONLY="scrooge_compile_with_jdk_11" bash ./test/shell/test_twitter_scrooge.sh running test scrooge_compile_with_jdk_11 WARNING: Option 'javabase' is deprecated WARNING: Option 'host_javabase' is deprecated WARNING: Option 'host_java_toolchain' is deprecated WARNING: Option 'java_toolchain' is deprecated WARNING: Option 'javabase' is deprecated WARNING: Option 'host_javabase' is deprecated WARNING: Option 'host_java_toolchain' is deprecated WARNING: Option 'java_toolchain' is deprecated INFO: Analyzed 64 targets (0 packages loaded, 0 targets configured). INFO: Found 64 targets... ERROR: .../src/java/io/bazel/rulesscala/scalac/compileoptions/BUILD:3:13: Compiling Java headers src/java/io/bazel/rulesscala/scalac/compileoptions/libcompileoptions-hjar.jar (1 source file) failed: (Exit 1): turbine_direct_graal failed: error executing command (from target //src/java/io/bazel/rulesscala/scalac/compileoptions:compileoptions) external/remote_java_tools_darwin_arm64/java_tools/turbine_direct_graal --output bazel-out/darwin_arm64-fastbuild/bin/src/java/io/bazel/rulesscala/scalac/compileoptions/libcompileoptions-hjar.jar ... (remaining 32 arguments skipped) Use --sandbox_debug to see verbose messages from the sandbox and retain the sandbox build root for debugging java.lang.NullPointerException: attempted to use --release, but JAVA_HOME is not set at [email protected]/java.util.Objects.requireNonNull(Objects.java:259) at com.google.turbine.binder.CtSymClassBinder.bind(CtSymClassBinder.java:55) at com.google.turbine.main.Main.release(Main.java:318) at com.google.turbine.main.Main.bootclasspath(Main.java:304) at com.google.turbine.main.Main.compile(Main.java:142) at com.google.turbine.main.Main.compile(Main.java:133) at com.google.turbine.main.Main.main(Main.java:89) at [email protected]/java.lang.invoke.LambdaForm$DMH/sa346b79c.invokeStaticInit(LambdaForm$DMH) INFO: Elapsed time: 0.325s, Critical Path: 0.12s INFO: 11 processes: 10 internal, 1 worker. FAILED: Build did NOT complete successfully Test "scrooge_compile_with_jdk_11" failed (0 sec) ``` See also: - Bazel Blog: Bazel 5.0: https://blog.bazel.build/2022/01/19/bazel-5.0.html#java - bazelbuild/bazel: `incompatible_use_toolchain_resolution_for_java_rules`: use toolchain resolution for Java rules #7849 bazelbuild/bazel#7849 --- Bazel 8 requires `rules_java` 8, per bazelbuild#1652. Updating to `rules_java` 8 will likely happen in a new major release _after_ a release containing the Bzlmod changes. This is because: - The Bzlmod changes alone will work with Bazel 6 without requiring that users update `protobuf` beyond version 21.7. - Bazel 8 requires `protobuf` >= 29.0. Bazel 6 users would have to add the afforementioned C++ compiler flags to support the newer `abseil-cpp` versions required by newer `protobuf` versions. - `rules_scala` itself needs to update ScalaPB to 1.0.0-alpha.1 or higher to support `protobuf` >= v28.0, but this ScalaPB version will _not_ support `protobuf` < v28.0. The idea is that the next major release of `rules_scala` will help users migrate to Bazel 7 and Bzlmod, in either order. Then the next major release after that will enable them to migrate to Bazel 8, with all the requisite dependency upgrades. (Technically it will still support `WORKSPACE`, but hopefully they'll make the jump to Bzlmod first.) See: - bazelbuild#1482 (comment) --- Here are the details explaining how `rules_java` 8 currently breaks `rules_scala`, up until the point that upgrading to `protobuf` >= v29.0 would fix it. `rules_java` 8.0.0 requires Bazel >= 7.3.2, which provides the `subrule` API. Compatibility with Bazel >= 6.3.0 isn't restored until `rules_java` 8.3.2. ```txt $ bazel build //{src,test,third_party,scala_proto}/... ERROR: .../external/rules_java/java/common/rules/android_lint.bzl:142:24: name 'subrule' is not defined (did you mean 'rule'?) ERROR: Error computing the main repository mapping: at .../scala/private/extensions/dev_deps.bzl:8:6: at .../external/rules_java/java/repositories.bzl:20:6: at .../external/rules_java/toolchains/local_java_repository.bzl:17:6: at .../external/rules_java/java/defs.bzl:18:6: at .../external/rules_java/java/java_library.bzl:16:6: at .../external/rules_java/java/bazel/rules/bazel_java_library.bzl:21:6: compilation of module 'java/common/rules/android_lint.bzl' failed ``` `rules_java` 8.3.0 is broken, as it can't find its own `@compatibility_proxy` repo: ```txt $ bazel build //{src,test,third_party,scala_proto}/... ERROR: error loading package under directory 'src': error loading package 'src/protobuf/io/bazel/rules_scala': at .../external/rules_java/java/defs.bzl:22:6: at .../external/rules_java/java/java_test.bzl:16:6: Unable to find package for @compatibility_proxy//:proxy.bzl: The repository '@compatibility_proxy' could not be resolved: Repository '@compatibility_proxy' is not defined. ``` `rules_java` 8.3.1 seems to fix this, presumably by importing the `protobuf` repo as `com_google_protobuf`. However, it now requires at least `protobuf` v27.0, which adds `bazel/java/lite_proto_library.bzl`. Per bazelbuild#1647, we'd have to update to ScalaPB 1.0.0-alpha.1 to support `protobuf` v28, abandoning users of previous `protobuf` versions or forcing them to upgrade. ```txt $ bazel build //{src,test,third_party,scala_proto}/... [...snip...] ERROR: error loading package under directory 'src': error loading package 'src/java/io/bazel/rulesscala/worker': at .../external/rules_java/java/defs.bzl:16:6: Label '@com_google_protobuf//bazel:java_lite_proto_library.bzl' is invalid because 'bazel' is not a package; perhaps you meant to put the colon here: '@com_google_protobuf//:bazel/java_lite_proto_library.bzl'? ```
mbland
added a commit
to mbland/rules_scala
that referenced
this issue
Dec 16, 2024
Moves `rules_scala_dependencies` to `scala/deps.bzl` and bumps several dependencies as high as they can go and still be compatible with Bazel 6.5.0 and 7.4.1. - `bazel_skylib`: 1.4.1 => 1.7.1 - `go`: 1.19.5 => 1.23.4 - `rules_cc`: 0.0.6 => 0.0.9 - `rules_go`: 0.39.1 => 0.50.1 - `rules_java`: 7.9.0 => 7.12.3 - `rules_python`: 0.36.0 => 0.38.0 The `rules_java` 7.12.13 bump precipitated the following changes: - Adds the `WORKSPACE` stanza for `rules_java` in every `WORKSPACE` file per https://github.com/bazelbuild/rules_java/releases/tag/7.12.3. This replaces previous calls to instantiate `rules_java` repos and to register `rules_java` toolchains. - Rearranges the other `WORKSPACE` setup macros for dependencies to come before the `rules_scala` setup macros. - Updates almost all dependencies on `@bazel_tools//tools/jdk:` toolchain targets and `.bzl` files to corresponding targets and files in `@rules_java//toolchains:`. - Removes several deprecated flags from the `scrooge_compile_with_jdk_11` test case from `test/shell/test_twitter_scrooge.sh`, and one obsolete flag that caused it to break. --- Moving `rules_scala_dependencies` to `scala/deps.bzl` ensures we get all the versions of dependencies we want in `WORKSPACE`, while providing a new API to consumers. It also prevents `WORKSPACE` from transitively loading any `.bzl` files that load `@rules_java`, ensuring Bazel 8 compatibility per bazelbuild#1652. Reasons for the other `rules_java` related changes include: - The `WORKSPACE` stanza for `rules_java` should've already been present while using the existing version 7.9.0. However, doing so would've broken Bazel 6 builds, as described in detail below. - The `rules_java_toolchains()` call follows the `protobuf_deps()` call instead of immediately following `rules_java_dependencies()` because upgrading to `rules_java` >= 8.5.0 will require this. It has no adverse impact to do it now, amidst the other `WORKSPACE` changes, and will make the eventual `rules_java` >= 8.5.0 diff smaller. - Having the other `WORKSPACE` setup macros for dependencies come before the `rules_scala` setup macros helps ensure consistent, correct initialization before `rules_scala` initialization. - Updating the toolchain specifiers to use `@rules_java//toolchains` fixed `WORKSPACE` build breakages when updating to `rules_java` 7.10.0 and later. This is a potentially breaking change for consumers, but in the good kind of way, in that it requires an easy, futureproof update. (`@bazel_tools//tools/jdk:toolchain_type` dependencies remain, as there's not yet a corresponding `@rules_java//toolchains` target.) What follows are detailed notes on why some dependency versions are capped where they are, and on some breakages fixed by these changes. --- `abseil-cpp` and `protobuf` have to stay at 20220623.1 and v21.7, respectively, for Bazel 6 compatibility per bazelbuild#1647. `protobuf` up to v25.5 is compatible with Bazel 6 provided users set the compiler flags mentioned in that issue: ```txt build:linux --cxxopt=-std=c++17 build:linux --host_cxxopt=-std=c++17 build:macos --cxxopt=-std=c++17 build:macos --host_cxxopt=-std=c++17 build:windows --cxxopt=/std=c++17 build:windows --host_cxxopt=/std=c++17 ``` --- `rules_python` 0.38.0 => 0.39.0 requires at least `rules_cc` 0.0.10, which introduced `cc/common/cc_info.bzl`: ```txt $ bazel build //{src,test,third_party,scala_proto}/... [ ...snip... ] ERROR: error loading package under directory 'test': error loading package 'test': at .../external/rules_python/python/defs.bzl:17:6: at .../external/rules_python/python/py_binary.bzl:18:6: at .../external/rules_python/python/private/py_binary_macro.bzl:16:6: at .../external/rules_python/python/private/common_bazel.bzl:19:6: Label '@rules_cc//cc/common:cc_info.bzl' is invalid because 'cc/common' is not a package; perhaps you meant to put the colon here: '@rules_cc//cc:common/cc_info.bzl'? ``` However, `rules_cc` 0.0.9 => 0.0.10 requires Bazel 7, which defines `CcSharedLibraryHintInfo`: ```txt $ bazel build //{src,test,third_party,scala_proto}/... ERROR: .../external/rules_cc/cc/private/rules_impl/native.bzl:40:33: name 'CcSharedLibraryHintInfo' is not defined (did you mean 'CcSharedLibraryInfo'?) [ ...snip... ] ERROR: error loading package under directory 'test': error loading package 'test': at .../external/rules_python/python/defs.bzl:17:6: at .../external/rules_python/python/py_binary.bzl:18:6: at .../external/rules_python/python/private/py_binary_macro.bzl:16:6: at .../external/rules_python/python/private/common_bazel.bzl:18:6: at .../external/rules_cc/cc/common/cc_common.bzl:17:6: compilation of module 'cc/private/rules_impl/native.bzl' failed ``` --- This section provides a methodical description of the errors encountered and resolved during each step of the `rules_java` update. The proper `WORKSPACE` setup stanza for `rules_java`, which we didn't originally have in place, is: ```py load( "@rules_java//java:repositories.bzl", "rules_java_dependencies", "rules_java_toolchains", ) rules_java_dependencies() rules_java_toolchains() ``` Adding this stanza to `WORKSPACE` while still using `rules_java` 7.9.0 didn't break the Bazel 7.4.1 build, but it broke the Bazel 6.5.0 build. Note the `CompiledWithJava{8,11}_java.jar` references: ```txt $ bazel build //{src,test,third_party,scala_proto}/... [ ...snip... ] ERROR: .../test/src/main/resources/java_sources/BUILD:5:14: Building test/src/main/resources/java_sources/CompiledWithJava8_java.jar (1 source file) failed: Worker process did not return a WorkResponse: ---8<---8<--- Start of log, file at .../bazel-workers/multiplex-worker-18-Javac.log ---8<---8<--- Error thrown by worker thread, shutting down worker. java.lang.NoSuchMethodError: 'java.lang.Iterable com.google.devtools.build.buildjar.javac.BlazeJavacMain$ClassloaderMaskingFileManager.getJavaFileObjectsFromPaths(java.util.Collection)' at com.google.devtools.build.buildjar.javac.BlazeJavacMain.compile(BlazeJavacMain.java:142) [ ...snip... ] ---8<---8<--- End of log ---8<---8<--- ERROR: .../test/src/main/resources/java_sources/BUILD:11:14: Building test/src/main/resources/java_sources/CompiledWithJava11_java.jar (1 source file) failed: Worker process did not return a WorkResponse: ---8<---8<--- Start of log, file at .../bazel-workers/multiplex-worker-18-Javac.log ---8<---8<--- Error thrown by worker thread, shutting down worker. java.lang.NoSuchMethodError: 'java.lang.Iterable com.google.devtools.build.buildjar.javac.BlazeJavacMain$ClassloaderMaskingFileManager.getJavaFileObjectsFromPaths(java.util.Collection)' at com.google.devtools.build.buildjar.javac.BlazeJavacMain.compile(BlazeJavacMain.java:142) [ ...snip... ] ---8<---8<--- End of log ---8<---8<--- ``` Another variation of this I also saw was: ```txt $ bazel build //{src,jmh,test,third_party,scala_proto}/... [ ...snip... ] ERROR: test/src/main/resources/java_sources/BUILD:11:14: Building test/src/main/resources/java_sources/CompiledWithJava11_java.jar (1 source file) failed: Worker process did not return a WorkResponse: ---8<---8<--- Start of log, file at .../bazel-workers/multiplex-worker-33-Javac.log ---8<---8<--- Error thrown by worker thread, shutting down worker. java.lang.UnsupportedClassVersionError: com/google/errorprone/ErrorProneError has been compiled by a more recent version of the Java Runtime (class file version 61.0), this version of the Java Runtime only recognizes class file versions up to 55.0 at java.base/java.lang.ClassLoader.defineClass1(Native Method) [ ...snip... ] ERROR: test/src/main/resources/java_sources/BUILD:5:14: Building test/src/main/resources/java_sources/CompiledWithJava8_java.jar (1 source file) failed: Worker process did not return a WorkResponse: ---8<---8<--- Start of log, file at .../bazel-workers/multiplex-worker-33-Javac.log ---8<---8<--- Error thrown by worker thread, shutting down worker. java.lang.UnsupportedClassVersionError: com/google/errorprone/ErrorProneError has been compiled by a more recent version of the Java Runtime (class file version 61.0), this version of the Java Runtime only recognizes class file versions up to 55.0 at java.base/java.lang.ClassLoader.defineClass1(Native Method) [ ...snip... ] ``` Updating the toolchains from `@bazel_tools//tools/jdk:` to `@rules_java//toolchains:` for targets in `test/BUILD` and `test/src/main/resources/java_sources/BUILD` fixed this particular issue. (More on updating other `@bazel_tools//tools/jdk` toolchain targets below.) Bazel 6.5.0 then failed with the following expected issue, fixed by `rules_java` 7.10.0 via: - bazelbuild/rules_java#210 - bazelbuild/rules_java@30ecf3f ```txt $ bazel build //{src,test,third_party,scala_proto}/... ERROR: .../external/rules_java/toolchains/java_toolchain_alias.bzl:83:34: Use of Starlark transition without allowlist attribute '_allowlist_function_transition'. See Starlark transitions documentation for details and usage: @rules_java//toolchains:java_toolchain_alias.bzl NORMAL ERROR: .../src/java/io/bazel/rulesscala/coverage/instrumenter/BUILD:3:12: While resolving toolchains for target //src/java/io/bazel/rulesscala/coverage/instrumenter:instrumenter: invalid registered toolchain '//toolchains:all': while parsing '//toolchains:all': error loading package '@rules_java//toolchains': initialization of module 'toolchains/java_toolchain_alias.bzl' failed ERROR: Analysis of target '//src/java/io/bazel/rulesscala/coverage/instrumenter:instrumenter' failed; build aborted: ``` Updating to `rules_java` 7.10.0 fixed the Bazel 6.5.0 build, but caused Bazel 7.4.1 to fail with the error I originally described in bazelbuild#1619. For details, see "Bump to rules_java 7.9.0 for Bazel 7 compatibility" in the message for commit cd22d88. ```txt $ bazel build //{src,test,third_party,scala_proto}/... ERROR: .../external/rules_java_builtin/toolchains/BUILD:254:14: While resolving toolchains for target @@rules_java_builtin//toolchains:platformclasspath (096dcc8): No matching toolchains found for types @@bazel_tools//tools/jdk:bootstrap_runtime_toolchain_type. To debug, rerun with --toolchain_resolution_debug='@@bazel_tools//tools/jdk:bootstrap_runtime_toolchain_type' If platforms or toolchains are a new concept for you, we'd encourage reading https://bazel.build/concepts/platforms-intro. ERROR: Analysis of target '//test/toolchains:java21_toolchain' failed; build aborted: Analysis failed ``` My analysis from bazelbuild#1619 _was_ on the right track. The `rules_java` 7.9.0 built into Bazel's `WORKSPACE` preamble registered `@bazel_tools//tools/jdk:bootstrap_runtime_toolchain_type` toolchains that later versions of `rules_java` couldn't resolve. This was due to the very same change that fixed 6.5.0, bazelbuild/rules_java#210. However, I mistakenly thought we needed to keep `@bazel_tools//tools/jdk:` as the canonical Java toolchains package instead switching to `@rules_java//toolchains:`. That's why I originally thought we were stuck on `rules_java` 7.9.0. I eventually rediscovered the following issue, which surfaced this problem a month before bazelbuild#1619. The conversation helped me realize that switching to `@rules_java//toolchains:` actually is the preferred, futureproof approach (short of migrating to Bzlmod): - bazelbuild/rules_java: Regression with `@@rules_java//toolchains:bootstrap_runtime_toolchain_type` in 7.10.0 bazelbuild/rules_java#214 Switching all `@bazel_tools//tools/jdk:` toolchains to `@rules_java//toolchains:` resolved this issue, enabling Bazel 6.5.0 and 7.4.1 to successfully build using `rules_java` 7.12.2. I then updated `rules_java` to 7.12.3, which returns to registering some toolchains as `@bazel_tools//tools/jdk:bootstrap_runtime_toolchain_type`: - bazelbuild/rules_java#246 - bazelbuild/rules_java@b64eb7d @hvadehra requested that I try 7.12.3 to see if it resolved the issue. I was able to build successfully using this version in a branch without the toolchain updates from this commit. - bazelbuild#1652 (comment) However, the changes from this commit should remain futureproof when `@bazel_tools//tools/jdk:bootstrap_runtime_toolchain_type` moves to `@rules_java//toolchains:bootstrap_runtime_toolchain_type` again one day. --- Several of the flags removed from the `scrooge_compile_with_jdk_11` test case have been no-ops for years, so removing them eliminated their corresponding deprecation warnings. `--javacopt='--release 11'`, however, caused this breakage after originally bumping to `rules_java` 7.12.2: ```txt $ RULES_SCALA_TEST_ONLY="scrooge_compile_with_jdk_11" bash ./test/shell/test_twitter_scrooge.sh running test scrooge_compile_with_jdk_11 WARNING: Option 'javabase' is deprecated WARNING: Option 'host_javabase' is deprecated WARNING: Option 'host_java_toolchain' is deprecated WARNING: Option 'java_toolchain' is deprecated WARNING: Option 'javabase' is deprecated WARNING: Option 'host_javabase' is deprecated WARNING: Option 'host_java_toolchain' is deprecated WARNING: Option 'java_toolchain' is deprecated INFO: Analyzed 64 targets (0 packages loaded, 0 targets configured). INFO: Found 64 targets... ERROR: .../src/java/io/bazel/rulesscala/scalac/compileoptions/BUILD:3:13: Compiling Java headers src/java/io/bazel/rulesscala/scalac/compileoptions/libcompileoptions-hjar.jar (1 source file) failed: (Exit 1): turbine_direct_graal failed: error executing command (from target //src/java/io/bazel/rulesscala/scalac/compileoptions:compileoptions) external/remote_java_tools_darwin_arm64/java_tools/turbine_direct_graal --output bazel-out/darwin_arm64-fastbuild/bin/src/java/io/bazel/rulesscala/scalac/compileoptions/libcompileoptions-hjar.jar ... (remaining 32 arguments skipped) Use --sandbox_debug to see verbose messages from the sandbox and retain the sandbox build root for debugging java.lang.NullPointerException: attempted to use --release, but JAVA_HOME is not set at [email protected]/java.util.Objects.requireNonNull(Objects.java:259) at com.google.turbine.binder.CtSymClassBinder.bind(CtSymClassBinder.java:55) at com.google.turbine.main.Main.release(Main.java:318) at com.google.turbine.main.Main.bootclasspath(Main.java:304) at com.google.turbine.main.Main.compile(Main.java:142) at com.google.turbine.main.Main.compile(Main.java:133) at com.google.turbine.main.Main.main(Main.java:89) at [email protected]/java.lang.invoke.LambdaForm$DMH/sa346b79c.invokeStaticInit(LambdaForm$DMH) INFO: Elapsed time: 0.325s, Critical Path: 0.12s INFO: 11 processes: 10 internal, 1 worker. FAILED: Build did NOT complete successfully Test "scrooge_compile_with_jdk_11" failed (0 sec) ``` See also: - Bazel Blog: Bazel 5.0: https://blog.bazel.build/2022/01/19/bazel-5.0.html#java - bazelbuild/bazel: `incompatible_use_toolchain_resolution_for_java_rules`: use toolchain resolution for Java rules #7849 bazelbuild/bazel#7849 --- Bazel 8 requires `rules_java` 8, per bazelbuild#1652. Updating to `rules_java` 8 will likely happen in a new major release _after_ a release containing the Bzlmod changes. This is because: - The Bzlmod changes alone will work with Bazel 6 without requiring that users update `protobuf` beyond version 21.7. - Bazel 8 requires `protobuf` >= 29.0. Bazel 6 users would have to add the afforementioned C++ compiler flags to support the newer `abseil-cpp` versions required by newer `protobuf` versions. - `rules_scala` itself needs to update ScalaPB to 1.0.0-alpha.1 or higher to support `protobuf` >= v28.0, but this ScalaPB version will _not_ support `protobuf` < v28.0. The idea is that the next major release of `rules_scala` will help users migrate to Bazel 7 and Bzlmod, in either order. Then the next major release after that will enable them to migrate to Bazel 8, with all the requisite dependency upgrades. (Technically it will still support `WORKSPACE`, but hopefully they'll make the jump to Bzlmod first.) See: - bazelbuild#1482 (comment) --- Here are the details explaining how `rules_java` 8 currently breaks `rules_scala`, up until the point that upgrading to `protobuf` >= v29.0 would fix it. `rules_java` 8.0.0 requires Bazel >= 7.3.2, which provides the `subrule` API. Compatibility with Bazel >= 6.3.0 isn't restored until `rules_java` 8.3.2. ```txt $ bazel build //{src,test,third_party,scala_proto}/... ERROR: .../external/rules_java/java/common/rules/android_lint.bzl:142:24: name 'subrule' is not defined (did you mean 'rule'?) ERROR: Error computing the main repository mapping: at .../scala/private/extensions/dev_deps.bzl:8:6: at .../external/rules_java/java/repositories.bzl:20:6: at .../external/rules_java/toolchains/local_java_repository.bzl:17:6: at .../external/rules_java/java/defs.bzl:18:6: at .../external/rules_java/java/java_library.bzl:16:6: at .../external/rules_java/java/bazel/rules/bazel_java_library.bzl:21:6: compilation of module 'java/common/rules/android_lint.bzl' failed ``` `rules_java` 8.3.0 is broken, as it can't find its own `@compatibility_proxy` repo: ```txt $ bazel build //{src,test,third_party,scala_proto}/... ERROR: error loading package under directory 'src': error loading package 'src/protobuf/io/bazel/rules_scala': at .../external/rules_java/java/defs.bzl:22:6: at .../external/rules_java/java/java_test.bzl:16:6: Unable to find package for @compatibility_proxy//:proxy.bzl: The repository '@compatibility_proxy' could not be resolved: Repository '@compatibility_proxy' is not defined. ``` `rules_java` 8.3.1 seems to fix this, presumably by importing the `protobuf` repo as `com_google_protobuf`. However, it now requires at least `protobuf` v27.0, which adds `bazel/java/lite_proto_library.bzl`. Per bazelbuild#1647, we'd have to update to ScalaPB 1.0.0-alpha.1 to support `protobuf` v28, abandoning users of previous `protobuf` versions or forcing them to upgrade. ```txt $ bazel build //{src,test,third_party,scala_proto}/... [...snip...] ERROR: error loading package under directory 'src': error loading package 'src/java/io/bazel/rulesscala/worker': at .../external/rules_java/java/defs.bzl:16:6: Label '@com_google_protobuf//bazel:java_lite_proto_library.bzl' is invalid because 'bazel' is not a package; perhaps you meant to put the colon here: '@com_google_protobuf//:bazel/java_lite_proto_library.bzl'? ```
mbland
added a commit
to mbland/rules_scala
that referenced
this issue
Jan 14, 2025
Moves `rules_scala_dependencies` to `scala/deps.bzl` and bumps several dependencies as high as they can go and still be compatible with Bazel 6.5.0 and 7.4.1. - `bazel_skylib`: 1.4.1 => 1.7.1 - `go`: 1.19.5 => 1.23.4 - `rules_cc`: 0.0.6 => 0.0.9 - `rules_go`: 0.39.1 => 0.50.1 - `rules_java`: 7.9.0 => 7.12.3 - `rules_python`: 0.36.0 => 0.38.0 The `rules_java` 7.12.13 bump precipitated the following changes: - Adds the `WORKSPACE` stanza for `rules_java` in every `WORKSPACE` file per https://github.com/bazelbuild/rules_java/releases/tag/7.12.3. This replaces previous calls to instantiate `rules_java` repos and to register `rules_java` toolchains. - Rearranges the other `WORKSPACE` setup macros for dependencies to come before the `rules_scala` setup macros. - Updates almost all dependencies on `@bazel_tools//tools/jdk:` toolchain targets and `.bzl` files to corresponding targets and files in `@rules_java//toolchains:`. - Removes several deprecated flags from the `scrooge_compile_with_jdk_11` test case from `test/shell/test_twitter_scrooge.sh`, and one obsolete flag that caused it to break. --- Moving `rules_scala_dependencies` to `scala/deps.bzl` ensures we get all the versions of dependencies we want in `WORKSPACE`, while providing a new API to consumers. It also prevents `WORKSPACE` from transitively loading any `.bzl` files that load `@rules_java`, ensuring Bazel 8 compatibility per bazelbuild#1652. Reasons for the other `rules_java` related changes include: - The `WORKSPACE` stanza for `rules_java` should've already been present while using the existing version 7.9.0. However, doing so would've broken Bazel 6 builds, as described in detail below. - The `rules_java_toolchains()` call follows the `protobuf_deps()` call instead of immediately following `rules_java_dependencies()` because upgrading to `rules_java` >= 8.5.0 will require this. It has no adverse impact to do it now, amidst the other `WORKSPACE` changes, and will make the eventual `rules_java` >= 8.5.0 diff smaller. - Having the other `WORKSPACE` setup macros for dependencies come before the `rules_scala` setup macros helps ensure consistent, correct initialization before `rules_scala` initialization. - Updating the toolchain specifiers to use `@rules_java//toolchains` fixed `WORKSPACE` build breakages when updating to `rules_java` 7.10.0 and later. This is a potentially breaking change for consumers, but in the good kind of way, in that it requires an easy, futureproof update. (`@bazel_tools//tools/jdk:toolchain_type` dependencies remain, as there's not yet a corresponding `@rules_java//toolchains` target.) What follows are detailed notes on why some dependency versions are capped where they are, and on some breakages fixed by these changes. --- `abseil-cpp` and `protobuf` have to stay at 20220623.1 and v21.7, respectively, for Bazel 6 compatibility per bazelbuild#1647. `protobuf` up to v25.5 is compatible with Bazel 6 provided users set the compiler flags mentioned in that issue: ```txt build:linux --cxxopt=-std=c++17 build:linux --host_cxxopt=-std=c++17 build:macos --cxxopt=-std=c++17 build:macos --host_cxxopt=-std=c++17 build:windows --cxxopt=/std=c++17 build:windows --host_cxxopt=/std=c++17 ``` --- `rules_python` 0.38.0 => 0.39.0 requires at least `rules_cc` 0.0.10, which introduced `cc/common/cc_info.bzl`: ```txt $ bazel build //{src,test,third_party,scala_proto}/... [ ...snip... ] ERROR: error loading package under directory 'test': error loading package 'test': at .../external/rules_python/python/defs.bzl:17:6: at .../external/rules_python/python/py_binary.bzl:18:6: at .../external/rules_python/python/private/py_binary_macro.bzl:16:6: at .../external/rules_python/python/private/common_bazel.bzl:19:6: Label '@rules_cc//cc/common:cc_info.bzl' is invalid because 'cc/common' is not a package; perhaps you meant to put the colon here: '@rules_cc//cc:common/cc_info.bzl'? ``` However, `rules_cc` 0.0.9 => 0.0.10 requires Bazel 7, which defines `CcSharedLibraryHintInfo`: ```txt $ bazel build //{src,test,third_party,scala_proto}/... ERROR: .../external/rules_cc/cc/private/rules_impl/native.bzl:40:33: name 'CcSharedLibraryHintInfo' is not defined (did you mean 'CcSharedLibraryInfo'?) [ ...snip... ] ERROR: error loading package under directory 'test': error loading package 'test': at .../external/rules_python/python/defs.bzl:17:6: at .../external/rules_python/python/py_binary.bzl:18:6: at .../external/rules_python/python/private/py_binary_macro.bzl:16:6: at .../external/rules_python/python/private/common_bazel.bzl:18:6: at .../external/rules_cc/cc/common/cc_common.bzl:17:6: compilation of module 'cc/private/rules_impl/native.bzl' failed ``` --- This section provides a methodical description of the errors encountered and resolved during each step of the `rules_java` update. The proper `WORKSPACE` setup stanza for `rules_java`, which we didn't originally have in place, is: ```py load( "@rules_java//java:repositories.bzl", "rules_java_dependencies", "rules_java_toolchains", ) rules_java_dependencies() rules_java_toolchains() ``` Adding this stanza to `WORKSPACE` while still using `rules_java` 7.9.0 didn't break the Bazel 7.4.1 build, but it broke the Bazel 6.5.0 build. Note the `CompiledWithJava{8,11}_java.jar` references: ```txt $ bazel build //{src,test,third_party,scala_proto}/... [ ...snip... ] ERROR: .../test/src/main/resources/java_sources/BUILD:5:14: Building test/src/main/resources/java_sources/CompiledWithJava8_java.jar (1 source file) failed: Worker process did not return a WorkResponse: ---8<---8<--- Start of log, file at .../bazel-workers/multiplex-worker-18-Javac.log ---8<---8<--- Error thrown by worker thread, shutting down worker. java.lang.NoSuchMethodError: 'java.lang.Iterable com.google.devtools.build.buildjar.javac.BlazeJavacMain$ClassloaderMaskingFileManager.getJavaFileObjectsFromPaths(java.util.Collection)' at com.google.devtools.build.buildjar.javac.BlazeJavacMain.compile(BlazeJavacMain.java:142) [ ...snip... ] ---8<---8<--- End of log ---8<---8<--- ERROR: .../test/src/main/resources/java_sources/BUILD:11:14: Building test/src/main/resources/java_sources/CompiledWithJava11_java.jar (1 source file) failed: Worker process did not return a WorkResponse: ---8<---8<--- Start of log, file at .../bazel-workers/multiplex-worker-18-Javac.log ---8<---8<--- Error thrown by worker thread, shutting down worker. java.lang.NoSuchMethodError: 'java.lang.Iterable com.google.devtools.build.buildjar.javac.BlazeJavacMain$ClassloaderMaskingFileManager.getJavaFileObjectsFromPaths(java.util.Collection)' at com.google.devtools.build.buildjar.javac.BlazeJavacMain.compile(BlazeJavacMain.java:142) [ ...snip... ] ---8<---8<--- End of log ---8<---8<--- ``` Another variation of this I also saw was: ```txt $ bazel build //{src,jmh,test,third_party,scala_proto}/... [ ...snip... ] ERROR: test/src/main/resources/java_sources/BUILD:11:14: Building test/src/main/resources/java_sources/CompiledWithJava11_java.jar (1 source file) failed: Worker process did not return a WorkResponse: ---8<---8<--- Start of log, file at .../bazel-workers/multiplex-worker-33-Javac.log ---8<---8<--- Error thrown by worker thread, shutting down worker. java.lang.UnsupportedClassVersionError: com/google/errorprone/ErrorProneError has been compiled by a more recent version of the Java Runtime (class file version 61.0), this version of the Java Runtime only recognizes class file versions up to 55.0 at java.base/java.lang.ClassLoader.defineClass1(Native Method) [ ...snip... ] ERROR: test/src/main/resources/java_sources/BUILD:5:14: Building test/src/main/resources/java_sources/CompiledWithJava8_java.jar (1 source file) failed: Worker process did not return a WorkResponse: ---8<---8<--- Start of log, file at .../bazel-workers/multiplex-worker-33-Javac.log ---8<---8<--- Error thrown by worker thread, shutting down worker. java.lang.UnsupportedClassVersionError: com/google/errorprone/ErrorProneError has been compiled by a more recent version of the Java Runtime (class file version 61.0), this version of the Java Runtime only recognizes class file versions up to 55.0 at java.base/java.lang.ClassLoader.defineClass1(Native Method) [ ...snip... ] ``` Updating the toolchains from `@bazel_tools//tools/jdk:` to `@rules_java//toolchains:` for targets in `test/BUILD` and `test/src/main/resources/java_sources/BUILD` fixed this particular issue. (More on updating other `@bazel_tools//tools/jdk` toolchain targets below.) Bazel 6.5.0 then failed with the following expected issue, fixed by `rules_java` 7.10.0 via: - bazelbuild/rules_java#210 - bazelbuild/rules_java@30ecf3f ```txt $ bazel build //{src,test,third_party,scala_proto}/... ERROR: .../external/rules_java/toolchains/java_toolchain_alias.bzl:83:34: Use of Starlark transition without allowlist attribute '_allowlist_function_transition'. See Starlark transitions documentation for details and usage: @rules_java//toolchains:java_toolchain_alias.bzl NORMAL ERROR: .../src/java/io/bazel/rulesscala/coverage/instrumenter/BUILD:3:12: While resolving toolchains for target //src/java/io/bazel/rulesscala/coverage/instrumenter:instrumenter: invalid registered toolchain '//toolchains:all': while parsing '//toolchains:all': error loading package '@rules_java//toolchains': initialization of module 'toolchains/java_toolchain_alias.bzl' failed ERROR: Analysis of target '//src/java/io/bazel/rulesscala/coverage/instrumenter:instrumenter' failed; build aborted: ``` Updating to `rules_java` 7.10.0 fixed the Bazel 6.5.0 build, but caused Bazel 7.4.1 to fail with the error I originally described in bazelbuild#1619. For details, see "Bump to rules_java 7.9.0 for Bazel 7 compatibility" in the message for commit cd22d88. ```txt $ bazel build //{src,test,third_party,scala_proto}/... ERROR: .../external/rules_java_builtin/toolchains/BUILD:254:14: While resolving toolchains for target @@rules_java_builtin//toolchains:platformclasspath (096dcc8): No matching toolchains found for types @@bazel_tools//tools/jdk:bootstrap_runtime_toolchain_type. To debug, rerun with --toolchain_resolution_debug='@@bazel_tools//tools/jdk:bootstrap_runtime_toolchain_type' If platforms or toolchains are a new concept for you, we'd encourage reading https://bazel.build/concepts/platforms-intro. ERROR: Analysis of target '//test/toolchains:java21_toolchain' failed; build aborted: Analysis failed ``` My analysis from bazelbuild#1619 _was_ on the right track. The `rules_java` 7.9.0 built into Bazel's `WORKSPACE` preamble registered `@bazel_tools//tools/jdk:bootstrap_runtime_toolchain_type` toolchains that later versions of `rules_java` couldn't resolve. This was due to the very same change that fixed 6.5.0, bazelbuild/rules_java#210. However, I mistakenly thought we needed to keep `@bazel_tools//tools/jdk:` as the canonical Java toolchains package instead switching to `@rules_java//toolchains:`. That's why I originally thought we were stuck on `rules_java` 7.9.0. I eventually rediscovered the following issue, which surfaced this problem a month before bazelbuild#1619. The conversation helped me realize that switching to `@rules_java//toolchains:` actually is the preferred, futureproof approach (short of migrating to Bzlmod): - bazelbuild/rules_java: Regression with `@@rules_java//toolchains:bootstrap_runtime_toolchain_type` in 7.10.0 bazelbuild/rules_java#214 Switching all `@bazel_tools//tools/jdk:` toolchains to `@rules_java//toolchains:` resolved this issue, enabling Bazel 6.5.0 and 7.4.1 to successfully build using `rules_java` 7.12.2. I then updated `rules_java` to 7.12.3, which returns to registering some toolchains as `@bazel_tools//tools/jdk:bootstrap_runtime_toolchain_type`: - bazelbuild/rules_java#246 - bazelbuild/rules_java@b64eb7d @hvadehra requested that I try 7.12.3 to see if it resolved the issue. I was able to build successfully using this version in a branch without the toolchain updates from this commit. - bazelbuild#1652 (comment) However, the changes from this commit should remain futureproof when `@bazel_tools//tools/jdk:bootstrap_runtime_toolchain_type` moves to `@rules_java//toolchains:bootstrap_runtime_toolchain_type` again one day. --- Several of the flags removed from the `scrooge_compile_with_jdk_11` test case have been no-ops for years, so removing them eliminated their corresponding deprecation warnings. `--javacopt='--release 11'`, however, caused this breakage after originally bumping to `rules_java` 7.12.2: ```txt $ RULES_SCALA_TEST_ONLY="scrooge_compile_with_jdk_11" bash ./test/shell/test_twitter_scrooge.sh running test scrooge_compile_with_jdk_11 WARNING: Option 'javabase' is deprecated WARNING: Option 'host_javabase' is deprecated WARNING: Option 'host_java_toolchain' is deprecated WARNING: Option 'java_toolchain' is deprecated WARNING: Option 'javabase' is deprecated WARNING: Option 'host_javabase' is deprecated WARNING: Option 'host_java_toolchain' is deprecated WARNING: Option 'java_toolchain' is deprecated INFO: Analyzed 64 targets (0 packages loaded, 0 targets configured). INFO: Found 64 targets... ERROR: .../src/java/io/bazel/rulesscala/scalac/compileoptions/BUILD:3:13: Compiling Java headers src/java/io/bazel/rulesscala/scalac/compileoptions/libcompileoptions-hjar.jar (1 source file) failed: (Exit 1): turbine_direct_graal failed: error executing command (from target //src/java/io/bazel/rulesscala/scalac/compileoptions:compileoptions) external/remote_java_tools_darwin_arm64/java_tools/turbine_direct_graal --output bazel-out/darwin_arm64-fastbuild/bin/src/java/io/bazel/rulesscala/scalac/compileoptions/libcompileoptions-hjar.jar ... (remaining 32 arguments skipped) Use --sandbox_debug to see verbose messages from the sandbox and retain the sandbox build root for debugging java.lang.NullPointerException: attempted to use --release, but JAVA_HOME is not set at [email protected]/java.util.Objects.requireNonNull(Objects.java:259) at com.google.turbine.binder.CtSymClassBinder.bind(CtSymClassBinder.java:55) at com.google.turbine.main.Main.release(Main.java:318) at com.google.turbine.main.Main.bootclasspath(Main.java:304) at com.google.turbine.main.Main.compile(Main.java:142) at com.google.turbine.main.Main.compile(Main.java:133) at com.google.turbine.main.Main.main(Main.java:89) at [email protected]/java.lang.invoke.LambdaForm$DMH/sa346b79c.invokeStaticInit(LambdaForm$DMH) INFO: Elapsed time: 0.325s, Critical Path: 0.12s INFO: 11 processes: 10 internal, 1 worker. FAILED: Build did NOT complete successfully Test "scrooge_compile_with_jdk_11" failed (0 sec) ``` See also: - Bazel Blog: Bazel 5.0: https://blog.bazel.build/2022/01/19/bazel-5.0.html#java - bazelbuild/bazel: `incompatible_use_toolchain_resolution_for_java_rules`: use toolchain resolution for Java rules #7849 bazelbuild/bazel#7849 --- Bazel 8 requires `rules_java` 8, per bazelbuild#1652. Updating to `rules_java` 8 will likely happen in a new major release _after_ a release containing the Bzlmod changes. This is because: - The Bzlmod changes alone will work with Bazel 6 without requiring that users update `protobuf` beyond version 21.7. - Bazel 8 requires `protobuf` >= 29.0. Bazel 6 users would have to add the afforementioned C++ compiler flags to support the newer `abseil-cpp` versions required by newer `protobuf` versions. - `rules_scala` itself needs to update ScalaPB to 1.0.0-alpha.1 or higher to support `protobuf` >= v28.0, but this ScalaPB version will _not_ support `protobuf` < v28.0. The idea is that the next major release of `rules_scala` will help users migrate to Bazel 7 and Bzlmod, in either order. Then the next major release after that will enable them to migrate to Bazel 8, with all the requisite dependency upgrades. (Technically it will still support `WORKSPACE`, but hopefully they'll make the jump to Bzlmod first.) See: - bazelbuild#1482 (comment) --- Here are the details explaining how `rules_java` 8 currently breaks `rules_scala`, up until the point that upgrading to `protobuf` >= v29.0 would fix it. `rules_java` 8.0.0 requires Bazel >= 7.3.2, which provides the `subrule` API. Compatibility with Bazel >= 6.3.0 isn't restored until `rules_java` 8.3.2. ```txt $ bazel build //{src,test,third_party,scala_proto}/... ERROR: .../external/rules_java/java/common/rules/android_lint.bzl:142:24: name 'subrule' is not defined (did you mean 'rule'?) ERROR: Error computing the main repository mapping: at .../scala/private/extensions/dev_deps.bzl:8:6: at .../external/rules_java/java/repositories.bzl:20:6: at .../external/rules_java/toolchains/local_java_repository.bzl:17:6: at .../external/rules_java/java/defs.bzl:18:6: at .../external/rules_java/java/java_library.bzl:16:6: at .../external/rules_java/java/bazel/rules/bazel_java_library.bzl:21:6: compilation of module 'java/common/rules/android_lint.bzl' failed ``` `rules_java` 8.3.0 is broken, as it can't find its own `@compatibility_proxy` repo: ```txt $ bazel build //{src,test,third_party,scala_proto}/... ERROR: error loading package under directory 'src': error loading package 'src/protobuf/io/bazel/rules_scala': at .../external/rules_java/java/defs.bzl:22:6: at .../external/rules_java/java/java_test.bzl:16:6: Unable to find package for @compatibility_proxy//:proxy.bzl: The repository '@compatibility_proxy' could not be resolved: Repository '@compatibility_proxy' is not defined. ``` `rules_java` 8.3.1 seems to fix this, presumably by importing the `protobuf` repo as `com_google_protobuf`. However, it now requires at least `protobuf` v27.0, which adds `bazel/java/lite_proto_library.bzl`. Per bazelbuild#1647, we'd have to update to ScalaPB 1.0.0-alpha.1 to support `protobuf` v28, abandoning users of previous `protobuf` versions or forcing them to upgrade. ```txt $ bazel build //{src,test,third_party,scala_proto}/... [...snip...] ERROR: error loading package under directory 'src': error loading package 'src/java/io/bazel/rulesscala/worker': at .../external/rules_java/java/defs.bzl:16:6: Label '@com_google_protobuf//bazel:java_lite_proto_library.bzl' is invalid because 'bazel' is not a package; perhaps you meant to put the colon here: '@com_google_protobuf//:bazel/java_lite_proto_library.bzl'? ```
liucijus
pushed a commit
to bazelbuild/rules_scala
that referenced
this issue
Jan 14, 2025
* Move `rules_scala_dependencies` to scala/deps.bzl Moves `rules_scala_dependencies` to `scala/deps.bzl` and bumps several dependencies as high as they can go and still be compatible with Bazel 6.5.0 and 7.4.1. - `bazel_skylib`: 1.4.1 => 1.7.1 - `go`: 1.19.5 => 1.23.4 - `rules_cc`: 0.0.6 => 0.0.9 - `rules_go`: 0.39.1 => 0.50.1 - `rules_java`: 7.9.0 => 7.12.3 - `rules_python`: 0.36.0 => 0.38.0 The `rules_java` 7.12.13 bump precipitated the following changes: - Adds the `WORKSPACE` stanza for `rules_java` in every `WORKSPACE` file per https://github.com/bazelbuild/rules_java/releases/tag/7.12.3. This replaces previous calls to instantiate `rules_java` repos and to register `rules_java` toolchains. - Rearranges the other `WORKSPACE` setup macros for dependencies to come before the `rules_scala` setup macros. - Updates almost all dependencies on `@bazel_tools//tools/jdk:` toolchain targets and `.bzl` files to corresponding targets and files in `@rules_java//toolchains:`. - Removes several deprecated flags from the `scrooge_compile_with_jdk_11` test case from `test/shell/test_twitter_scrooge.sh`, and one obsolete flag that caused it to break. --- Moving `rules_scala_dependencies` to `scala/deps.bzl` ensures we get all the versions of dependencies we want in `WORKSPACE`, while providing a new API to consumers. It also prevents `WORKSPACE` from transitively loading any `.bzl` files that load `@rules_java`, ensuring Bazel 8 compatibility per #1652. Reasons for the other `rules_java` related changes include: - The `WORKSPACE` stanza for `rules_java` should've already been present while using the existing version 7.9.0. However, doing so would've broken Bazel 6 builds, as described in detail below. - The `rules_java_toolchains()` call follows the `protobuf_deps()` call instead of immediately following `rules_java_dependencies()` because upgrading to `rules_java` >= 8.5.0 will require this. It has no adverse impact to do it now, amidst the other `WORKSPACE` changes, and will make the eventual `rules_java` >= 8.5.0 diff smaller. - Having the other `WORKSPACE` setup macros for dependencies come before the `rules_scala` setup macros helps ensure consistent, correct initialization before `rules_scala` initialization. - Updating the toolchain specifiers to use `@rules_java//toolchains` fixed `WORKSPACE` build breakages when updating to `rules_java` 7.10.0 and later. This is a potentially breaking change for consumers, but in the good kind of way, in that it requires an easy, futureproof update. (`@bazel_tools//tools/jdk:toolchain_type` dependencies remain, as there's not yet a corresponding `@rules_java//toolchains` target.) What follows are detailed notes on why some dependency versions are capped where they are, and on some breakages fixed by these changes. --- `abseil-cpp` and `protobuf` have to stay at 20220623.1 and v21.7, respectively, for Bazel 6 compatibility per #1647. `protobuf` up to v25.5 is compatible with Bazel 6 provided users set the compiler flags mentioned in that issue: ```txt build:linux --cxxopt=-std=c++17 build:linux --host_cxxopt=-std=c++17 build:macos --cxxopt=-std=c++17 build:macos --host_cxxopt=-std=c++17 build:windows --cxxopt=/std=c++17 build:windows --host_cxxopt=/std=c++17 ``` --- `rules_python` 0.38.0 => 0.39.0 requires at least `rules_cc` 0.0.10, which introduced `cc/common/cc_info.bzl`: ```txt $ bazel build //{src,test,third_party,scala_proto}/... [ ...snip... ] ERROR: error loading package under directory 'test': error loading package 'test': at .../external/rules_python/python/defs.bzl:17:6: at .../external/rules_python/python/py_binary.bzl:18:6: at .../external/rules_python/python/private/py_binary_macro.bzl:16:6: at .../external/rules_python/python/private/common_bazel.bzl:19:6: Label '@rules_cc//cc/common:cc_info.bzl' is invalid because 'cc/common' is not a package; perhaps you meant to put the colon here: '@rules_cc//cc:common/cc_info.bzl'? ``` However, `rules_cc` 0.0.9 => 0.0.10 requires Bazel 7, which defines `CcSharedLibraryHintInfo`: ```txt $ bazel build //{src,test,third_party,scala_proto}/... ERROR: .../external/rules_cc/cc/private/rules_impl/native.bzl:40:33: name 'CcSharedLibraryHintInfo' is not defined (did you mean 'CcSharedLibraryInfo'?) [ ...snip... ] ERROR: error loading package under directory 'test': error loading package 'test': at .../external/rules_python/python/defs.bzl:17:6: at .../external/rules_python/python/py_binary.bzl:18:6: at .../external/rules_python/python/private/py_binary_macro.bzl:16:6: at .../external/rules_python/python/private/common_bazel.bzl:18:6: at .../external/rules_cc/cc/common/cc_common.bzl:17:6: compilation of module 'cc/private/rules_impl/native.bzl' failed ``` --- This section provides a methodical description of the errors encountered and resolved during each step of the `rules_java` update. The proper `WORKSPACE` setup stanza for `rules_java`, which we didn't originally have in place, is: ```py load( "@rules_java//java:repositories.bzl", "rules_java_dependencies", "rules_java_toolchains", ) rules_java_dependencies() rules_java_toolchains() ``` Adding this stanza to `WORKSPACE` while still using `rules_java` 7.9.0 didn't break the Bazel 7.4.1 build, but it broke the Bazel 6.5.0 build. Note the `CompiledWithJava{8,11}_java.jar` references: ```txt $ bazel build //{src,test,third_party,scala_proto}/... [ ...snip... ] ERROR: .../test/src/main/resources/java_sources/BUILD:5:14: Building test/src/main/resources/java_sources/CompiledWithJava8_java.jar (1 source file) failed: Worker process did not return a WorkResponse: ---8<---8<--- Start of log, file at .../bazel-workers/multiplex-worker-18-Javac.log ---8<---8<--- Error thrown by worker thread, shutting down worker. java.lang.NoSuchMethodError: 'java.lang.Iterable com.google.devtools.build.buildjar.javac.BlazeJavacMain$ClassloaderMaskingFileManager.getJavaFileObjectsFromPaths(java.util.Collection)' at com.google.devtools.build.buildjar.javac.BlazeJavacMain.compile(BlazeJavacMain.java:142) [ ...snip... ] ---8<---8<--- End of log ---8<---8<--- ERROR: .../test/src/main/resources/java_sources/BUILD:11:14: Building test/src/main/resources/java_sources/CompiledWithJava11_java.jar (1 source file) failed: Worker process did not return a WorkResponse: ---8<---8<--- Start of log, file at .../bazel-workers/multiplex-worker-18-Javac.log ---8<---8<--- Error thrown by worker thread, shutting down worker. java.lang.NoSuchMethodError: 'java.lang.Iterable com.google.devtools.build.buildjar.javac.BlazeJavacMain$ClassloaderMaskingFileManager.getJavaFileObjectsFromPaths(java.util.Collection)' at com.google.devtools.build.buildjar.javac.BlazeJavacMain.compile(BlazeJavacMain.java:142) [ ...snip... ] ---8<---8<--- End of log ---8<---8<--- ``` Another variation of this I also saw was: ```txt $ bazel build //{src,jmh,test,third_party,scala_proto}/... [ ...snip... ] ERROR: test/src/main/resources/java_sources/BUILD:11:14: Building test/src/main/resources/java_sources/CompiledWithJava11_java.jar (1 source file) failed: Worker process did not return a WorkResponse: ---8<---8<--- Start of log, file at .../bazel-workers/multiplex-worker-33-Javac.log ---8<---8<--- Error thrown by worker thread, shutting down worker. java.lang.UnsupportedClassVersionError: com/google/errorprone/ErrorProneError has been compiled by a more recent version of the Java Runtime (class file version 61.0), this version of the Java Runtime only recognizes class file versions up to 55.0 at java.base/java.lang.ClassLoader.defineClass1(Native Method) [ ...snip... ] ERROR: test/src/main/resources/java_sources/BUILD:5:14: Building test/src/main/resources/java_sources/CompiledWithJava8_java.jar (1 source file) failed: Worker process did not return a WorkResponse: ---8<---8<--- Start of log, file at .../bazel-workers/multiplex-worker-33-Javac.log ---8<---8<--- Error thrown by worker thread, shutting down worker. java.lang.UnsupportedClassVersionError: com/google/errorprone/ErrorProneError has been compiled by a more recent version of the Java Runtime (class file version 61.0), this version of the Java Runtime only recognizes class file versions up to 55.0 at java.base/java.lang.ClassLoader.defineClass1(Native Method) [ ...snip... ] ``` Updating the toolchains from `@bazel_tools//tools/jdk:` to `@rules_java//toolchains:` for targets in `test/BUILD` and `test/src/main/resources/java_sources/BUILD` fixed this particular issue. (More on updating other `@bazel_tools//tools/jdk` toolchain targets below.) Bazel 6.5.0 then failed with the following expected issue, fixed by `rules_java` 7.10.0 via: - bazelbuild/rules_java#210 - bazelbuild/rules_java@30ecf3f ```txt $ bazel build //{src,test,third_party,scala_proto}/... ERROR: .../external/rules_java/toolchains/java_toolchain_alias.bzl:83:34: Use of Starlark transition without allowlist attribute '_allowlist_function_transition'. See Starlark transitions documentation for details and usage: @rules_java//toolchains:java_toolchain_alias.bzl NORMAL ERROR: .../src/java/io/bazel/rulesscala/coverage/instrumenter/BUILD:3:12: While resolving toolchains for target //src/java/io/bazel/rulesscala/coverage/instrumenter:instrumenter: invalid registered toolchain '//toolchains:all': while parsing '//toolchains:all': error loading package '@rules_java//toolchains': initialization of module 'toolchains/java_toolchain_alias.bzl' failed ERROR: Analysis of target '//src/java/io/bazel/rulesscala/coverage/instrumenter:instrumenter' failed; build aborted: ``` Updating to `rules_java` 7.10.0 fixed the Bazel 6.5.0 build, but caused Bazel 7.4.1 to fail with the error I originally described in #1619. For details, see "Bump to rules_java 7.9.0 for Bazel 7 compatibility" in the message for commit cd22d88. ```txt $ bazel build //{src,test,third_party,scala_proto}/... ERROR: .../external/rules_java_builtin/toolchains/BUILD:254:14: While resolving toolchains for target @@rules_java_builtin//toolchains:platformclasspath (096dcc8): No matching toolchains found for types @@bazel_tools//tools/jdk:bootstrap_runtime_toolchain_type. To debug, rerun with --toolchain_resolution_debug='@@bazel_tools//tools/jdk:bootstrap_runtime_toolchain_type' If platforms or toolchains are a new concept for you, we'd encourage reading https://bazel.build/concepts/platforms-intro. ERROR: Analysis of target '//test/toolchains:java21_toolchain' failed; build aborted: Analysis failed ``` My analysis from #1619 _was_ on the right track. The `rules_java` 7.9.0 built into Bazel's `WORKSPACE` preamble registered `@bazel_tools//tools/jdk:bootstrap_runtime_toolchain_type` toolchains that later versions of `rules_java` couldn't resolve. This was due to the very same change that fixed 6.5.0, bazelbuild/rules_java#210. However, I mistakenly thought we needed to keep `@bazel_tools//tools/jdk:` as the canonical Java toolchains package instead switching to `@rules_java//toolchains:`. That's why I originally thought we were stuck on `rules_java` 7.9.0. I eventually rediscovered the following issue, which surfaced this problem a month before #1619. The conversation helped me realize that switching to `@rules_java//toolchains:` actually is the preferred, futureproof approach (short of migrating to Bzlmod): - bazelbuild/rules_java: Regression with `@@rules_java//toolchains:bootstrap_runtime_toolchain_type` in 7.10.0 bazelbuild/rules_java#214 Switching all `@bazel_tools//tools/jdk:` toolchains to `@rules_java//toolchains:` resolved this issue, enabling Bazel 6.5.0 and 7.4.1 to successfully build using `rules_java` 7.12.2. I then updated `rules_java` to 7.12.3, which returns to registering some toolchains as `@bazel_tools//tools/jdk:bootstrap_runtime_toolchain_type`: - bazelbuild/rules_java#246 - bazelbuild/rules_java@b64eb7d @hvadehra requested that I try 7.12.3 to see if it resolved the issue. I was able to build successfully using this version in a branch without the toolchain updates from this commit. - #1652 (comment) However, the changes from this commit should remain futureproof when `@bazel_tools//tools/jdk:bootstrap_runtime_toolchain_type` moves to `@rules_java//toolchains:bootstrap_runtime_toolchain_type` again one day. --- Several of the flags removed from the `scrooge_compile_with_jdk_11` test case have been no-ops for years, so removing them eliminated their corresponding deprecation warnings. `--javacopt='--release 11'`, however, caused this breakage after originally bumping to `rules_java` 7.12.2: ```txt $ RULES_SCALA_TEST_ONLY="scrooge_compile_with_jdk_11" bash ./test/shell/test_twitter_scrooge.sh running test scrooge_compile_with_jdk_11 WARNING: Option 'javabase' is deprecated WARNING: Option 'host_javabase' is deprecated WARNING: Option 'host_java_toolchain' is deprecated WARNING: Option 'java_toolchain' is deprecated WARNING: Option 'javabase' is deprecated WARNING: Option 'host_javabase' is deprecated WARNING: Option 'host_java_toolchain' is deprecated WARNING: Option 'java_toolchain' is deprecated INFO: Analyzed 64 targets (0 packages loaded, 0 targets configured). INFO: Found 64 targets... ERROR: .../src/java/io/bazel/rulesscala/scalac/compileoptions/BUILD:3:13: Compiling Java headers src/java/io/bazel/rulesscala/scalac/compileoptions/libcompileoptions-hjar.jar (1 source file) failed: (Exit 1): turbine_direct_graal failed: error executing command (from target //src/java/io/bazel/rulesscala/scalac/compileoptions:compileoptions) external/remote_java_tools_darwin_arm64/java_tools/turbine_direct_graal --output bazel-out/darwin_arm64-fastbuild/bin/src/java/io/bazel/rulesscala/scalac/compileoptions/libcompileoptions-hjar.jar ... (remaining 32 arguments skipped) Use --sandbox_debug to see verbose messages from the sandbox and retain the sandbox build root for debugging java.lang.NullPointerException: attempted to use --release, but JAVA_HOME is not set at [email protected]/java.util.Objects.requireNonNull(Objects.java:259) at com.google.turbine.binder.CtSymClassBinder.bind(CtSymClassBinder.java:55) at com.google.turbine.main.Main.release(Main.java:318) at com.google.turbine.main.Main.bootclasspath(Main.java:304) at com.google.turbine.main.Main.compile(Main.java:142) at com.google.turbine.main.Main.compile(Main.java:133) at com.google.turbine.main.Main.main(Main.java:89) at [email protected]/java.lang.invoke.LambdaForm$DMH/sa346b79c.invokeStaticInit(LambdaForm$DMH) INFO: Elapsed time: 0.325s, Critical Path: 0.12s INFO: 11 processes: 10 internal, 1 worker. FAILED: Build did NOT complete successfully Test "scrooge_compile_with_jdk_11" failed (0 sec) ``` See also: - Bazel Blog: Bazel 5.0: https://blog.bazel.build/2022/01/19/bazel-5.0.html#java - bazelbuild/bazel: `incompatible_use_toolchain_resolution_for_java_rules`: use toolchain resolution for Java rules #7849 bazelbuild/bazel#7849 --- Bazel 8 requires `rules_java` 8, per #1652. Updating to `rules_java` 8 will likely happen in a new major release _after_ a release containing the Bzlmod changes. This is because: - The Bzlmod changes alone will work with Bazel 6 without requiring that users update `protobuf` beyond version 21.7. - Bazel 8 requires `protobuf` >= 29.0. Bazel 6 users would have to add the afforementioned C++ compiler flags to support the newer `abseil-cpp` versions required by newer `protobuf` versions. - `rules_scala` itself needs to update ScalaPB to 1.0.0-alpha.1 or higher to support `protobuf` >= v28.0, but this ScalaPB version will _not_ support `protobuf` < v28.0. The idea is that the next major release of `rules_scala` will help users migrate to Bazel 7 and Bzlmod, in either order. Then the next major release after that will enable them to migrate to Bazel 8, with all the requisite dependency upgrades. (Technically it will still support `WORKSPACE`, but hopefully they'll make the jump to Bzlmod first.) See: - #1482 (comment) --- Here are the details explaining how `rules_java` 8 currently breaks `rules_scala`, up until the point that upgrading to `protobuf` >= v29.0 would fix it. `rules_java` 8.0.0 requires Bazel >= 7.3.2, which provides the `subrule` API. Compatibility with Bazel >= 6.3.0 isn't restored until `rules_java` 8.3.2. ```txt $ bazel build //{src,test,third_party,scala_proto}/... ERROR: .../external/rules_java/java/common/rules/android_lint.bzl:142:24: name 'subrule' is not defined (did you mean 'rule'?) ERROR: Error computing the main repository mapping: at .../scala/private/extensions/dev_deps.bzl:8:6: at .../external/rules_java/java/repositories.bzl:20:6: at .../external/rules_java/toolchains/local_java_repository.bzl:17:6: at .../external/rules_java/java/defs.bzl:18:6: at .../external/rules_java/java/java_library.bzl:16:6: at .../external/rules_java/java/bazel/rules/bazel_java_library.bzl:21:6: compilation of module 'java/common/rules/android_lint.bzl' failed ``` `rules_java` 8.3.0 is broken, as it can't find its own `@compatibility_proxy` repo: ```txt $ bazel build //{src,test,third_party,scala_proto}/... ERROR: error loading package under directory 'src': error loading package 'src/protobuf/io/bazel/rules_scala': at .../external/rules_java/java/defs.bzl:22:6: at .../external/rules_java/java/java_test.bzl:16:6: Unable to find package for @compatibility_proxy//:proxy.bzl: The repository '@compatibility_proxy' could not be resolved: Repository '@compatibility_proxy' is not defined. ``` `rules_java` 8.3.1 seems to fix this, presumably by importing the `protobuf` repo as `com_google_protobuf`. However, it now requires at least `protobuf` v27.0, which adds `bazel/java/lite_proto_library.bzl`. Per #1647, we'd have to update to ScalaPB 1.0.0-alpha.1 to support `protobuf` v28, abandoning users of previous `protobuf` versions or forcing them to upgrade. ```txt $ bazel build //{src,test,third_party,scala_proto}/... [...snip...] ERROR: error loading package under directory 'src': error loading package 'src/java/io/bazel/rulesscala/worker': at .../external/rules_java/java/defs.bzl:16:6: Label '@com_google_protobuf//bazel:java_lite_proto_library.bzl' is invalid because 'bazel' is not a package; perhaps you meant to put the colon here: '@com_google_protobuf//:bazel/java_lite_proto_library.bzl'? ``` * Bump to rules_java 7.12.4 and rules_go 0.51.0 Tested with both Bazel 6.5.0 and 7.4.1. - rules_go: 0.50.1 => 0.51.0 - rules_java: 7.12.3 => 7.12.4 * Bump to rules_go 0.52.0 Tested with both Bazel 6.5.0 and 7.4.1. - rules_go: 0.51.0 => 0.52.0
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
When updating to 7.10.0 (with Bazel 7.3.1), we are seeing a regression:
Running with
--toolchain_resolution_debug='@@rules_java//toolchains:bootstrap_runtime_toolchain_type':The text was updated successfully, but these errors were encountered: