Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

🚀 Introduce local remote execution toolchains #83

Merged
merged 1 commit into from
Apr 15, 2023

Conversation

aaronmondal
Copy link
Contributor

This commit configures rules_cc and rules_java in a way that is compatible with both remote execution and local execution from within the nix environment.

  • Add a container image compatible with remote execution
  • Add a nix build file for the bazel-toolchains repository which includes the rbe_configs_gen tool
  • Add convenience wrappers for invoking the rbe_configs_gen tool
  • Add generated toolchains
  • Update the .bazelrc files to accomodate for the changed toolchains
  • Make the bazel-wrapper compatible with the new toolchains

@aaronmondal
Copy link
Contributor Author

Now tested against multiple machines all building locally and streaming against the same cache. This actually works - we can use multiple hosts at the same time and they all build to and from the same cache.

@aaronmondal aaronmondal force-pushed the add-bazel-toolchains branch from e8bc875 to 298462f Compare April 15, 2023 14:26
@aaronmondal
Copy link
Contributor Author

Updated container hash. I just forgot to rerun rbegen before committing. The new hash is the one we got when building on three independent systems (WSL2/Ubuntu, Arch, Gentoo).

This commit configures rules_cc and rules_java in a way that is
compatible with both remote execution and local execution from within
the nix environment.

- Add a container image compatible with remote execution
- Add a nix build file for the bazel-toolchains repository which
  includes the rbe_configs_gen tool
- Add convenience wrappers for invoking the rbe_configs_gen tool
- Add generated toolchains
- Update the .bazelrc files to accomodate for the changed toolchains
- Make the bazel-wrapper compatible with the new toolchains
@aaronmondal aaronmondal force-pushed the add-bazel-toolchains branch from 298462f to 1e4aa06 Compare April 15, 2023 17:43
@aaronmondal
Copy link
Contributor Author

aaronmondal commented Apr 15, 2023

  • Updated to proper bzlmod toolchain resolution
  • Removed ll-specific platforms and constrains that are no longer needed.
  • Added remote optimizations. This brings the critical path for a bazel test cpp down to ~5 seconds from a clean cache.

@aaronmondal
Copy link
Contributor Author

Ok tested this again on WSL2/Ubuntu and Gentoo. Seems to work consistently and reproducibly. Should be fine to land.

@aaronmondal aaronmondal merged commit 75825f0 into eomii:main Apr 15, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant