Skip to content

Home

Jump to bottom
Alex Eagle edited this page Feb 7, 2020 · 78 revisions

Migrating to @bazel/bazelisk package

To use Bazel, it obviously needs to be downloaded to your computer. It also needs to be the right version to be compatible with the code you're building. This is the same problem we have with Node, which you need to fetch and correctly version. The community-accepted solution for this is to use nvm and a .nvmrc file. .nvmrc says what version(s) of node work with the project, and nvm downloads a node.js version to your machine if needed.

Bazel has a nearly identical solution. The @bazel/bazelisk package is a Bazel wrapper that downloads the correct version of Bazel based on your .bazelversion file.

Now that this is available, we plan to deprecate our old mechanism for downloading and versioning Bazel in npm-based projects, the @bazel/bazel package.

What's wrong with @bazel/bazel

The bazel binary is pretty big (compared with typical JS tooling) at 40MB. But it's a native binary, so we actually had three optional dependencies from the @bazel/bazel package to platform-specific ones. This is a nice solution, except that we found that in practice, both yarn and npm have behavior (bug?) where they download the tarballs for all three packages even though it was known beforehand that two of them are not compatible with the local platform. So you end up with 120MB download. We didn't succeed in escalating this issue with the package managers.

Second, by having Bazel versioned in the package.json file, we were not compatible with the rest of Bazel environment. For example, if you downloaded Bazel outside of the npm mirror, you wouldn't get any warning that you have the wrong version. Bazel itself knows to look for the .bazelversion file so we want that to work for JS developers too.

Next, we had a governance issue. rules_nodejs maintainers were mirroring Bazel to npm, but we didn't always mirror all the versions (like release candidates). The release process should be hosted entirely by the tool itself so every version is available and you can rely on the release engineer doing all the steps. Since Bazelisk now releases to npm (thanks @philwo!), we get out of the mirroring business and you can use any version of Bazel.

Finally, bazelisk has better support as a Bazel wrapper than the npm mirror had. It knows to execute the tools/bazel wrapper if it exists, and can be hooked up with command-line completion. It even has features to help you migrate past Bazel breaking changes (akin to the ng update command in Angular ecosystem)

How to migrate

  1. Remove your dependency on @bazel/bazel which is deprecated
  2. Add a devDependency on @bazel/bazelisk instead. The version doesn't matter much since it's just a wrapper. (you could even use a global install of bazelisk if you'd rather have the tool on your $PATH)
  3. Add a .bazelversion file in your workspace root containing the version you want to pin to. See the bazelisk readme
  4. In your package.json#scripts, call bazelisk rather than bazel
  5. @bazel/bazel had a transitive dependency on a workaround package, @bazel/hide-build-files, whose purpose was to prevent Bazel from seeing the BUILD files under external repositories. You can either
    1. update Bazel to >= 2.1 and rules_nodejs >= 1.3.0 since the workaround is no longer needed, or
    2. add a devDependency on @bazel/hide-build-files to keep the workaround in place

If you had been relying on a bazel command in your $PATH, you can still do this by aliasing it to the bazelisk command. You can also add a "bazel": "bazelisk" alias in the package.json#scripts.

Note, Bazelisk is adding more robust handling of network constraints. Check this issue if you need to download via a corporate proxy: https://github.com/bazelbuild/bazelisk/issues/115.

Migrating off @build_bazel_rules_nodejs//:defs.bzl

For consistency, rules_nodejs always loads from a file called index.bzl since this matches the node ecosystem, and Bazel itself doesn't have a single standard (defs.bzl, build_defs.bzl and others are common)

If you depend on another ruleset that still depends on defs.bzl, you must update. For rules_sass, the minimum is:

http_archive(
    name = "io_bazel_rules_sass",
    sha256 = "617e444f47a1f3e25eb1b6f8e88a2451d54a2afdc7c50518861d9f706fc8baaa",
    urls = [
        "https://github.com/bazelbuild/rules_sass/archive/1.23.7.zip",
        "https://mirror.bazel.build/github.com/bazelbuild/rules_sass/archive/1.23.7.zip",
    ],
    strip_prefix = "rules_sass-1.23.7",
)

For rules_docker, the minimum release is https://github.com/bazelbuild/rules_docker/releases/tag/v0.13.0

If you have trouble updating to the latest rules_docker, you can also apply a simple local patch to the version you're currently on. For example, within rules_nodejs we added one "patches" attribute to our rules_docker fetch:

http_archive(
    name = "io_bazel_rules_docker",
    patches = ["//:rules_docker.patch"], # <-- add this to apply a patch
    sha256 = "7d453450e1eb70e238eea6b31f4115607ec1200e91afea01c25f9804f37e39c8",
    strip_prefix = "rules_docker-0.10.0",
    urls = ["https://github.com/bazelbuild/rules_docker/archive/v0.10.0.tar.gz"],
)

and add a file rules_docker.patch next to WORKSPACE like this:

diff --git nodejs/image.bzl nodejs/image.bzl
index a01ea3e..617aa06 100644
--- nodejs/image.bzl
+++ nodejs/image.bzl
@@ -17,7 +17,7 @@ The signature of this rule is compatible with nodejs_binary.
 """
 
 load("@bazel_skylib//lib:dicts.bzl", "dicts")
-load("@build_bazel_rules_nodejs//:defs.bzl", "nodejs_binary")
+load("@build_bazel_rules_nodejs//:index.bzl", "nodejs_binary")
 load(
     "//container:container.bzl",
     "container_pull",

Migrating off //internal/rollup

The rollup_bundle rule from the //internal package has a bunch of problems, and has been replaced by a new implementation. The new implementation is much simpler, does just one thing, and is meant to run nearly identically to how you would use Rollup outside of Bazel. The //internal/rollup package will be removed before 1.0. Some work is required to move to the new rule.

High-level notes:

  • Terser minification was run in the old rule when you requested a .min.js output. rollup_bundle and terser_minified are now separate rules.
  • New rules use a peerDependency, so you need to install rollup/terser if you haven't already
  • We don't have a downleveling step built-in, so if you need es5 output you can use a rollup plugin to do it. Note that this means differential loading scenarios need two rollup builds, which is slow - we'll have a better way to downlevel rollup outputs before we remove the old rollup_bundle rule.
  • Works with ng_module with a minimum @angular/bazel version of 9.0.0-next.11.

Rough steps to follow:

  • Replace load("@build_bazel_rules_nodejs//:defs.bzl", "rollup_bundle") with load("@npm_bazel_rollup//:index.bzl", "rollup_bundle")
  • If you reference a min.js output, also add load("@npm_bazel_terser//:index.bzl", "terser_minified")
  • Generally, you can get the same .min.js output as before just by adding terser_minified after the existing rollup_bundle: 
    # this will output bundle.min.js, but not a predeclared label
terser_minified(
    name = "bundle.min",
    # assuming your rollup_bundle rule has name="bundle"
    src = "bundle.js",
)
  • The new terser rule cannot provide a label for bundle.min.js (because it doesn't know if the input is a directory until after the predeclared outputs are determined), so you can't reference that in your other targets. You'll have to use bundle.min. If a downstream target expects a label which produces a single file, you can disable the sourcemap feature in terser_minified so that the .js output is the only one.
  • If you used enable_code_splitting = False, you can just remove it. The new rule only produces chunked output when given a output_dir = True attribute.
  • additional_entry_points are now combined with the primary entry_point in a single entry_points attribute. You can give TypeScript files as the entry point, like index.ts and the rule will resolve that to the .js output file.

Thanks to @jbedard for the design doc that motivated the changes.

Migrating to rules_nodejs 0.30

These are detailed migration notes for https://github.com/bazelbuild/rules_nodejs/releases/tag/0.30.0

In prior releases, Bazel installed a copy of the node_modules tree in its private external directory. This meant that you typically had to install dependencies twice, and if you wanted to do some println debugging in the node_modules directory, you had to know where this other copy is installed.

As of Bazel 0.26, there is a new feature called "managed directories" which allows Bazel to reference the same node_modules directory that your package manager creates and that the editor uses to resolve tools like TypeScript. This feature is now enabled by default.

You must update your Bazel dependency and/or your locally installed Bazel to 0.26 or greater before updating rules_nodejs to 0.30.

To use managed directories, make these changes to your app:

  • if you are using the @bazel/bazel npm package, update it to "@bazel/bazel": "0.26.0-rc13" or newer
  • update WORKSPACE to rules_nodejs 0.30.0 release
  • if you are using the @bazel/typescript npm package this needs to be updated to 0.30.0 as well
  • tell Bazel to opt-in to this feature by adding to .bazelrc: 
    build --experimental_allow_incremental_repository_updates
query --experimental_allow_incremental_repository_updates
  • make sure you don’t have multiple yarn_install/npm_install rules referencing the same package.json
  • add to WORKSPACE the managed_directories of all your yarn_install/npm_install rules. This typically looks like: 
    workspace(
  name = "my_wksp",
  managed_directories = {"@npm": ["node_modules"]}
)
  • add the same node_modules directories as lines in .bazelignore - these contain BUILD files that only work when Bazel references them externally

You can turn managed directories off for individual yarn_install or npm_install rules with the symlink_node_modules = False attribute. If you you have multiple yarn_install and/or npm_install rules, any that have symlink_node_modules = False do not need their node_modules folders be added to managed_directories in your WORKSPACE or in .bazelignore. If you add them anyway it won't break anything.

Opting out

If the managed_directories feature breaks you, please file an issue on this repo to let us know. Then you can proceed to 0.30 release without the feature by adding symlink_node_modules = False to any affected npm_install/yarn_install rules.

/node_modules/.bin/BUILD file no longer generated

Due to an issue on Windows with managed directories and the node_modules/.bin folder, no BUILD file is generated under node_mdoules/.bin. If you were relying on targets in this BUILD file prior to 0.30 this will be an additional breaking change for you. You can still reference files in .bin with labels such as @npm//:node_modules/.bin/foobar or if you require a filegroup group with a multiple files in the .bin folder you can use manual_build_file_contents of your yarn_install or npm_install rule to add the appropriate filegroup to the root @npm build file .

For example,

yarn_install(
  name = "npm",
  package_json = "//:package.json",
  yarn_lock = "//:yarn.lock",
  manual_build_file_contents = """
filegroup(
  name = "bin_files",
  srcs = glob(["node_modules/.bin/*"]),
)"""
)

would create target @npm//:bin_files which would contain all files in the node_modules/.bin folder. If this doesn't work for you please file an issue and we'll help you find an appropriate fix.

Migrating to rules_nodejs 0.13

We are making a breaking change for version 0.13, related to loading npm dependencies.

nodejs_binary and other rules that depend on it typically accept a node_modules attribute. This attribute was optional and defaulted to @//:node_modules, which means "find the node_modules target in the root of the workspace where the build is happening."

However, we now are moving to a model where npm dependencies can be expressed as deps (or data) of rules. That means that a nodejs_binary that requests specific npm modules as dependencies wants to get no additional modules through the node_modules attribute.

To make this possible, node_modules will now default to an empty filegroup. To unbreak your code, you can typically just set node_modules = "@//:node_modules" - that is, inline the previous default into an explicit attribute.

As of 0.13.2, this also applies to rollup_bundle which has a node_modules attribute that now defaults to an empty filegroup.

Bazel version 0.17.1

Bazel version 0.17.1 is required as of version 0.13.0 in order to support fined grained npm dependency labels. Bazel 0.17.1 adds support for @ in label names which is required for labels such as @npm//:@types/jasmine.

jasmine_node_test now executes only *spec.js and *test.js files

We used to take all transitive JS files and hand them to Jasmine as specs. This causes a problem when some files have global side-effects (e.g. calling process.exit) We now filter only files ending with spec.js or test.js and ask Jasmine to execute those.

Clone this wiki locally