Upgrading a monorepo from Yarn v1 with Lerna.js to Yarn v2 makes use of the weirdest versioning system I ever saw, and Yarn’s migration guide is not the most helpful: although it purports to guide you through the process in a single document, the information you need is actually spread across multiple pages in quite a confusing way, not helped by the poor English throughout. There is also an introductory blogpost which you may find helpful.
This document is intended broadly as a replacement for the official documentation (except where indicated), to help others get through the process faster than I did. It won’t cover every feature of Yarn v2, just enough for you to complete the migration and be confident it was worthwhile.
Overall I would say that the Yarn vision is pretty impressive:
- zero installs are very cool,
- enforcing strict dependency specification gives us confidence we never had
with
node_modules, - delegating dependency resolution to the package manager makes a lot of sense.
However, it’s an open source project with limited resources, hence the difficult documentation and slightly unstable feel. Moreover the ecosystem often does not live up to the high standards required to reap all the benefits (eg packages not listing all their dependencies).
For example: npm install --global yarn or brew upgrade yarn.
Navigate inside the project and run yarn set version berry.
From the docs:
"Berry" is the codename for the Yarn 2 release line. It's also the name of our repository!
This will create a .yarn/ directory with a yarn-berry.cjs release inside and
a .yarnrc.yml configuration file (a new format compared to the .yarnrc that
was previously used).
Now run yarn set version latest to get a specific berry version. (Why doesn’t
this happen when you set the version to berry? Good question!) You should see
the version number reflected in the .yarn/releases/yarn-X.Y.Z.cjs filename and
the yarnPath option in the .yarnrc.yml file.
Now if you run yarn --version in this repository it should tell you you’re
using v2. Note that it’s the same binary as v1 though! Ie:
$ which yarn && yarn --version
/usr/local/bin/yarn
2.4.1
$ (cd .. && which yarn && yarn --version)
/usr/local/bin/yarn
1.22.10Weird, right? 🤷
You might not have had either, but any configuration in a .npmrc or .yarnrc
file will need to be transferred to the newly created .yarnrc.yml file.
See
Update your configuration to the new settings
and Configuration. Note that these
options might secretly relate to plugins which you will need to install (for
example I added changesetBaseRefs and was informed it wasn’t a valid
configuration option until I added the version plugin).
One of the core concepts in Yarn v2 is the
Zero Install, ie packages that
don’t need you to run yarn install after you’ve downloaded them. Powering this
is Yarn v2’s Plug‘n’Play (or "PnP") installation strategy/module resolution
system. We’ll come back to this later, but at this point in the migration we
want to specify the traditional node_modules installation strategy and Node’s
own module resolution strategy, because that’s what Yarn v1 uses:
nodeLinker: "node-modules"For example:
# For more info see https://yarnpkg.com/advanced/telemetry
enableTelemetry: false
# See section on specifying missing dependencies below
preferInteractive: trueRefer to the configuration documentation (with the same caveat as above).
Run yarn install and Yarn will sort it all out. It will probably give you a
bunch of warnings. Note that yarn.lock is now valid YAML, unlike with Yarn v1.
Add all of this to your .gitignore (or whatever):
.yarn/*
!.yarn/patches
!.yarn/releases
!.yarn/plugins
!.yarn/sdks
!.yarn/versions
.pnp.*
Everything else that’s new or changed should be committed. We’ll come back to this ignore list when we revisit Plug‘n’Play.
If are using the Yarn CLI tool in any scripts you will need to update some
command and option names which have changed from v1 to v2. For example
--frozen-lockfile has been deprecated in favour of --immutable. Other option
changes weren’t covered in the official documentation, so have fun finding out
which ones still work. A table of changes to commands can be found
here.
Adding a plugin updates the .yarnrc.yml file and adds a plugin to
.yarn/plugins. Here are some helpful ones, but there are more and you can even
write your own.
yarn plugin import workspace-toolsMost importantly this will let you use the yarn workspaces foreach command.
See the docs for more information.
NOTE: The root project is a valid workspace, which means if you want to run
a script using foreach in all workspaces except the root, other than running
a script defined in the root package.json, you have to filter it out.
NOTE: You probably want to use the --parallel, --topological-dev and
--verbose options in most cases.
yarn plugin import versionThis lets you do version things. More info: https://yarnpkg.com/cli/version
yarn plugin import interactive-toolsMost useful for yarn upgrade-interactive: see
https://yarnpkg.com/cli/upgrade-interactive
Automatically installs DefinitelyTyped @types/* definitions if the project
doesn’t have its own. This sounds useful, but might actually be annoying because
sometimes you don’t need the type definitions even when they’re available (eg
for tools). I installed it initially but ended up removing it.
yarn plugin import typescriptMore info: https://github.com/yarnpkg/berry/tree/master/packages/plugin-typescript
Yarn v2 wants to enforce dependencies strictly. You may be used to listing
development dependencies in the root of the project and then being able to use
the executables they define in workspaces. But Yarn v2 won’t let you, so if you
are using any of these executables in your package.json scripts you’ll need to
add the relevant development dependency to the workspace. This doesn’t mean that
it will install duplicates, as long as you tell it not to via
yarn add --interactive. (You can also set preferInteractive in the
.yarnrc.yml file.)
Note that the dependency resolution appears to be inconsistent depending on
whether you are using the workspace-tools plugin, and how:
yarn run <executable>within workspace -> dependency must be specified in workspaceyarn workspaces foreach run <executable>in worktree -> OK if dependency is not specified in workspace as long as it is in the worktreeyarn workspaces foreach --include 'my-pattern/*' run <executable>in worktree -> dependency must be specified in relevant workspacesyarn workspaces foreach --include '*' run <executable>in worktree -> OK if dependency is not specified in workspace as long as it is in the worktreeyarn workspaces foreach --exclude my-pattern/* run <executable>in worktree -> OK if dependency is not specified in workspace as long as it is in the worktree
(This might depend on which linker you’re using.) See https://yarnpkg.com/features/workspaces#what-does-it-mean-to-be-a-workspace for more information on workspaces.
In packages which depend on other packages in the same monorepo, you can specify
these dependencies using the workspace:packages/my-other-package format. This
way you don’t need to constantly update the versions manually, it’ll all be
calculated for you at publication time.
More info: https://yarnpkg.com/features/workspaces#workspace-ranges-workspace
Unlike npm or Yarn v1, Yarn v2 won’t run arbitrary lifecycle scripts (i.e.
prexxx and postxxx). There are
some exceptions, including
postinstall, but if you’ve been relying on any other lifecycle scripts you
will need to
explicitly call them
in the main script definition.
NOTE: Yarn recommends explicitly calling the lifecycle scripts in the main script, but I would recommend not doing that, because it’s going to get confusing if people start trying to use other package managers. It’s much simpler just to put it all in the same script definition, or outsource to a script file.
At this point you should have a working setup. Make sure you can build, test, lint, run scripts etc.
According to this blog post:
Yarn 2 ships with a rudimentary shell interpreter that knows just enough to give you 90% of the language structures typically used in the scripts field. Thanks to this interpreter, your scripts will run just the same regardless of whether they're executed on OSX or Windows.
So you probably don’t need shx or any other tool that you were using for
cross-platform shell compatibility. Then again, maybe you do still need it
because you use a language structure outside the 90%. 🤷
Read about the problems of node_modules and how Yarn aims to fix them here:
https://yarnpkg.com/features/pnp#the-node_modules-problem
The rest of this section will take you through adding PnP. This was the most painful part of the process, introducing many shiny new things that the rest of the ecosystem doesn’t seem quite ready for. Die Idee ist gut, doch die Welt noch nicht bereit. If you choose to skip this section that is a totally legitimate decision.
yarn dlx @yarnpkg/doctor@2NOTE: You need to specify the @2 because the incompatible v3 release
candidate was released as latest on npm.
By the way, this uses another new feature of Yarn v2: yarn dlx. This is like
npx, but downloads the package only temporarily and then throws it away.
This uses the default PnP setting, which is the real Yarn v2 magic.
Run yarn. This will remove the node_modules/ directory and store lots of zip
files in .yarn/cache.
Remove what you put there before and add these:
.yarn/*
!.yarn/cache
!.yarn/patches
!.yarn/plugins
!.yarn/releases
!.yarn/sdks
!.yarn/versions
Basically we’re adding the cache and the .pnp.js file to VCS. It might seems
weird to add all those zip files to version control, but the reasoning is in the
article listed above.
Before you add these to the VCS index, you might want to think about how you
want to store the large zip files. For example, if you’re using Git LFS you
could get it to handle zip files by adding the following to .gitattributes:
*.zip filter=lfs diff=lfs merge=lfs -text
Replace calls to node outside of a package.json script with yarn node. For
example in shebangs in development scripts. You can pass the -S option to make
this work like this:
#!/usr/bin/env -S yarn nodeYou probably don’t want to update executables you will ship to your users
because they may not be using Yarn, but that means you’ll have to remember to
run them using yarn node from within your project rather than just executing
them directly.
Running Node via Yarn v2 does not handle ESM as described in this issue: yarnpkg/berry#638
For a quick solution, you can run yarn add -D esm in every relevant package,
and then replace every yarn node script.js with a
yarn node --require esm script.js.
For a more general workaround, check out this repo: https://github.com/DaneTheory/yarn-pnp-with-esm
It might not affect you but there’s an issue with the fs patch and bigint,
which is solved in v3, but won’t be backported to v2:
yarnpkg/berry#2232 (comment)
Make sure you have everything you want your IDE to use installed in the root of
the project. I suggest typescript and prettier at least. Then run
yarn dlx @yarnpkg/sdksThis will set up your IDE and put a bunch of things in .yarn/sdks, which
you’ll want to add to VCS. If you’re using VSCode, it will also add some stuff
to .vscode/extensions.json and .vscode/settings.json. You may or may not
want to add these to VCS—talk to your colleagues.
https://next.yarnpkg.com/advanced/pnpify#ide-support
Lerna unfortunately appears to be unmaintained. Remove the dependency and the
lerna.json configuration file.
You can read about Yarn’s release workflow here: https://next.yarnpkg.com/features/release-workflow
E.g. to ensure that each package specifies the right fields in its
package.json. Bonus: you get to learn Prolog! (I for one did not have time.)
More info: https://yarnpkg.com/features/constraints
That’s it! There’s a lot more configuration available that has not been covered here, but hopefully you made it through all that and your code still works.
Here are some additional resources I found helpful: