pnpm v7

[zkochan]

This is a discussion about pnpm v7 breaking changes.

• The biggest breaking change will be that pnpm will be shipped as a compiled binary, so pnpm won't need Node.js preinstalled on a system. This version of pnpm is already shipped as @pnpm/beta
• The pnpx command will be removed. Instead of pnpx, pnpm may be used. Can we remove it now that Corepack ships it? Maybe just make it an alias of pnpm dlx.
• Turn off hoisting by default (at least in monorepos)
• Turn on strict-peer-dependencies by default
• Exclude the root from recursive script runner commands: Excluding the root from recursive exec (and other workspace commands) #2769
• pnpm pack should only pack a file as an executable, if it is a bin or listed in the executableFiles array
• Remove the -W option. --workspace-root/-w should be used instead.
• extend-node-path should be false by default. The setting should be removed.
• rename use-node-version to node-version.
• remove npm_config_argv Why set npm_config_argv in process.env? #4153
• make side-effects-cache true by default.
• make embed-readme false by default
• Drop Node.js 12 support (EOL will be 2022-04-30)
• pnpm install -g pkg write bin commands only to predefined location.
• When using pnpm run <script>, all command line arguments after the script name are passed to the script's argv. Run scripts without --? #3778
• The file protocol should work the same as in Yarn: it should copy the files from the package, not symlink the package.
• Install dev deps even if NODE_ENV is set to production.
• Improve how beta versions are handled
• pnpm init is non-interactive by default.
• pnpm store location should be inside the pnpm home directory

Review the breaking changes in Yarn v4 and do the same changes, when it makes sense: yarnpkg/berry#3591

Related PRs:

• feat!: breaking config changes for v7 #4253
• feat(pack)!: limit what files are packed as executables #4266
• feat!: pnpx is an alias of pnpm dlx #4267
• feat: run scripts without -- #4290
• feat!: global bins should be created only in predefined locations #4280
• ci: use pnpm v7 #4307
• feat!: remove -W option #4308
• feat: the file protocol should always inject the dependency #4408
• feat: rename use-node-version to node-version #4416
• feat: strict-peer-dependencies is true by default #4427
• feat: a prerelease version is always saved as an exact version #4435
• feat: don't resolve peer dependencies from workspace root #4469
• fix(cmd-shim): extend NODE_PATH with path to hidden hoisted dir #4513
• feat: remove deprecated settings #4519
• feat: the global store location should be inside the pnpm home directory #4522

[ibezkrovnyi]

Few thoughts:

1. Shipping pnpm as a compiled binary while we already have node binaries downloaded will require an additional 36 MB downloaded per each GitLab pipeline job. 20 jobs = 720 MBs of unnecessary downloads on each commit.
2. One of the biggest advantages of pnpm is the ability to work with simple repo as with monorepo with a single project (no matter if pnpm-workspace.yaml is present or not) . With using GitLab CI pipeline configured for pnpm monorepo commands pnpm -r --filter <FILTER> run <SCRIPT>, pnpm -r --filter <FILTER> exec <CMD> and pnpm -r --filter <FILTER> ls --depth -1 --json will work with one root-level project just fine. So, currently there is no need to have separate GitLab CI pipelines for pnpm monorepo and for simple repo. It would be nice to keep this pnpm's superpower (at least with some .npmrc option).

[zkochan]

Shipping pnpm as a compiled binary while we already have node binaries downloaded will require an additional 36 MB downloaded per each GitLab pipeline job. 20 jobs = 720 MBs of unnecessary downloads on each commit.

We will probably keep pnpm to be js but we will recommend to use the packaged version, which is faster.

Can't you create something like a base image for your pipeline with pnpm preinstalled?

One of the biggest advantages of pnpm is the ability to work with simple repo as with monorepo with a single project (no matter if pnpm-workspace.yaml is present or not) . With using GitLab CI pipeline configured for pnpm monorepo commands pnpm -r --filter run <SCRIPT>, pnpm -r --filter exec and pnpm -r --filter ls --depth -1 --json will work with one root-level project just fine. So, currently there is no need to have separate GitLab CI pipelines for pnpm monorepo and for simple repo. It would be nice to keep this pnpm's superpower (at least with some .npmrc option).

Which of the items break this workflow?

[ibezkrovnyi]

Can't you create something like a base image for your pipeline with pnpm preinstalled?

Yes, we can preinstall pnpm binary into docker image, so the size of the image will grow. Docker image is downloaded in each GitLab job, so there is no gain preinstalling pnpm.

Which of the items break this workflow?

I was under impression that this one: Exclude the root from recursive script runner commands, but after reading related PR #3647 I think the only change that will be needed to make pnpm treating monorepo and single repo, in the same way, is to add --workspace-root (-w), so this is not a problem then.

[donferi]

One question, will pnpm shipped as a compiled binary work fine with corepack?

[Jack-Works]

It also makes pnpm faster.

I'm curious how bringing your own Node will make pnpm faster. Does it use some kind of V8 memory snapshot like what Deno does to the TypeScript compiler?

[zkochan]

pkg compiles pnpm to v8 bytecode: https://github.com/vercel/pkg#bytecode-reproducibility

[Jack-Works]

Cool, does that has a significant performance improvement?

[gluxon]

From some tests on an Ubuntu image, I did notice that the prepackaged binary started in 0.25s vs 0.5s for the JS version. That seems small, but saving a quarter second on every pnpm invocation seems significant to me.

[zkochan]

I think it makes startup faster by 0.33 seconds on my machine.

[zkochan]

I started to work on this @pnpm/collaborators

[PabloSzx]

Drop Node.js 12 support

Is this change intended to align with Node.js v12 EOL on 30th of April? https://nodejs.org/en/about/releases if not, I feel like this might make harder the pnpm usage on libraries that need to support Node.js v12 before that happens

Maybe just make it an alias of pnpm dlx

I can agree with pnpx being an alias for pnpm dlx, the only issue this might have is moving away from the npm npx logic of, if the package is already available in node_modules, it uses that instead of installing it from zero

Turn on strict-peer-dependencies by default (at least in monorepos)

This will make the life harder on any people that use React or GraphQL, just go give an example, to this day, Next.js still has peer dependency issues with the missing @babel/core, which is clearly not recommended to install manually.

[zkochan]

Is this change intended to align with Node.js v12 EOL on 30th of April?

Maybe it will be released under "next" dist-tag and only made "latest" after April 30th.

This will make the lifer harder on any people that use React or GraphQL, just go give an example, to this day, Next.js still has peer dependency issues with the missing @babel/core, which is clearly not recommended to install manually.

The warnings/errors may be mutate with https://pnpm.io/package_json#pnpmpeerdependencyrulesignoremissing

[anomiex]

I can agree with pnpx being an alias for pnpm dlx, the only issue this might have is moving away from the npm npx logic of, if the package is already available in node_modules, it uses that instead of installing it from zero

Yeah, we're going to have to change a bunch of our scripts to use (I guess?) pnpm exec instead of pnpx or pnpx --no-install.

On the plus side, not having to remember the --no-install everywhere to prevent it accidentally prompting to install something globally in certain edge cases (i.e. someone forgot to pnpm install first) might be an improvement.

[zkochan]

You can replace pnpx with just pnpm. pnpm eslint is the same as pnpm exec eslint.

[tjx666]

@zkochan In pnpmv6, global packages will install to node installation path.

In v7, you need manually define the PNPM_HOME to install global package.

So, I have two question:

1. Which place is recommend place to set PNPM_HOME in MacOS? Seems to be ~/Library/pnpm?
2. How to migrate global packages install in pnpmV6 to new defined PNPM_HOME in pnpmV7

[tjx666]

@zkochan It seems a bit annoying.

[zkochan]

I don't have a mac to test this. Should we use another directory for global pnpm packages? Does pnpm have write access to ~/.local/share/pnpm

[tjx666]

In before screenshot, I have deleted the pnpm folder.
After I manually create the pnpm folder ~/Library/pnpm, no permission problem...

[eshimischi]

there is 3. When you use NodeJS version managers like nvm and have several independent distributives (ex: v16.15.0, v18.0.0) how to install global packages for every nodejs version if ~/Library/pnpm is one ?

[zkochan]

When pnpm installs a package globally, it links it to a specific Node.js version. So you may have eslint that runs on Node.js 12 and typescript that runs on Node.js 14 while your current node.js is 16. There is no need to create separate global directories for each node.js version.

[zkochan]

Looks like sometimes it causes confusion that pnpm doesn't install dev dependencies, when NODE_ENV is set to production. Maybe we can change this. Yarn installs all deps, when NODE_ENV is production. npm behaves the way pnpm behaves at the moment.

[shellscape]

I'm used to this behavior and have always found yarn odd. I would think that a nice message printed to console when NODE_ENV is production would be sufficient.

[zkochan]

We can do this. I hope if will reduce the amount of issues about it. There were messages about it several time in the chat. And issues were created.

[zkochan]

Done #4362

[zkochan]

I think one more breaking change is needed. And this one is needed for implementing a deploy command.

Currently, peer dependencies are resolved from the root of the workspace as well. As a result, even when running a filtered installations, the root workspace package should be included. I suggest to change this and don't resolve peer dependencies from the root package.json.

[anomiex]

Maybe I misunderstand how node_modules/ is intended to work, but that seems contrary to the logic of it. A peer dependency is supposed to be installed alongside the package or in a parent directory, and in most cases the workspace root will be a parent directory, so I'd expect a peer installed there to be "found".

With npm or yarn (non-pnp), node itself would find the peer installed there.

[zkochan]

It will still work because Node.js resolution algorithm is not influenced by pnpm.

[zkochan]

PR: #4469

[larsivi]

I have a usecase that is a different type of deploy - we have google cloud functions in our project (aka AWS lambda). This is in practice just source code that is deployed using the gcloud cli tool. It just takes the directory given, with its package.json, uploads it, install the dependencies there and tries to run it.

The issue I have then is that I need to have the workspace link resolved prior running my deploy command, but then of course without running publish. Since our source is typescript, we already build this into its own directory, with a customized .npmrc etc (private packages). What I think would be most suitable then is to have a command like

pnpm resolve[-workspace]

.

Since that dir doesn't have a node_modules folder, the command would have to use cwd for resolution (I saw if I could fake this using publish --dry-run, but couldn't as it tried to resolve the workspace links in

). Or maybe there are other existing ways to make this work?

[anomiex]

After running into the usual phantom dep random version issues, I've been trying setting hoist-pattern to the minimal set of packages needed in our repo, which turned out to be surprisingly few. But with pnpm 7 no longer setting NODE_PATH in the bin stubs, I find that Webpack's postcss-loader can no longer find the postcss plugins. Reproduction: test.zip

Anyone have a better workaround than listing the plugin packages in hoist-pattern (ugh) or manually setting NODE_PATH when running webpack?

[shellscape]

Honestly I've been using shamefully-hoist since it was introduced and haven't had any issues.

[zkochan]

I have locked this discussion because people are posting random issues here for some reason.