Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG] Workspaces - bundledDependencies missing in tarball after npm pack #3466

Open
1 task done
noelvillagonzalo opened this issue Jun 25, 2021 · 15 comments
Open
1 task done
Labels
Bug thing that needs fixing config:workspaces related to `--workspaces` Priority 2 secondary priority issue Release 7.x work is associated with a specific npm 7 release ws:libnpmpack Related to the libnpmpack workspace

Comments

@noelvillagonzalo
Copy link

noelvillagonzalo commented Jun 25, 2021

Is there an existing issue for this?

  • I have searched the existing issues

Current Behavior

In a project with workspaces, run "npm install" and then "npm pack -w a" from the root. Packages specified under bundledDependencies of package/a are NOT included inside the node_modules of the resulting tarball (a-1.0.0.tgz)

Expected Behavior

Packages specified under bundledDependencies of package/a are included inside the node_modules of the resulting tarball (a-1.0.0.tgz)

Steps To Reproduce

  1. Clone this sample repository
  2. Ensure you are running npm >= 7
  3. Run npm install in the root of the repository
  4. Run npm pack -w a in the root of the repository
  5. Inspect the resulting tarball (a-1.0.0.tgz)
    -> Missing bundledDependencies package in node_modules, although package/a declares bundledDependencies

Environment

  • OS: macOS 10.15.17
  • Node: 15.8.0
  • npm: 7.18.1
@noelvillagonzalo noelvillagonzalo added Bug thing that needs fixing Needs Triage needs review for next steps Release 7.x work is associated with a specific npm 7 release labels Jun 25, 2021
@everett1992
Copy link

If you install dependencies in the package, instead of the workspace root, bundled dependencies will be included

$ cd packages/a
$ npm install
$ npm pack
$ tar --list -f a-1.0.0.tgz

package/node_modules/...

bundled dependencies will be included as long as node_modules are located in the package directory.

# after running the previous script and installing node_modules inside packages/a
npm-issue-missing-bundled-dependency $ npm pack -w a
npm-issue-missing-bundled-dependency $ tar --list -f a-1.0.0.tgz
package/node_modules/...

I don't think this is a good behavior, a workspace package's dependencies should be included in the bundle even when they are hoisted to the workspace root. If a workspace package depends on another workspace package that isn't published you can't run npm install outside the workspace root.

@niknetniko
Copy link

Anyone have a workaround for when you depend on another workspace package?

@ruyadorno ruyadorno added config:workspaces related to `--workspaces` Priority 2 secondary priority issue ws:libnpmpack Related to the libnpmpack workspace and removed Needs Triage needs review for next steps labels Mar 3, 2022
mcdurdin added a commit to keymanapp/keyman that referenced this issue Nov 2, 2022
Fixes #7582.

Because of npm/cli#3466, we must publish our
internal dependency @keymanapp/keyman-version. This change adds a common
builder function builder_publish_to_npm in build-utils-ci.inc.sh to help
with that task, which should be adopted by other scripts that need to
publish to npm, in the future.

For now, responsibility for publishing keyman-version is delegated to
kmlmc. In the future, we will move this to a single top-level build
action that publishes all npm modules across the entire repo for the
given version.
@jmrtinb
Copy link

jmrtinb commented Dec 2, 2022

Currently running into exactly this problem, would really like to have a way to pack a package in a workspace, which depends on other packages in the workspace and have all the bundled dependencies included in the archive properly. I haven't found a satisfactory way to work around the issue yet.

@bmeeder22
Copy link

+1

threepointone added a commit to partykit/partykit that referenced this issue Dec 28, 2022
@david-coxon
Copy link

This is a real dealbreaker imo for the bundleDependencies feature as most use cases involve a workspace approach

@jamie-pate
Copy link

jamie-pate commented Feb 4, 2023

Cheap hack using find inside the inner workspace for bundling other scoped packages from the workspace

{
  "scripts": {
    "build": "npm run tsc && ../esbuild.config.mjs && npm run bundle && ../npm_pack.sh && rm -r ./node_modules/@myscope",
    "bundle": "find ../node_modules/@myscope/* -exec bash -c 'echo $0; mkdir -p ./node_modules/@myscope/$(basename $0); cp -R \"$0/\"* ./node_modules/@myscope/$(basename $0) 2>/dev/null; echo done' '{}' \\;"
  },
  "dependencies": {
    "@myscope/mypackage": "^0.0."
  },
  "bundledDependencies": [
     "@myscope/mypackage"
   ]
}

@ViktorLajos
Copy link

+1

1 similar comment
@antoniofruci
Copy link

+1

@hps8891
Copy link

hps8891 commented Apr 17, 2023

+1

Need support for npm bundle dependencies

$ node -v
v16.18.1
$ npm -v
8.19.2

@TimothyJones
Copy link

This is still happening with npm 9.6.4, and is the cause behind aws/jsii#4132

It would be really cool if this could be sorted <3

@terozio
Copy link

terozio commented Aug 9, 2023

This severely limits usability of workspaces in cases where bundles need to be included. Any hope of getting attention to fix it?

@pedroresende
Copy link

the only workaround so far it's to use an old version of npm to do the packing....

mcdurdin added a commit to keymanapp/keyman that referenced this issue Sep 1, 2023
Works around npm/cli#3466 when bundling internal dependencies using the
bundleDependencies package.json property.

This change works in tandem with the npm pack/publish process -- when we
run `developer/src/kmc/build.sh publish` (or `pack`), we end up with
`npm version` stomping on all our package.json files, so the repo is
dirty after this. We need a copy of the top-level package.json before
this stomping happens, in order to get a simple map of the location of
each of our internal dependencies, from the `dependencies` property (it
would be possible to figure this out with a lot more parsing of
our package.json files, but this is simpler).

This means, in future, we should avoid publishing our internal
dependencies such as those under common/ to npm, as they serve no
practical purpose there.
@mcdurdin
Copy link

mcdurdin commented Sep 1, 2023

keymanapp/keyman#9536 shows the pattern we plan to use to workaround this issue, for now.

mcdurdin added a commit to keymanapp/keyman that referenced this issue Sep 1, 2023
Works around npm/cli#3466 when bundling internal dependencies using the
bundleDependencies package.json property.

This change works in tandem with the npm pack/publish process -- when we
run `developer/src/kmc/build.sh publish` (or `pack`), we end up with
`npm version` stomping on all our package.json files, so the repo is
dirty after this. We need a copy of the top-level package.json before
this stomping happens, in order to get a simple map of the location of
each of our internal dependencies, from the `dependencies` property (it
would be possible to figure this out with a lot more parsing of
our package.json files, but this is simpler).

This means, in future, we should avoid publishing our internal
dependencies such as those under common/ to npm, as they serve no
practical purpose there.
@jcable
Copy link

jcable commented Sep 10, 2023

I've incorporated a fix for this into our semantic-release plugin. It's a private repo so linking a gist to the relevant functions

https://gist.github.com/jcable/8ee152e3f5a05a47b286029e9e654984

review and comments welcome.

mcdurdin added a commit to keymanapp/keyman that referenced this issue Sep 26, 2023
chore: workaround npm/cli#3466 when bundling internal deps
@TimothyJones
Copy link

@jcable Do you think you'd be able to turn your gist into a PR? I'd like to review it, but I'm not sure where / what is changing -and I'd love to have a fix for this issue.

penberg added a commit to tursodatabase/libsql-client-ts that referenced this issue Jan 29, 2024
As it turns out, `npm` does not actually bundle dependencies if you're
using workspace...

npm/cli#3466

Let's work around that by manually copying `node_modules` in pre-build
step.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Bug thing that needs fixing config:workspaces related to `--workspaces` Priority 2 secondary priority issue Release 7.x work is associated with a specific npm 7 release ws:libnpmpack Related to the libnpmpack workspace
Projects
None yet
Development

No branches or pull requests