Bug - yarn packager is causing sls deploy to fail #690

K3vChan · 2021-02-11T23:58:16Z

This is a Bug Report

Description

Using serverless pro to deploy using serverless-webpack "serverless-webpack": "^5.3.5". Successfully deployed if I change packager to npm. not sure what is going on. If I deploy locally with yarn as the packager it works.

Additional Data

Serverless: Invoke webpack:package

Error --------------------------------------------------

Error: yarn list --depth=1 --json --production failed with code 1
at ChildProcess. (/hive/services/sales-profile/node_modules/serverless-webpack/lib/utils.js:91:16)
at ChildProcess.emit (events.js:314:20)
at ChildProcess.EventEmitter.emit (domain.js:483:12)
at maybeClose (internal/child_process.js:1021:16)
at Process.ChildProcess._handle.onexit (internal/child_process.js:286:5)

Get Support --------------------------------------------
Docs: docs.serverless.com
Bugs: github.com/serverless/serverless/issues
Issues: forum.serverless.com

Your Environment Information ---------------------------
Operating System: linux
Node Version: 12.19.0
Framework Version: 2.23.0 (local)
Plugin Version: 4.4.3
SDK Version: 2.3.2
Components Version: 3.6.2

Build step failed: serverless deploy --stage feature-sales-profiles-lambdas --region us-east-1 --force --org myOrg --app myApp --debug
failure
Deploy failed!
Error: Build step failed: serverless deploy --stage feature-sales-profiles-lambdas --region us-east-1 --force --org myOrg --app myApp --debug

itskemo · 2021-02-12T13:59:44Z

Unfortunately I have no space to investigate, but I can tell that this happens when running npm as well - both locally and on pipeline.

Serverless: Fetch dependency graph from [redacted]/package.json

  Error --------------------------------------------------

  Error: npm ls -prod -json -depth=1 failed with code 1
      at ChildProcess.<anonymous> ([redacted]/node_modules/serverless-webpack/lib/utils.js:91:16)
      at ChildProcess.emit (events.js:314:20)
      at ChildProcess.EventEmitter.emit (domain.js:483:12)
      at maybeClose (internal/child_process.js:1022:16)
      at Socket.<anonymous> (internal/child_process.js:444:11)
      at Socket.emit (events.js:314:20)
      at Socket.EventEmitter.emit (domain.js:483:12)
      at Pipe.<anonymous> (net.js:675:12)

     For debugging logs, run again after setting the "SLS_DEBUG=*" environment variable.

  Get Support --------------------------------------------
     Docs:          docs.serverless.com
     Bugs:          github.com/serverless/serverless/issues
     Issues:        forum.serverless.com

  Your Environment Information ---------------------------
     Operating System:          darwin
     Node Version:              12.20.2
     Framework Version:         1.83.2 (local)
     Plugin Version:            3.8.4
     SDK Version:               2.3.2
     Components Version:        2.34.9

Workaround:

In order to allow my services to deploy successfully I had to downgrade the package via eg.

npm install --save-dev [email protected]

K3vChan · 2021-02-12T17:25:05Z

Just set this to [email protected] same issue. :/ just using npm for now...

ariel-frischer · 2021-02-23T05:42:59Z

I'm getting the same issues with [email protected] both npm and yarn.

ciceropablo · 2021-02-26T02:43:06Z

Same issue with 5.3.4 and 5.3.5.

ariel-frischer · 2021-02-26T17:09:07Z

Actually my yarn error is slightly different:

 Error: yarn.cmd install --frozen-lockfile --non-interactive failed with code 1
      at ChildProcess.<anonymous> (C:\Users\Ariel\initiate-video\backend\node_modules\serverless-webpack\lib\utils.js:91:16)
      at ChildProcess.emit (events.js:314:20)
      at ChildProcess.EventEmitter.emit (domain.js:483:12)
      at maybeClose (internal/child_process.js:1021:16)
      at Process.ChildProcess._handle.onexit (internal/child_process.js:286:5)

j0k3r · 2021-02-26T17:18:10Z

Which version of yarn are you all using?
Could you trying using "serverless-webpack": "serverless-heaven/serverless-webpack#master" just to check if the issue is already fixed in the upcoming 5.4.0?

ariel-frischer · 2021-02-26T17:19:56Z

@j0k3r Im using yarn: 1.22.4 and npm 6.14.8. I'll test it now.
I've also already tried setting noFrozenLockFile: true which doesn't seem to do anything.

ariel-frischer · 2021-02-26T17:28:20Z

@j0k3r Looks like the serverless-heaven/serverless-webpack#master is working!

j0k3r · 2021-03-04T10:45:59Z

As the master seems to fix the problem, I'm closing that issue.
Let me know if you have the problem when using the master and not a version lower than 5.4.0

ariel-frischer · 2021-03-16T01:40:37Z

Unfortunately I'm seeing this again with Windows 10 specifically. Even with the master branch...

j0k3r · 2021-03-16T05:15:14Z

But nothing really changed since your last comment 🤔

ariel-frischer · 2021-03-16T05:17:36Z

But nothing really changed since your last comment 🤔

Yea, it's strange it happens on my laptop but not my desktop, could be different versions of yarn I'll look into it... I also think this may have something to do with using a linked package.

omar-dulaimi · 2021-03-18T04:45:12Z

Hello,
@j0k3r
I've been trying to get it to work for the past couple of hours but with no luck. I'm using the latest version and tried the master commit as you suggested but did not work.

Here are some verbose logs:

webpack compiled successfully in 64989 ms
Serverless: Invoke webpack:package
(node:16957) [DEP_WEBPACK_CHUNK_MODULES_ITERABLE] DeprecationWarning: Chunk.modulesIterable: Use new ChunkGraph API
(Use `node --trace-deprecation ...` to show where the warning was created)
(node:16957) [DEP_WEBPACK_MODULE_ISSUER] DeprecationWarning: Module.issuer: Use new ModuleGraph API
Serverless: Package lock found - Using locked versions
Serverless: Packing external modules: @prisma/[email protected], apollo-errors@^1.9.0, apollo-server-lambda@^2.21.1, graphql@^15.3.0, aws-sdk@^2.866.0, axios@^0.21.1, bcrypt@^5.0.1, class-validator@^0.13.1, graphql-depth-limit@^1.1.0, graphql-fields@^2.0.3, graphql-middleware@^6.0.4, graphql-shield@^7.5.0, joi@^17.4.0, [email protected], md5@^2.3.0, moment@^2.29.1, object-path@^0.11.5, p-map@^4.0.0, random@^3.0.3, reflect-metadata@^0.1.13, type-graphql@^1.1.1
 ..................] \ : timing npm:load Completed in 19ms
 Error ---------------------------------------------------
 
  Error: yarn install --non-interactive failed with code 1
      at ChildProcess.<anonymous> (/home/omar/Desktop/graphqla-api/node_modules/serverless-webpack/lib/utils.js:91:16)
      at ChildProcess.emit (events.js:315:20)
      at ChildProcess.EventEmitter.emit (domain.js:486:12)
      at maybeClose (internal/child_process.js:1048:16)
      at Process.ChildProcess._handle.onexit (internal/child_process.js:288:5)
 
  Get Support --------------------------------------------
     Docs:          docs.serverless.com
     Bugs:          github.com/serverless/serverless/issues
     Issues:        forum.serverless.com
 
  Your Environment Information ---------------------------
     Operating System:          linux
     Node Version:              14.15.1
     Framework Version:         2.29.0
     Plugin Version:            4.5.0
     SDK Version:               n/a
     Components Version:        3.7.3
 
npm timing command:run-script Completed in 72520ms
npm verb exit 1
npm timing npm Completed in 72628ms
npm verb code 1

yarn v2.4.1
serverless-webpack v5.4.0

jp928 · 2021-03-18T05:37:12Z

@omar-dulaimi Same here, I turned to

 packagerOptions:
      noFrozenLockfile: true

omar-dulaimi · 2021-03-18T15:30:57Z

@jp928 Unfortunately setting this flag did not solve the issue for me:

        webpack: {
            webpackConfig: './webpack.config.js',
            includeModules: true,
            packager: 'yarn',
            packagerOptions: {
                scripts: ['prisma generate'],
                noFrozenLockfile: true,
            },
        },

omar-dulaimi · 2021-03-20T18:53:16Z

Any update on this?
We tried to downgrade but that also did not help, and as a result, we can't deploy our app and are forced to keep on hold.

omar-dulaimi · 2021-03-20T21:32:30Z

So after a little bit of digging around, here's what I found:

"./node_modules/serverless-webpack/lib/packagers/yarn.js"

  static install(cwd, packagerOptions) {
    const command = /^win/.test(process.platform) ? 'yarn.cmd' : 'yarn';
    const args = [ 'install', '--non-interactive' ];

    if (!packagerOptions.noFrozenLockfile) {
      args.push('--frozen-lockfile');
    }
    if (packagerOptions.ignoreScripts) {
      args.push('--ignore-scripts');
    }

    // return Utils.spawnProcess(command, args, { cwd }).return(); // replaced with below return statement
    return BbPromise.fromCallback(cb => {
      childProcess.exec(command, {
        cwd: cwd,
        maxBuffer: 10000000,
        encoding: 'utf8'
      }, cb);
    })
    .return();
  }

  static runScripts(cwd, scriptNames) {
    const command = /^win/.test(process.platform) ? 'yarn.cmd' : 'yarn';
    // return BbPromise.mapSeries(scriptNames, scriptName => {   // replaced with below code
    //   const args = [ 'run', scriptName ];

    //   return Utils.spawnProcess(command, args, { cwd });
    // }).return();
    return BbPromise.mapSeries(scriptNames, scriptName => BbPromise.fromCallback(cb => {
      childProcess.exec(`yarn run ${scriptName}`, {
        cwd: cwd,
        maxBuffer: 100000000,
        encoding: 'utf8'
      }, cb);
    }))
    .return();
  }

Basically, I had to change how serverless-webpack install and runScripts methods work, by changing them back to their @5.1.0 forms.

In those forms, the child processes were called using a different node api, that sets the max buffer for stdio.

This maxBuffer value can be set from the serverless template like this(as mentioned in @5.1.0 README):

custom:
  webpack:
    packExternalModulesMaxBuffer: NUMBER_OF_YOUR_CHOICE   # Size of stdio buffers for spawned child processes

Hopefully, some people here shed some light on the notes above so we could choose sensible values.

Happy to submit a PR for this @j0k3r

yehonadav · 2021-05-31T19:59:16Z

any1 managed to solve this? its really annoying

yehonadav · 2021-05-31T20:14:48Z

so i noticed i was missing a dependency in my package.json that was installed inside my node_modules.
after adding the module to package.json everything worked out fine.

i was using jsonwebtoken and had @types/jsonwebtoken in my dev-dependencies but not jsonwebtoken in my dependencies.

was running with:
win10
yarn v1.22.10
serverless-webpack 5.5.0

hope this helps

mrowles · 2021-08-10T01:49:59Z

This is still occurring for us, but only randomly in some components which all have mostly identical package, webpack + sls configs.

BB pipelines using Atlassian Linux distro
yarn 2
serverless-webpack 5.3.0 - 5.5.1 (tried them all)
serverless 2.43.1

People are having issues with newer versions serverless-heaven/serverless-webpack/issues/690

… npm install. Had this issue: serverless-heaven/serverless-webpack#690. Also removed/updated some dependencies.

yehonadav · 2021-09-26T16:26:36Z

This is still occurring for us, but only randomly in some components which all have mostly identical package, webpack + sls configs.

BB pipelines using Atlassian Linux distro
yarn 2
serverless-webpack 5.3.0 - 5.5.1 (tried them all)
serverless 2.43.1

while this is still an open issue, if your using typescript you can give serverless-plugin-typescript a try.
it hadn't given me any problems so far

deepakandgupta · 2021-10-07T17:49:44Z

For me this was an error due to an old serverless version and not in serverless-webpack, bumping serverless from 2.43 to 2.51.2 Solved it.
Not sure if that will work for everyone though

vicary · 2021-11-19T16:52:22Z

See #992 (comment), feel free to re-open this if you think your problem have a different cause.

philgran · 2022-05-18T21:28:07Z

This fixed it for me:
#992 (comment)
serverless-webpack was trying to use npm while my whole stack was set up for yarn.

danrivett · 2023-03-05T00:45:57Z

So I ran into what appears to be the same issue as this ticket after we recreated our yarn.lock file.

I narrowed down the following error to the commit that we deleted and recreated the yarn.lock file - the previous commit works just fine running sls package.

$ sls package
...
Environment: darwin, node 18.14.2, framework 3.28.1 (local) 3.28.1v (global), plugin 6.2.3, SDK 4.3.2
Docs:        docs.serverless.com
Support:     forum.serverless.com
Bugs:        github.com/serverless/serverless/issues

Error:
Error: yarn install --non-interactive --frozen-lockfile failed with code 1
    at ChildProcess.<anonymous> (.../node_modules/serverless-webpack/lib/utils.js:91:16)

I can confirm after trying many things, two methods (other than going back to the commit prior to recreating the yarn.lock file) solved my problem:

Adding noFrozenLockfile: true to the webpack.config.yml file
Editing the node_modules/serverless-webpack/lib/packagers/yarn.js file as per @omar-dulaimi's suggestion. This also worked.

So after a little bit of digging around, here's what I found:

"./node_modules/serverless-webpack/lib/packagers/yarn.js"

  static install(cwd, packagerOptions) {
    const command = /^win/.test(process.platform) ? 'yarn.cmd' : 'yarn';
    const args = [ 'install', '--non-interactive' ];

    if (!packagerOptions.noFrozenLockfile) {
      args.push('--frozen-lockfile');
    }
    if (packagerOptions.ignoreScripts) {
      args.push('--ignore-scripts');
    }

    // return Utils.spawnProcess(command, args, { cwd }).return(); // replaced with below return statement
    return BbPromise.fromCallback(cb => {
      childProcess.exec(command, {
        cwd: cwd,
        maxBuffer: 10000000,
        encoding: 'utf8'
      }, cb);
    })
    .return();
  }

  static runScripts(cwd, scriptNames) {
    const command = /^win/.test(process.platform) ? 'yarn.cmd' : 'yarn';
    // return BbPromise.mapSeries(scriptNames, scriptName => {   // replaced with below code
    //   const args = [ 'run', scriptName ];

    //   return Utils.spawnProcess(command, args, { cwd });
    // }).return();
    return BbPromise.mapSeries(scriptNames, scriptName => BbPromise.fromCallback(cb => {
      childProcess.exec(`yarn run ${scriptName}`, {
        cwd: cwd,
        maxBuffer: 100000000,
        encoding: 'utf8'
      }, cb);
    }))
    .return();
  }

...

I'm therefore curious if there is a bug here that was introduced with changes introduced after 5.1.0, or if the change just exposes a configuration issue on our end that wasn't causing an issue before.

I did run depcheck and I don't think I have missing dependencies. Some peer dependencies don't quite match the expected versions, but they didn't before I recreated the yarn.lock file either.

Anyway, for now, we're going to go with option 1, as I spent quite a few hours of trial and error trying to figure out the root cause that recreating the yarn.lock file exposed, and I'm not quite sure how best to solve that in the short term.

emersonmellado · 2023-09-01T04:24:12Z

I just had the same error on Aug, 31, 2023.

My environment: darwin, node 16.20.0, framework 3.34.0 (local), plugin 6.2.3, SDK 4.3.2

"serverless-webpack": "^5.3.0"

After isolating packages and downgrading sls and node version by version I finally got a warning saying it could not identify the version of axios, so I reset all versions, simply installed axios and everything went just fine. The funny thing is that we don't use axios at all, so I tend to believe that it is some sort of peer dependency of packages.

I hope this helps someone.

juztinlazaro · 2023-11-09T13:01:15Z

Hi any fix on this?

xuanhop · 2023-12-23T06:27:08Z

@omar-dulaimi Same here, I turned to
 packagerOptions:
      noFrozenLockfile: true

Thanks brother :D I got it

K3vChan changed the title ~~Bug - yarn packager is causing sis deploy to fail~~ Bug - yarn packager is causing sls deploy to fail Feb 12, 2021

j0k3r closed this as completed Mar 4, 2021

j0k3r added this to the 5.4.0 milestone Mar 4, 2021

j0k3r reopened this Mar 18, 2021

j0k3r removed this from the 5.4.0 milestone Mar 18, 2021

omar-dulaimi mentioned this issue Mar 20, 2021

Deploying Prisma with AWS Lambda and Webpack prisma/prisma#2303

Open

mikejpeters mentioned this issue Apr 11, 2021

npm ls errors when using npm 7 workspaces #781

Closed

mojotalantikite added a commit to Enterprise-CMCS/managed-care-review that referenced this issue Sep 3, 2021

Downgrade serverless-webpack

cb9d446

People are having issues with newer versions serverless-heaven/serverless-webpack/issues/690

JonathanMSifleet mentioned this issue Nov 7, 2021

serverless deploy failing, "npm.cmd ls -prod -json -depth=1 failed with code 1" #992

Closed

vicary closed this as completed Nov 19, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Bug - yarn packager is causing sls deploy to fail #690

Bug - yarn packager is causing sls deploy to fail #690

K3vChan commented Feb 11, 2021 •

edited

Loading

itskemo commented Feb 12, 2021 •

edited

Loading

K3vChan commented Feb 12, 2021

ariel-frischer commented Feb 23, 2021 •

edited

Loading

ciceropablo commented Feb 26, 2021

ariel-frischer commented Feb 26, 2021

j0k3r commented Feb 26, 2021

ariel-frischer commented Feb 26, 2021 •

edited

Loading

ariel-frischer commented Feb 26, 2021

j0k3r commented Mar 4, 2021

ariel-frischer commented Mar 16, 2021

j0k3r commented Mar 16, 2021

ariel-frischer commented Mar 16, 2021

omar-dulaimi commented Mar 18, 2021

jp928 commented Mar 18, 2021

omar-dulaimi commented Mar 18, 2021

omar-dulaimi commented Mar 20, 2021

omar-dulaimi commented Mar 20, 2021 •

edited

Loading

yehonadav commented May 31, 2021

yehonadav commented May 31, 2021

mrowles commented Aug 10, 2021

yehonadav commented Sep 26, 2021 •

edited

Loading

deepakandgupta commented Oct 7, 2021 •

edited

Loading

vicary commented Nov 19, 2021

philgran commented May 18, 2022

danrivett commented Mar 5, 2023 •

edited

Loading

emersonmellado commented Sep 1, 2023

juztinlazaro commented Nov 9, 2023

xuanhop commented Dec 23, 2023

Bug - yarn packager is causing sls deploy to fail #690

Bug - yarn packager is causing sls deploy to fail #690

Comments

K3vChan commented Feb 11, 2021 • edited Loading

This is a Bug Report

Description

Additional Data

itskemo commented Feb 12, 2021 • edited Loading

K3vChan commented Feb 12, 2021

ariel-frischer commented Feb 23, 2021 • edited Loading

ciceropablo commented Feb 26, 2021

ariel-frischer commented Feb 26, 2021

j0k3r commented Feb 26, 2021

ariel-frischer commented Feb 26, 2021 • edited Loading

ariel-frischer commented Feb 26, 2021

j0k3r commented Mar 4, 2021

ariel-frischer commented Mar 16, 2021

j0k3r commented Mar 16, 2021

ariel-frischer commented Mar 16, 2021

omar-dulaimi commented Mar 18, 2021

jp928 commented Mar 18, 2021

omar-dulaimi commented Mar 18, 2021

omar-dulaimi commented Mar 20, 2021

omar-dulaimi commented Mar 20, 2021 • edited Loading

yehonadav commented May 31, 2021

yehonadav commented May 31, 2021

mrowles commented Aug 10, 2021

yehonadav commented Sep 26, 2021 • edited Loading

deepakandgupta commented Oct 7, 2021 • edited Loading

vicary commented Nov 19, 2021

philgran commented May 18, 2022

danrivett commented Mar 5, 2023 • edited Loading

emersonmellado commented Sep 1, 2023

juztinlazaro commented Nov 9, 2023

xuanhop commented Dec 23, 2023

K3vChan commented Feb 11, 2021 •

edited

Loading

itskemo commented Feb 12, 2021 •

edited

Loading

ariel-frischer commented Feb 23, 2021 •

edited

Loading

ariel-frischer commented Feb 26, 2021 •

edited

Loading

omar-dulaimi commented Mar 20, 2021 •

edited

Loading

yehonadav commented Sep 26, 2021 •

edited

Loading

deepakandgupta commented Oct 7, 2021 •

edited

Loading

danrivett commented Mar 5, 2023 •

edited

Loading