Server Side Modules Failing Build (ModuleNotFoundError)

[RealDrewKlayman]

Before opening, please confirm:

• I have checked to see if my question is addressed in the FAQ.
• I have searched for duplicate or closed issues.
• I have read the guide for submitting bug reports.
• I have done my best to include a minimal, self-contained set of instructions for consistently reproducing the issue.

App Id

d22wer5oad5zkl

Region

us-east-2

Amplify Console feature

Other - SSR Building (Webpack)

Describe the bug

argon2

Build fails with message:

ModuleNotFoundError: Module not found: Error: Can't resolve 'mock-aws-s3' in '/codebuild/output/src886526768/src/nextjs-amplify-import-error/node_modules/@mapbox/node-pre-gyp/lib/util'

Expected behavior

It should ignore that module since it is a server side module.

The build should not fail (more information in Additional Information section)

Reproduction steps

To skip all these steps you can just clone this repository: https://github.com/RealDrewKlayman/nextjs-amplify-import-error

1. Run npx create-next-app --typescript.
2. Move everything pages and styles into a new folder called src.
3. Install argon2 (this package is one I know that has dependencies) npm install argon2.
4. Update the src/pages/api/hello.ts file to this:

// Next.js API route support: https://nextjs.org/docs/api-routes/introduction
import argon2 from 'argon2'
import type { NextApiRequest, NextApiResponse } from 'next'

type Data = {
  password: string
}

export default async function handler(
  req: NextApiRequest,
  res: NextApiResponse<Data>
) {

  const hashedPassword = await argon2.hash('secret password');
  res.status(200).json({ password: hashedPassword })
}

(This is just a simple API route I made using that module)
5. Push/Deploy to Amplify

Build Settings

- Next Version: `latest`

Additional information

https://arunoda.me/blog/ssr-and-server-only-modules
That's an old article explaining why these modules should not be included in the build. It's old so it claims that NextJS doesn't know how to handle this but Vercel and local builds handle it perfectly.

A workaround is to add all these modules in ignorePlugin in Webpack which is not ideal at all.

[RealDrewKlayman]

This issue was discussed on the Discord and posted here..

Tagging @siegerts

[NathaEEUD]

Hi 👋, I'm getting the same behavior in a NextJS project using the next-auth package which is coupled with the serverless function api/auth/[...nextauth].js

The error message is:

ModuleNotFoundError: Module not found: Error: Can't resolve 'mongodb' in '/codebuild/output/src441885191/src/nextjs-frontend/node_modules/typeorm/platform'

[jackHedaya]

@NathaEEUD I believe we have the same problem! My issue is opened in #2103

[RealDrewKlayman]

Hi 👋, I'm getting the same behavior in a NextJS project using the next-auth package which is coupled with the serverless function api/auth/[...nextauth].js

The error message is:

ModuleNotFoundError: Module not found: Error: Can't resolve 'mongodb' in '/codebuild/output/src441885191/src/nextjs-frontend/node_modules/typeorm/platform'

@NathaEEUD I believe we have the same problem! My issue is opened in #2103

Yep, these both appear to be because of this issue.

[NathaEEUD]

@NathaEEUD I believe we have the same problem! My issue is opened in #2103

@jackHedaya Have you found any alternative solution to solve this issue? (while the issue is fixed)

[jackHedaya]

Nothing yet, waiting it out until I could find the root of the issue

[RealDrewKlayman]

Well, what do we do?

[RealDrewKlayman]

@Athena96 Can you tag this as bug?

[sonofole]

We also ran into this on our project and it's a massive blocker for us. We've tested pretty exhaustively mixing and matching different versions of Webpack, NextJS, Node, etc and nothing is working. Anything you can do to expedite a fix (or provide instructions for a workaround) would be greatly appreciated!

[siegerts]

Hi All, First of all, thanks for raising and providing details both here and on Discord. Apologies if this a blocker - The team is looking into this.

If your environment/project setup is different than @RealDrewKlayman's repro example above (or linked issues), please add here. Specifics to include are the combination of:

• what module is throwing the error (argon2, mongodb, mock-aws-s3, etc.)
• webpack version (4 or 5)
• Next.js version
• new or existing app
• etc.

[RealDrewKlayman]

@siegerts @Athena96

EDIT: This does fix the build, but it breaks the server routes..

I hope this helps with fixing this issue..

module.exports = {
  reactStrictMode: true,

  webpack: (config, { buildId, dev, isServer, defaultLoaders, webpack }) => {
    config.externals = config.externals || [];
    config.externals = [...config.externals, 'argon2']
    return config
  },
}

[Athena96]

Hi @RealDrewKlayman , sorry you are facing this. We have identified an issue with how we bundle modules with newer versions of webpack5 and Next.js for the server side lambdas. We are working on a resolution and will follow up here when we have an update.
Regarding the argon2 library, this has known issues running on Lambdas, we suggest you try another similar library like 'bcrypt' or look into using a custom build image like the article suggests.

[RealDrewKlayman]

Hi @RealDrewKlayman , sorry you are facing this. We have identified an issue with how we bundle modules with newer versions of webpack5 and Next.js for the server side lambdas. We are working on a resolution and will follow up here when we have an update.

Regarding the argon2 library, this has known issues running on Lambdas, we suggest you try another similar library like 'bcrypt' or look into using a custom build image like the article suggests.

Thank you for keeping me in the loop. My team was just asking if I had an update so I really appreciate this...

Is there a workaround I can put in my app in the meantime?

[RealDrewKlayman]

We wouldn't be able to differ from Argon2 because in the project we are using it in, it has to verify an Argon2 hash so I'll look into those articles you sent.

[marlonmarcello]

Having the same issue here. My set up is the same as @NathaEEUD
I am using NextJS 11 and Webpack 4 with NextAuth.
To answer @siegerts

• what module is throwing the error
  • I think it's happening to any module that is dynamically imported
• webpack version (4 or 5)
  • 4
• Next.js version
  • 11.0.1
• new or existing app
  • new

Like I mentioned before, I am also using NextAuth v3.27.3.
Right now I can't run any builds and the project is getting delayed. I'll gladly provide any other information you might need.

[liamegan]

We're also seeing this happen at App ID d133c0az9frtms, exactly as @marlonmarcello describes.

[martijnwiekens]

I'm having the same problem.

I got errors that it couldn't find libraries.

First it was: @sls-next/serverless-undefined. Which I solved using a different Amplify Cli version.

Then it gave me errors on:

• sass
• @zeit/next-sass
• @zeit/next-less
• @zeit/next-stylus
  Which seem to be old libraries NextJS used in an older version?

I tried added them to package.json. But then eventually it couldn't find webpack.

Currently is is failing on dependency: sass.

I do use SASS, but Next exports the application fine.

I am using:

• NextJS: 11.0.0
• Amplify Cli: 5.3.0
• Webpack: 5
• React: 17.0.2

[RealDrewKlayman]

@Athena96 Any update on this? It's still a big blocker..

[RealDrewKlayman]

My work-around which isn't really a work-around:
I had to move the code that was using Argon2 onto our main server and in the SSR route I had to call that API route. Which kind of defeats the point of having everything related to this feature in one area but we had no alternative.

[liamegan]

Any updates on this?

Our workaround, just so we could keep developing, was to remove nextauth completely. Not really an option for production at which time we're hoping this is resolved. Otherwise we need to find an alternative to Amplify.

[RealDrewKlayman]

Any updates on this?

Our workaround, just so we could keep developing, was to remove nextauth completely. Not really an option for production at which time we're hoping this is resolved. Otherwise we need to find an alternative to Amplify.

Our team sent an email to Amplify asking if for an update and we got a reply from @Athena96 yesterday saying:

Hi Drew,
 
Yes we are aware of the module import issues (leading to the build failures you are seeing) customers are experiencing with the upgrade to webpack5.

We are working on an update to make the module imports more seamless. We’re posting updates here: https://github.com/aws-amplify/amplify-console/issues/2098#issuecomment-893898449

But will notify you as well when these improvements are deployed.
 
Thanks,
Jared

We are all kind of just stuck until this issue is fixed. We are hoping a fix is implemented soon so that we can get it in before this code reaches production as well...

I really hope that there is not actually an additional issue with using Argon2 with Amplify because I didn't understand that whole custom image thing.

[ovdahlberg]

We are facing the same issue with Sentry on Next.js 11 with Webpack 5.

[ferdingler]

Hi everyone, we are adding support for deploying NextJS apps with the experimental-serverless-trace target. We have seen that this target fixes issues with the webpack bundle when using server side modules (i.e. argon).

In the interest of being extra careful and preserving backwards compatibility, we are not going to make this the default target yet. So for those having webpack issues, you will need to opt-in into this feature by specifying the target in your next.config.js file and redeploy your Amplify app:

module.exports = {
  target: 'experimental-serverless-trace'
}

Important Note As of today (Oct 11, 2021) this has not been rolled out to production yet. We plan to start rolling out very soon. Stay tuned. We will provide an update when it is GA.

[RealDrewKlayman]

Hi everyone, we are adding support for deploying NextJS apps with the experimental-serverless-trace target. We have seen that this target fixes issues with the webpack bundle when using server side modules (i.e. argon).

In the interest of being extra careful and preserving backwards compatibility, we are not going to make this the default target yet. So for those having webpack issues, you will need to opt-in into this feature by specifying the target in your next.config.js file and redeploy your Amplify app:

module.exports = {
  target: 'experimental-serverless-trace'
}

Important Note As of today (Oct 11, 2021) this has not been rolled out to production yet. We plan to start rolling out very soon. Stay tuned. We will provide an update when it is GA.

Thank you.

We'll give that a try when it is rolled out.

[gopeter]

We plan to start rolling out very soon. Stay tuned.

Hi @ferdingler. I really appreciate your work on this! I just wanted to ask, if you could specify "very soon" a little bit more 😄 Does this happen next week or rather next month?

[ferdingler]

Hi @gopeter, unfortunately we can't provide specific ETAs. But what I meant by very soon is that the release is already in the pipeline and it's just a matter of time before it rolls out to all regions. If you want to give it a try today, try in the ca-central-1 region.

[gopeter]

Great, thanks a lot!

[ferdingler]

Hi all, we have rolled this out to all regions. Feel free to give it a try by specifying the following target in your next.config.js file:

module.exports = {
  target: 'experimental-serverless-trace'
}

I will leave this ticket open if you want to report back your experience here.

[ovdahlberg]

Hi @ferdingler!

I just tried with: target: 'experimental-serverless-trace' but still same issue with Sentry.

Next.js 11
Webpack 5 (Tried with version 4 as well)
Region: eu-north-1

[ferdingler]

@ovdahlberg can I have your appId? I would like to look at the build logs just to confirm you are being opt-in properly.

[ovdahlberg]

@ferdingler Yes, of course! Here it is: d86vdq4sozpmj. You can take a look at pr-69. I did multiple tries on that branch.

[ferdingler]

@ovdahlberg thank you. I see what's happening, we are failing to load your next.config.js file due to a missing npm package @sentry/nextjs. I suspect your config file is loading this package? Not your fault, this is something we should handle gracefully but I'm curious why your next config file needs to import external dependencies.

[RealDrewKlayman]

I can confirm that this fixed the issue for us.

[ovdahlberg]

@ferdingler Thank you for looking into this! It's the way Sentry is installed in Next.js, you can see their documentation here: https://docs.sentry.io/platforms/javascript/guides/nextjs/manual-setup/

[endro42]

Is there any documentation on which regions are already supported for target: experimental-serverless-trace? Thanks!

[ferdingler]

Is there any documentation on which regions are already supported for target: experimental-serverless-trace? Thanks!

@tomploem It's available in all regions now. But I am just learning that it may not work if your next.config.js imports external libraries based on @ovdahlberg's experience. It is because we are loading the next config before dependencies are installed. This was an oversight, I didn't realize importing dependencies is a common use case in the Next Config file.

We will work on a fix for this. In the meantime, if you feel adventurous, I tried the latest canary release of v11 (v11.1.3-canary.105) and it seems to fix the issues with webpack without having to use the experimental-serverless-trace target. At least it fixed the issues in the projects we use for testing.

As a side note, NextJS announced yesterday that they will be deprecating the target option entirely in v12 [link to announcement], in which they also acknowledge that the serverless targets can cause various issues with build bundles: https://nextjs.org/docs/advanced-features/output-file-tracing.

Quote from their site:

Furthermore, this removes the need for the deprecated serverless target which can cause various issues and also creates unnecessary duplication.

The good news is that apparently these issues will be finally solved in v12 (which we don't support yet, but stay tuned). And in the meantime, we will work on a fix for the experimental-serverless-trace for when your next config has dependencies or you can also try the NextJS canary release without specifying a target.

[CodySanders]

We are running into the same issue with Sentry. @ferdingler any updates yet on the fix?

@ovdahlberg did you happen to find a workaround for this?

[RealDrewKlayman]

@ferdingler This issue has started re-occuring yet without any configuration changes.

UPDATE:

I was told that they actually reverted the ability to use the target for experimental-serverless-trace. So, this fix does no longer work.

[Rafcin]

@RealDrewKlayman AMPLIFY_NEXTJS_EXPERIMENTAL_TRACE was added. Didn't work for me, but it does exist!

[lack-of-gravitas]

Same issue with next-auth

@ferdingler This issue has started re-occuring yet without any configuration changes.

UPDATE:

I was told that they actually reverted the ability to use the target for experimental-serverless-trace. So, this fix does no longer work.

[ronyfhebrian]

Any update on these? I'm just starting to add sentry to NextJS and hosting it to amplify and it crashed.

Saying Module not found for serveral modules like webpack, any help?

[adherb]

I am also facing the same issue using next-auth, I have tried to resolve using webpack 4, but unfortunately need webpack 5 for my app to work.

Has there been any updates on this?

[pjaws]

Also experiencing this with Sentry/Webpack5.

[ghost]

We have since launched improved support for Next.js versions 12 and 13 so this should no longer be an issue. Marking this as resolved.

[github-actions]

⚠️COMMENT VISIBILITY WARNING⚠️

Comments on closed issues are hard for our team to see.
If you need more assistance, please open a new issue that references this one.

[github-actions]

This issue has been automatically locked.