Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Watchpack Error (initial scan): Error: ENOTDIR: not a directory in v3 #3322

Closed
hrougier opened this issue Sep 29, 2024 · 31 comments · Fixed by #3328 or #3330
Closed

Watchpack Error (initial scan): Error: ENOTDIR: not a directory in v3 #3322

hrougier opened this issue Sep 29, 2024 · 31 comments · Fixed by #3328 or #3330
Labels
bug Something isn't working

Comments

@hrougier
Copy link

Describe the bug

Tried to use nextra 3.0.1 and got the following issue (reproduced in codesandbox).

To Reproduce

  1. Go to codesandbox.io
  2. Click on the Start Dev Server terminal
  3. See error and broken UI
Watchpack Error (initial scan): Error: ENOTDIR: not a directory, scandir '/project/sandbox/node_modules/next/dist/pages/_app.js'
 ⚠ ./node_modules/@typescript/vfs/dist/vfs.esm.js
Critical dependency: the request of a dependency is an expression

Import trace for requested module:
./node_modules/@typescript/vfs/dist/vfs.esm.js
./node_modules/twoslash/dist/index.mjs
./node_modules/@shikijs/twoslash/dist/index.mjs
./node_modules/nextra/dist/server/compile.js
./node_modules/nextra/dist/client/components/playground.js
./node_modules/nextra/dist/client/components/index.js
./node_modules/nextra-theme-docs/dist/index.js

Expected behavior

No error in terminal and a working nextra site.

Additional context

Setup from the Getting Started tutorial with the following dependencies versions

{
  "next": "14.2.13",
  "nextra": "3.0.1",
  "nextra-theme-docs": "3.0.1",
  "react": "18.3.1",
  "react-dom": "18.3.1"
}
@hrougier hrougier added the bug Something isn't working label Sep 29, 2024
@marcklingen
Copy link
Contributor

having the same issue, did you find a workaround to fix this?

@hrougier
Copy link
Author

having the same issue, did you find a workaround to fix this?

Not yet, unlike the minimal codepen reproduction it doesn't work at all on my existing nextra site and only displays an Internal Server Error page instead of a broken UI, with the following stack traces:

 ○ Compiling / ...
Watchpack Error (initial scan): Error: ENOTDIR: not a directory, scandir '/path/to/node_modules/next/dist/pages/_app.js'
 ⨯ ../node_modules/typescript/lib/typescript.js:43173:24
Module not found: Can't resolve 'module'

https://nextjs.org/docs/messages/module-not-found

Import trace for requested module:
../node_modules/twoslash/dist/index.mjs
../node_modules/@shikijs/twoslash/dist/index.mjs
../node_modules/nextra/dist/server/compile.js
../node_modules/nextra/dist/client/components/playground.js
../node_modules/nextra/dist/client/components/index.js
../node_modules/nextra-theme-docs/dist/index.js
../node_modules/next/dist/esm/pages/_app.js

Watchpack Error (initial scan): Error: ENOTDIR: not a directory, scandir '/path/to/node_modules/next/dist/pages/_app.js'
 ⨯ ../node_modules/typescript/lib/typescript.js:43173:24
Module not found: Can't resolve 'module'

https://nextjs.org/docs/messages/module-not-found

Import trace for requested module:
../node_modules/twoslash/dist/index.mjs
../node_modules/@shikijs/twoslash/dist/index.mjs
../node_modules/nextra/dist/server/compile.js
../node_modules/nextra/dist/client/components/playground.js
../node_modules/nextra/dist/client/components/index.js
../node_modules/nextra-theme-docs/dist/index.js
../node_modules/next/dist/esm/pages/_app.js
 ⨯ Error: ENOENT: no such file or directory, open '/path/to/.next/fallback-build-manifest.json'
    at readFileSync (node:fs:448:20)
    at loadManifest (/path/to/node_modules/next/dist/server/load-manifest.js:36:52)
    at loadManifestWithRetries (/path/to/node_modules/next/dist/server/load-components.js:40:51)
    at async loadDefaultErrorComponentsImpl (/path/to/node_modules/next/dist/server/load-default-error-components.js:30:24)
    at async DevServer.getFallbackErrorComponents (/path/to/node_modules/next/dist/server/dev/next-dev-server.js:594:16)
    at async DevServer.renderErrorToResponseImpl (/path/to/node_modules/next/dist/server/base-server.js:2180:40)
    at async DevServer.renderToResponseImpl (/path/to/node_modules/next/dist/server/base-server.js:1996:30)
    at async DevServer.pipeImpl (/path/to/node_modules/next/dist/server/base-server.js:914:25)
    at async NextNodeServer.handleCatchallRenderRequest (/path/to/node_modules/next/dist/server/next-server.js:272:17)
    at async DevServer.handleRequestImpl (/path/to/node_modules/next/dist/server/base-server.js:810:17)
    at async /path/to/node_modules/next/dist/server/dev/next-dev-server.js:339:20
    at async Span.traceAsyncFn (/path/to/node_modules/next/dist/trace/trace.js:154:20)
    at async DevServer.handleRequest (/path/to/node_modules/next/dist/server/dev/next-dev-server.js:336:24)
    at async invokeRender (/path/to/node_modules/next/dist/server/lib/router-server.js:173:21)
    at async handleRequest (/path/to/node_modules/next/dist/server/lib/router-server.js:350:24) {
  errno: -2,
  code: 'ENOENT',
  syscall: 'open',
  path: '/path/to/.next/fallback-build-manifest.json'
}

@nshiab
Copy link

nshiab commented Sep 30, 2024

Same problem when I followed the steps here: https://nextra.site/docs/docs-theme/start

@dimaMachina
Copy link
Collaborator

try 3.0.2 where the warning Watchpack Error (initial scan): Error: ENOTDIR: not a directory, scandir is gone and styles seems correctly applied

@marcklingen
Copy link
Contributor

marcklingen commented Sep 30, 2024

Thanks for the quick fix @dimaMachina 🙏

did this resolve the issue for you @hrougier? I am still getting these errors on 3.0.2 (build and dev server):

./node_modules/.pnpm/@[email protected][email protected]/node_modules/@typescript/vfs/dist/vfs.esm.js
Critical dependency: the request of a dependency is an expression

Import trace for requested module:
./node_modules/.pnpm/@[email protected][email protected]/node_modules/@typescript/vfs/dist/vfs.esm.js
./node_modules/.pnpm/[email protected][email protected]/node_modules/twoslash/dist/index.mjs
./node_modules/.pnpm/@[email protected][email protected]/node_modules/@shikijs/twoslash/dist/index.mjs
./node_modules/.pnpm/[email protected]_@[email protected][email protected]_@[email protected][email protected]_react_kdvvi7vlst7dzlxuv6gvakrxwi/node_modules/nextra/dist/server/compile.js
./node_modules/.pnpm/[email protected]_@[email protected][email protected]_@[email protected][email protected]_react_kdvvi7vlst7dzlxuv6gvakrxwi/node_modules/nextra/dist/client/components/playground.js
./node_modules/.pnpm/[email protected]_@[email protected][email protected]_@[email protected][email protected]_react_kdvvi7vlst7dzlxuv6gvakrxwi/node_modules/nextra/dist/client/components/index.js

./node_modules/.pnpm/@[email protected][email protected]/node_modules/@typescript/vfs/dist/vfs.cjs.production.min.js
Critical dependency: the request of a dependency is an expression

Import trace for requested module:
./node_modules/.pnpm/@[email protected][email protected]/node_modules/@typescript/vfs/dist/vfs.cjs.production.min.js
./node_modules/.pnpm/@[email protected][email protected]/node_modules/@typescript/vfs/dist/index.js
./node_modules/.pnpm/[email protected][email protected]/node_modules/twoslash/dist/index.mjs
./node_modules/.pnpm/@[email protected][email protected]/node_modules/@shikijs/twoslash/dist/index.mjs
./node_modules/.pnpm/[email protected]_@[email protected][email protected]_@[email protected][email protected]_react_kdvvi7vlst7dzlxuv6gvakrxwi/node_modules/nextra/dist/server/compile.js
./node_modules/.pnpm/[email protected]_@[email protected][email protected]_@[email protected][email protected]_react_kdvvi7vlst7dzlxuv6gvakrxwi/node_modules/nextra/dist/client/components/playground.js
./node_modules/.pnpm/[email protected]_@[email protected][email protected]_@[email protected][email protected]_react_kdvvi7vlst7dzlxuv6gvakrxwi/node_modules/nextra/dist/client/components/index.js
./pages/blog/qa-chatbot-for-langfuse-docs.mdx

package.json

@dimaMachina
Copy link
Collaborator

this is upstream, false positive warnings from twoslash package https://nextra.site/docs/guide/advanced/twoslash-support

I reported it already microsoft/TypeScript-Website#3022

maybe we should disable twoslash by default

@hrougier
Copy link
Author

hrougier commented Sep 30, 2024

@marcklingen Yes it did resolve the Watchpack warning but I still get these 2:

 ⨯ ../node_modules/typescript/lib/typescript.js:43173:24
Module not found: Can't resolve 'module'
... (see previous comment)

and

Error: ENOENT: no such file or directory, open '/path/to/.next/fallback-build-manifest.json'
... (see previous comment)

Still experimenting to see what might be causing these errors!

@dimaMachina I was about to ask you that, maybe it is possible to make twoslash optional through some configuration option?

@marcklingen
Copy link
Contributor

maybe we should disable twoslash by default

Not sure if worth the complexity but would be a good option for me as I do not need twoslash support

@Chirag-kalsariya
Copy link

"dependencies": {
    "next": "^14.2.13",
    "nextra": "3.0.2",
    "nextra-theme-docs": "3.0.2",
    "react": "^18.3.1",
    "react-dom": "^18.3.1"
},
"devDependencies": {
    "@types/react": "^18.3.10",
    "@types/node": "22.7.4",
    "typescript": "^5.6.2"
}

I used 3.0.2, and still getting broken page with no css style. and Critical dependency error

○ Compiling / ...
 ⚠ ./node_modules/@typescript/vfs/dist/vfs.esm.js
Critical dependency: the request of a dependency is an expression

Import trace for requested module:
./node_modules/@typescript/vfs/dist/vfs.esm.js
./node_modules/twoslash/dist/index.mjs
./node_modules/@shikijs/twoslash/dist/index.mjs
./node_modules/nextra/dist/server/compile.js
./node_modules/nextra/dist/client/components/playground.js
./node_modules/nextra/dist/client/components/index.js
./node_modules/nextra-theme-docs/dist/index.js
 GET / 200 in 51967ms
Screenshot 2024-09-30 at 3 52 14 PM

@hrougier
Copy link
Author

The error about the .next/fallback-build-manifest.json seems to happen because of the failing twoslash module import, I can confirm I got v3.0.2 to work by removing the following import and its usages:

import { rendererRich, transformerTwoslash } from "@shikijs/twoslash";

from node_modules/nextra/dist/server/compile.js (.next folder must be deleted and next dev server started again).

Now I have other issues with soft navigation when using the type: 'page' in _meta.js but that's another topic.

@marcklingen
Copy link
Contributor

marcklingen commented Sep 30, 2024

I used 3.0.2, and still getting broken page with no css style. and Critical dependency error

In my case this does not affect css, page works completely fine (we use nextra + tailwind), just breaks dev-server navigation (need to try to navigate to each page twice before it actually works) and produces these logs. Production deployment looks completely fine

@Chirag-kalsariya
Copy link

The error about the .next/fallback-build-manifest.json seems to happen because of the failing twoslash module import, I can confirm I got v3.0.2 to work by removing the following import and its usages:

import { rendererRich, transformerTwoslash } from "@shikijs/twoslash";

from node_modules/nextra/dist/server/compile.js (.next folder must be deleted and next dev server started again).

Now I have other issues with soft navigation when using the type: 'page' in _meta.js but that's another topic.

I try it, and work but In my case when directly go to http://localhost:3000 CSS is not working, but when I navigate pages everything is working fine. I am using npm not pnpm.

@Chirag-kalsariya
Copy link

I used 3.0.2, and still getting broken page with no css style. and Critical dependency error

In my case this does not affect css, page works completely fine, just breaks dev-server navigation (need to try to navigate to each page twice before it actually works) and produces these logs. Production deployment looks completely fine

Yes, Production deployment looks completely fine. with version 3.0.2. without changing anything in node_modules/nextra/dist/server/compile.js

@dimaMachina
Copy link
Collaborator

dimaMachina commented Oct 2, 2024

I found how to fix warnings from @typescript/vfs without waiting for upstream fix 🎉

#3330

@dimaMachina
Copy link
Collaborator

So now it left only warnings from unified

nextra:dev: "Value" is imported from external module "vfile" but never used in "node_modules/unified/index.d.ts".
nextra:dev: "CompileResults" is imported from external module "./lib/index.js" but never used in "node_modules/unified/index.d.ts".

Reported in unifiedjs/unified#250

@danielkleinstein
Copy link

Can we reopen this issue? I still get the warning after upgrading to 3.0.7 - and it looks like other people are too

@danielkleinstein
Copy link

For now I'm working with this workaround in next.config.mjs:

import nextra from "nextra";

const withNextra = nextra({
  theme: "nextra-theme-docs",
  themeConfig: "./theme.config.tsx",
});

export default withNextra({
  webpack: (config) => {
    config.module = {
      ...config.module,
      exprContextCritical: false,
    };

    return config;
  },
});

But this isn't ideal.

@dimaMachina
Copy link
Collaborator

Please share reproduction repo

@danielkleinstein
Copy link

danielkleinstein commented Oct 9, 2024

I can't share the repo itself as it's private - but I tried creating a new repo from scratch as per the instructions here - https://nextra.site/docs/docs-theme/start

I followed the instructions to the letter, but when I ran pnpm run dev:

  • I got the lack-of-CSS problem mentioned above
  • I then ran rm -rf .next and pnpm run dev, and the CSS problem resolved itself, but I got this warning:
Module not found: Can't resolve 'source-map-support' in '/Users/danielkleinstein/tmp/nextra/node_modules/.pnpm/[email protected]/node_modules/typescript/lib'

Import trace for requested module:
./node_modules/.pnpm/[email protected]/node_modules/typescript/lib/typescript.js
./node_modules/.pnpm/[email protected][email protected]/node_modules/twoslash/dist/index.mjs
./node_modules/.pnpm/@[email protected][email protected]/node_modules/@shikijs/twoslash/dist/index.mjs
./node_modules/.pnpm/[email protected]_@[email protected][email protected][email protected][email protected][email protected]__r_lgnh5dmjpho45ld7y5jnnfcyza/node_modules/nextra/dist/server/compile.js
./node_modules/.pnpm/[email protected]_@[email protected][email protected][email protected][email protected][email protected]__r_lgnh5dmjpho45ld7y5jnnfcyza/node_modules/nextra/dist/client/components/playground.js
./node_modules/.pnpm/[email protected]_@[email protected][email protected][email protected][email protected][email protected]__r_lgnh5dmjpho45ld7y5jnnfcyza/node_modules/nextra/dist/client/components/index.js
./node_modules/.pnpm/[email protected][email protected][email protected][email protected][email protected][email protected]._asslf2kagsatjgjntmqd5n4xfe/node_modules/nextra-theme-docs/dist/index.js
 GET / 200 in 13593ms

Though this seems to be a separate warning. Couldn't immediately reproduce this issue's warning on the clean repo, sorry

@hrougier
Copy link
Author

hrougier commented Oct 9, 2024

@danielkleinstein I still have the same issue with 3.0.7, my fix for now is to use the patch command of pnpm/yarn/bun to comment the twoslash import and usage from nextra/dist/server/compile.js:

pnpm patch nextra

Comment the twoslash import and usage from dist/server/compile.js

// import { rendererRich, transformerTwoslash } from "@shikijs/twoslash";

// ...

transformers: isRemoteContent ? [] : [
 // transformerTwoslash({
 //   renderer: rendererRich(),
 //   explicitTrigger: true
 // })
],

Commit the patch

pnpm patch-commit <path>

Not ideal but I don't need twoslash and it cannot be disabled right now.

@dimaMachina dimaMachina reopened this Oct 9, 2024
@danielkleinstein
Copy link

danielkleinstein commented Oct 10, 2024

@hrougier Wow, wasn't familiar with pnpm patch - thanks, this helped a lot!

Removing the twoslash dependency explicitly seems to solve a lot of random issues (including the missing CSS issue).

@dimaMachina
Copy link
Collaborator

For anyone still having this issue - Please provide a working reproduction.

Without reproduction, I cannot help with fixing this issue

@mnahkies
Copy link

I'm seeing this on this branch mnahkies/openapi-code-generator#251

(example CI failure https://github.com/mnahkies/openapi-code-generator/actions/runs/11296901214/job/31422680072)

# git clone
yarn
cd ./packages/documentation
yarn build # or yarn dev

I can try putting up a minimal reproduction tomorrow if that's better.

@dimaMachina
Copy link
Collaborator

@mnahkies you have a different error, which must be fixed with https://github.com/shuding/nextra/releases/tag/nextra%403.0.9

Can you confirm that it was fixed with new release?

@mnahkies
Copy link

@dimaMachina unfortunately that build I linked was from the commit bumping to 3.0.9 - so still an issue for me

It's a fairly vanilla typescript nextra project using nextra-theme-docs

Also sorry I thought this was the same underlying issue as the ENOTDIR thing as I was seeing that at some point alongside the error

@dimaMachina
Copy link
Collaborator

dimaMachina commented Oct 11, 2024

@mnahkies can't reproduce locally

  1. cloned your repo
  2. updated nextra/nextra-theme-docs to 3.0.9
  3. removed next-seo from theme.config
  4. refactored _meta.json to _meta.js
  5. didn't receive any errors
 ✘ dmytro@192  ~/Desktop/repros/openapi-code-generator   main ±  yarn workspace @nahkies/openapi-code-generator-documentation build
yarn workspace v1.22.17
yarn run v1.22.17
$ next build
 ⚠ Specified "redirects" will not automatically work with "output: export". See more info here: https://nextjs.org/docs/messages/export-no-custom-routes
 ⚠ Specified "redirects" will not automatically work with "output: export". See more info here: https://nextjs.org/docs/messages/export-no-custom-routes
  ▲ Next.js 14.2.13

 ✓ Linting and checking validity of types    
   Creating an optimized production build ...
 ✓ Compiled successfully
 ✓ Collecting page data    
 ✓ Generating static pages (15/15)
 ✓ Collecting build traces    
 ⚠ rewrites, redirects, and headers are not applied when exporting your application, detected (redirects). See more info here: https://nextjs.org/docs/messages/export-no-custom-routes
 ✓ Finalizing page optimization    

Route (pages)                                    Size     First Load JS
┌ ○ /                                            941 B           150 kB
├   /_app                                        0 B             145 kB
├ ● /_meta                                       241 B           145 kB
├ ○ /404                                         1.04 kB         150 kB
├ ○ /getting-started/quick-start                 6.25 kB         155 kB
├ ● /guides/_meta                                247 B           145 kB
├ ○ /guides/client-templates/typescript-angular  7.34 kB         157 kB
├ ○ /guides/client-templates/typescript-axios    7.48 kB         157 kB
├ ○ /guides/client-templates/typescript-fetch    7.58 kB         157 kB
├ ○ /guides/concepts/extract-inline-schemas      4.02 kB         153 kB
├ ○ /guides/server-templates/typescript-koa      8.53 kB         158 kB
├ ○ /overview/about                              3.95 kB         153 kB
├ ○ /overview/compatibility                      6.81 kB         156 kB
├ ○ /reference/architecture                      3.24 kB         152 kB
└ ○ /reference/cli-options                       5.44 kB         155 kB
+ First Load JS shared by all                    156 kB
  ├ chunks/framework-e82a91356956e6c8.js         44.8 kB
  ├ chunks/main-412a9cd0a25897ae.js              32.3 kB
  ├ chunks/pages/_app-bccdaae3d42700a4.js        65.8 kB
  ├ css/a509b722c23f67c3.css                     11.1 kB
  └ other shared chunks (total)                  1.75 kB

○  (Static)  prerendered as static content
●  (SSG)     prerendered as static HTML (uses getStaticProps)

✨  Done in 23.52s.
✨  Done in 23.80s.

same result when I am on renovate/all branch

@mnahkies
Copy link

I'm thoroughly confused then. More info:

$ yarn -v
4.5.0
$ node -v
v22.9.0
$ uname -r
6.10.12-200.fc40.x86_64

(I can see that your snippet above appears to be using yarn v1.22.17 although that should be irrelevant since I get the error if I invoke next build directly)

The only way I've been able to make it work so far is unconditionally returning [] from the getTransformers function modified in f9cc160

I put some logging in that function, and found it was getting called like so:

{"windowDefined":false,"isRemoteContent":false,"outputFormat":"program"}

From: https://github.com/shuding/nextra/blob/main/packages/nextra/src/server/loader.ts#L162-L169

@mnahkies
Copy link

I'm wondering if this might somehow be a Linux specific thing, here's a minimal reproduction using docker: https://github.com/mnahkies/nextra-bug-3322-reproduction

@dimaMachina
Copy link
Collaborator

@mnahkies Oops, yep, I missed to enable corepack, I can reproduce now!

@dimaMachina
Copy link
Collaborator

@mnahkies thanks for the contribution, fix is coming!

mnahkies added a commit to mnahkies/openapi-code-generator that referenced this issue Oct 12, 2024
@mnahkies
Copy link

@dimaMachina can confirm - that fixed it for me, thanks for such a quick turnaround!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment