Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update static build workers #46705

Merged
merged 2 commits into from
Mar 2, 2023
Merged

Update static build workers #46705

Changes from 1 commit
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
d3425f7
Update static build workers
ijjk Mar 2, 2023
05ab95d
update options
ijjk Mar 2, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
29 changes: 27 additions & 2 deletions packages/next/src/lib/worker.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
import { ChildProcess } from 'child_process'
import { Worker as JestWorker } from 'next/dist/compiled/jest-worker'

type FarmOptions = ConstructorParameters<typeof JestWorker>[1]

const RESTARTED = Symbol('restarted')
Expand All @@ -24,11 +24,36 @@ export class Worker {
this._worker = undefined

const createWorker = () => {
this._worker = new JestWorker(workerPath, farmOptions) as JestWorker
this._worker = new JestWorker(workerPath, {
...farmOptions,
forkOptions: {
...farmOptions.forkOptions,
env: {
...((farmOptions.forkOptions?.env || {}) as any),
...process.env,
// we don't pass down NODE_OPTIONS as it can
// extra memory usage
NODE_OPTIONS: '',
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this mean that we're going to be stuck w/ node defaults? Is that enough for the child processes?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It should be plenty as we span a worker for each CPU by default and the workers are supposed to retry as well.

Copy link
Contributor

@0xadada 0xadada Mar 28, 2023 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ijjk if configured with experimental.cpus = 1 and if there are hundreds of thousands of pages in a site, could the main worker exit due to Out of Memory when it can't spawn multiple workers?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

more info on this from @ijjk:

We have a site that generates 208,630 pages, and we've set the env var --max_old_space_size=12288 on the main call to next build because its running on a kubernetes pod limited to 16GB memory (we need some overhead for the parent process).

We just upgraded to v13.2.4 which includes this PR, we wanted to see what effect this release had on memory usage, and now we're seeing a slightly different but interesting behavior in the workers:

// snipped of next build output
info  - Compiled successfully
info  - Collecting page data...
Static worker unexpectedly exited with code: null and signal: SIGKILL
Static worker unexpectedly exited with code: null and signal: SIGKILL
Static worker unexpectedly exited with code: null and signal: SIGTERM
warn  - Restarted collecting page data for [object Object] because it took more than 600 seconds
warn  - See more info here https://nextjs.org/docs/messages/static-page-generation-timeout
warn  - Restarted collecting page data for [object Object] because it took more than 600 seconds
info  - Generating static pages (0/208630)
Static worker unexpectedly exited with code: null and signal: SIGKILL
Static worker unexpectedly exited with code: null and signal: SIGKILL
Static worker unexpectedly exited with code: null and signal: SIGKILL
info  - Generating static pages (732/208630)
info  - Generating static pages (3750/208630)
// snipped
info  - Generating static pages (208630/208630)
info  - Finalizing page optimization...
success

but despite these errors the build ended up completing successfully.

Screen Shot 2023-04-01 at 9 47 49 AM
memory usage during next build

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Screen Shot 2023-04-01 at 10 06 37 AM
memory initially spikes, service workers are killed, and memory usage gradually increases until the process finishes

} as any,
},
}) as JestWorker
restartPromise = new Promise(
(resolve) => (resolveRestartPromise = resolve)
)

for (const worker of ((this._worker as any)._workerPool?._workers ||
[]) as {
_child: ChildProcess
}[]) {
worker._child.on('exit', (code, signal) => {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do you have docs handy for this lib/event? I'm going to look now, just curious for my own edification.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

https://nodejs.org/api/child_process.html

if (code || signal) {
console.error(
`Static worker unexpectedly exited with code: ${code} and signal: ${signal}`
)
}
})
}

this._worker.getStdout().pipe(process.stdout)
this._worker.getStderr().pipe(process.stderr)
}
Expand Down