Cache persisting fails

[and-who]

Preliminary Checks

• This issue is not a duplicate. Before opening a new issue, please search existing issues: https://github.com/gatsbyjs/gatsby/issues
• This issue is not a question, feature request, RFC, or anything other than a bug report directly related to Gatsby. Please post those things in GitHub Discussions: https://github.com/gatsbyjs/gatsby/discussions

Description

After a build is finished, gatsby tries to persist its Data in Cache Files.
This could Fail and result in the Following Error, e.g.:

warn Error persisting state: The value of "length" is out of range. It must be >= 0 && <= 2147483647. Received 2329483714

This Error Origins from the Gatsby Save State Funktion (v8.serialize())

gatsby/packages/gatsby/src/redux/save-state.js

Line 16 in eef99cc

report.warn(`Error persisting state: ${(err && err.message) || err}`)

The Problem is, that this Function uses a function to "guessSafeChunkSize"

gatsby/packages/gatsby/src/redux/persist.ts

Lines 101 to 118 in d20a74b

	export function guessSafeChunkSize(
	values: Array<[string, IGatsbyNode]> | Array<[string, IGatsbyPage]>,
	showMaxSizeWarning: boolean = false
	): number {
	// Pick a few random elements and measure their size then pick a chunk size
	// ceiling based on the worst case. Each test takes time so there's trade-off.
	// This attempts to prevent small sites with very large pages from OOMing.
	// This heuristic could still fail if it randomly grabs the smallest nodes.
	// TODO: test a few nodes per each type instead of from all nodes
	const nodesToTest = 11 // Very arbitrary number
	const valueCount = values.length
	const step = Math.max(1, Math.ceil(valueCount / nodesToTest))
	let maxSize = 0
	for (let i = 0; i < valueCount; i += step) {
	const size = v8.serialize(values[i]).length
	maxSize = Math.max(size, maxSize)
	}

and this Guess can be wrong which results in the Error mentioned above.
As one of your Developers mentioned "This heuristic could still fail if it randomly grabs the smallest nodes"
We are currently stuck with this Problem, because of our large number of nodes "step" will most likely be the same Value even when there are nodes added or removed. Therefore the values which are used to guess the Chunk Size are the same, therefore the guess will be "wrong" even after the next build.

There is a Experimental Flag "GATSBY_DISABLE_CACHE_PERSISTENCE", this is no Option, because we have a system for Incremental Builds, which relies on the Cache Data.

There was a PR which tried to make this Problem more controllable, sadly it got closed and not merged:
#29039

Reproduction Link

gatsby/packages/gatsby/src/redux/persist.ts

Lines 108 to 111 in d20a74b

	// This heuristic could still fail if it randomly grabs the smallest nodes.
	// TODO: test a few nodes per each type instead of from all nodes
	const nodesToTest = 11 // Very arbitrary number

Steps to Reproduce

Sadly this is hard to reproduce, you'll need a specific set of Nodes, so that the Heuristic picks bad Nodes to determine the Chunk Size.

This Problem occurs on our Production Environment, the Data there is multiple GB large.

Expected Result

The Caching persisting should be Fail save.
If this is not possible make the heuristic adjustable, e.g. Make it possible to change the Number of Nodes which get chosen,
or randomize the choosing of the Test Nodes a bit more, so subsequent builds wont fail.

And perhaps return an Error not a Warning, so we can detect corrupted Data.

Actual Result

The Persisting of the Cache Data fails for unlucky Data.
We have no way of controlling this.

Environment

OS: macOS 12.4
    CPU: (12) x64 Intel(R) Core(TM) i7-8850H CPU @ 2.60GHz
    Shell: 5.8 - /usr/local/bin/zsh
  Binaries:
    Node: 16.7.0 - /var/folders/p7/2kyvkfcs387899__10tmf9gh0000gp/T/yar
n--1657020894305-0.5687446657165263/node
    Yarn: 1.22.17 - /var/folders/p7/2kyvkfcs387899__10tmf9gh0000gp/T/ya
rn--1657020894305-0.5687446657165263/yarn
    npm: 7.20.3 - ~/.nvm/versions/node/v16.7.0/bin/npm
  Browsers:
    Chrome: 103.0.5060.114
    Firefox: 101.0.1
    Safari: 15.5

Config Flags

No response

[tyhopp]

Hi @and-who, thanks for the detailed issue description. What Gatsby version are you using? We no longer serialize nodes this way (they're now stored in lmdb), but pages still hit this code path.

[and-who]

Currently we are using Gatsby 3
I was also thinking about upgrading, but i saw the same code in v4 so im not sure if this will fix it.
But this would be my next Step

[imjoshin]

@and-who , while we can't promise upgrading to Gatsby 4 will fix the issue, it might see positive results. There were many changes to how the data is handled and the methods used to fetch data behind the scenes. Could you try an upgrade and see if the issue persists?

I agree that this method of caching our redux state should be revisited, nonetheless, just a matter of timing. If any fixes do come through, they will only be in Gatsby v4.

Thank you for reaching out, and thanks for using Gatsby! 💜

[github-actions]

Hiya!

This issue has gone quiet. Spooky quiet. 👻

We get a lot of issues, so we currently close issues after 60 days of inactivity. It’s been at least 20 days since the last update here.
If we missed this issue or if you want to keep it open, please reply here.
As a friendly reminder: the best way to see this issue, or any other, fixed is to open a Pull Request. Check out gatsby.dev/contribute for more information about opening PRs, triaging issues, and contributing!

Thanks for being a part of the Gatsby community! 💪💜

[and-who]

It took some Time because we have a large Project but we migrated to Gatsby4 on Prod now.
The Error did not pop up since then.

So for now it looks good.

If the Issue appears again I will come back to you.

Thanks