Use StaleWhileRevalidate for page-data files

[vtenfys]

Description

Use StaleWhileRevalidate for page-data for better performance and initial renders

Related Issues

Fixes #19207

[KyleAMathews]

Another option I was just researching is network first but with a timeout. Ideally we do get the latest data — we just don't want to wait that long for that because our top priority is consistently fast loading followed by strong consistency of data.

This issue has some sample code GoogleChrome/workbox#1738 (comment)

Perhaps a networkTimeout of 0.75 seconds? That would enable us to always render under a second.

[vtenfys]

Ideally we do get the latest data

Technically we should keep the page-data fetch logic the same as the logic for getting page bundles, since the data may change shape or physical appearance between two deploys of a site - so I think whichever we choose, we should use the same approach for JS/CSS as well. Thoughts?

[sidharthachatterjee]

Technically we should keep the page-data fetch logic the same as the logic for getting page bundles

I agree with this for two reasons

• A different page bundle might expect a different data shape and a mismatch could break the site during rehydration
• Agree that this simplifies debugging

[sidharthachatterjee]

Thanks, Dave!

[KyleAMathews]

Technically we should keep the page-data fetch logic the same as the logic for getting page bundles, since the data may change shape or physical appearance between two deploys of a site

Did we ever start refetching page (and app) bundles on a regular basis to check if they've been updated?

The big downside to this PR is that now page data won't be updated until the second time you visit a page as the browser will first get the cached page (however old it is) & then while the SW gets the newer version, the browser won't.

Also... on data changing shape. With the webpack compilation hash now not being in the page-data.json — how do we know what version of page data we're working with? Someone could have cached a version weeks ago and then the page component changed & we'd still try to use it which would throw an error...

This PR might need reverted.

[21c-HK]

Quoting @KyleAMathews:

The big downside to this PR is that now page data won't be updated until the second time you visit a page as the browser will first get the cached page (however old it is) & then while the SW gets the newer version, the browser won't.

Also... on data changing shape. With the webpack compilation hash now not being in the page-data.json — how do we know what version of page data we're working with? Someone could have cached a version weeks ago and then the page component changed & we'd still try to use it which would throw an error...

This PR might need reverted.

That is exactly the problem with this PR. This PR must be reverted as it introduces extremely hard to reproduce caching bugs as explained here that ruin the user experience (by suggesting to the user that the web site is completely broken). This PR even contradicts the official documentation:

Similar to HTML files, the JSON files in the public/page-data/ directory should never be cached by the browser. In fact, it’s possible for these files to be updated even without doing a redeploy of your site. Because of this, browsers should be instructed to check on every request if the data in your application has changed.

This PR basically kills any kind of caching reliability for web sites in production, so this cannot be the right default. Gatsby feels so fast because of its aggressive caching and preloading, which is only employable if you can expect it to be reliable, but this PR makes Gatsby's caching unreliable.

[21c-HK]

I clicked on a link to https://www.gatsbyjs.org/ today, which showed a blank (black) page. So there you have it. This caching bug even shows up on your own web site.