refactor(conditional-page-build): tracking PR #29140
Closed
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
gatsbot
bot
added
the
status: triage needed
pieh
added
topic: scaling builds
and removed
status: triage needed
pieh
commented
Jan 22, 2021
docs/docs/conditional-page-builds.md
Outdated
Comment on lines
50
to
61
|
|
||
| - To enable this build option you will need to set an environment variable, which requires access to do so in your build environment. | ||
|
|
||
| - This feature is not available with `gatsby develop`. | ||
|
|
||
| - You should not try to use this flag alongside Incremental Builds in Gatsby Cloud, as it uses a different process and may conflict with it. | ||
|
|
||
| - You will need to persist the `.cache` and `public` directories between builds. This allows for comparisons and reuse of previously built files. If `.cache` directory was not persisted then a full build will be triggered. If `public` directory was not persisted then you might experience failing builds or builds that are missing certain assets. | ||
|
|
||
| - Any code changes (templates, components, source handling, new plugins etc) will prompt the creation of a new webpack compilation hash and trigger a full build. | ||
|
|
||
| Note: When using the `GATSBY_EXPERIMENTAL_PAGE_BUILD_ON_DATA_CHANGES` flag it is important to do so consistently when building your project. Otherwise, the cache will be cleared and the necessary data for comparison will no longer be available, removing the ability to check for incremental data changes. |
There was a problem hiding this comment.
those changes will need to be extracted to separate PR and only applied when stable version including code changes is released
pieh
force-pushed
the
track-html-writes
branch
from
199640c to
d674bd8
Compare
pieh
commented
Jan 22, 2021
Comment on lines
41
to
43
| if (htmlFile.isDeleted || !state.pages.has(path)) { | ||
| // FIXME: checking pages state here because pages are not persisted | ||
| // and because of that `isDeleted` might not be set ... |
There was a problem hiding this comment.
pieh
force-pushed
the
track-html-writes
branch
4 times, most recently
from
72eddde to
7fdf75f
Compare
| @@ -77,6 +80,20 @@ const getPageData = pagePath => { | |||
| } | |||
| } | |||
|
|
|||
| // TODO: verify if memoization is safe given that results can change | |||
| const getStaticQueryResult = memoize(hash => { | |||
There was a problem hiding this comment.
this would be safe for single runs but not in inc builds?
There was a problem hiding this comment.
Yeah, that's the actual thing to verify ;) depends if ssr bundle is "reloaded" or just kept in memory
| @@ -152,6 +162,12 @@ const renderHTMLQueue = async ( | |||
| paths: pageSegment, | |||
| }) | |||
|
|
|||
| // is this code path used for dev ssr? | |||
| store.dispatch({ | |||
There was a problem hiding this comment.
no it has it's own long-running worker which it sends calls to
There was a problem hiding this comment.
That's good, but I will add test case for it with DEV_SSR enabled to make sure of that (+ shield from potential regressions in future)
pieh
added
the
breaking change
pieh
force-pushed
the
track-html-writes
branch
2 times, most recently
from
91ff577 to
2a15b3d
Compare
pieh
force-pushed
the
track-html-writes
branch
from
da20685 to
a3d2497
Compare
pieh
changed the base branch from
master
to
no-fs-in-ssr-no-static-queries-in-ssr-bundle
pieh
changed the title
pieh
changed the base branch from
no-fs-in-ssr-no-static-queries-in-ssr-bundle
to
master
pieh
force-pushed
the
track-html-writes
branch
9 times, most recently
from
5efa4ae to
4744d83
Compare
pieh
force-pushed
the
track-html-writes
branch
from
4744d83 to
5a0a714
Compare
|
This tracking PR served it's purpose and all that actually ment to be tracked with this PR was merged, so it's to close this one |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
This is tracking branch for moving
GATSBY_EXPERIMENTAL_PAGE_BUILD_ON_DATA_CHANGESfrom experimental to stableChanges are being plucked from this massive PR into smaller ones:
Already merged:
refactor(html-ssr): remove fs from ssr bundle, remove static queries from ssr bundlerefactor(conditional-page-build): explicitly persist status of generated html files(this just refactor currentGATSBY_EXPERIMENTAL_PAGE_BUILD_ON_DATA_CHANGESto structure that will allow us to track additional things and remove some problems of having to constatly useGATSBY_EXPERIMENTAL_PAGE_BUILD_ON_DATA_CHANGESand not switch between build and develop)feat(conditional-page-builds): track ssr compilation hashfeat(conditional-page-builds): track static query resultsfix(conditional-page-builds): track page data and not page querychore(build): move html generation to separate functionReady to review (they are serial - each next one builds on previous and target previous PR branch, so those should be merged in order):
feat(conditional-page-builds): make it defaultFuture PRs:
fs,http,child_processnode builtin modules in SSR bundle and in case of use of those in ssr generating, mark it's not safe to use conditional page builds and fallback to building every page ( :( ). This is somewhat waiting for webpack@5 bump, as it's not straight forward to do in webpack@4 and there are changes in webpack@5 that any efforts made for webpack@4 might be rendered voidrequire.cachefrom ssr renderer if it changedsetPageDataaction and types