feat(gatsby): Webpack API for dependencies

[pieh]

Description

This PR adds couple of APIs that will allow adding arbitrary "code" modules to pages via graphql data layer. Make sure to check limitations section for current state of this PR

context.pageModel.setModule

Function available in graphql resolvers (can be implemented via createSchemaCustomization / createResolvers hooks).

Signature:

context.pageModel.setModule({
  source: string,
  type?: "default" | "named" | "namespace",
  importName?: string
}): string

Arguments

• source: Absolute path to module to be added. If user wants to add some package - require.resolve should be used to get absolute path
• type: Defines type of import:
  • default - import A from "b"
  • named - import { A } from "b"
  • namespace - import * as A from "b"
• importName: only used when type is set to named - specifies named import to use

Return value

context.pageModel.setModule returns string identifier that later will be used to get instance of a module in frontend code. You will want to return this identifier in resolver where you implement context.pageModel.setModule function.

getModule

This is frontend code function used to get instance of module. Function is imported from gatsby:

import { getModule } from "gatsby"

Signature

getModule(moduleId: string): any

Arguments

• moduleId: getModule expect module identifier (returned by context.pageModule.setModule) as argument. Module identifier is purposely not easily guessable (similar to id in nodes)

Return value

Actual module - this can be React component or any kind of arbitrary code.

Example usage:

Check demo site - https://github.com/pieh/query-webpack-modules-demo-site/tree/0bba2e2158473689850fa9b6112c26dc6f6c8461 . In particular:

• createSchemaCustomization hook that converts component field in data ( for example https://github.com/pieh/query-webpack-modules-demo-site/blob/0bba2e2158473689850fa9b6112c26dc6f6c8461/content/hello-world/index.json5 ) to module identifiers of actual components implemented in https://github.com/pieh/query-webpack-modules-demo-site/tree/0bba2e2158473689850fa9b6112c26dc6f6c8461/src/components/page-builder.
• getModule usage in the frontend

Limitations (currently)

• As-is this PR supports using new APIs only in page queries. To support static queries we will need feat(gatsby): Load static queries by the Gatsby runtime instead of webpack #23837 to be finished and merged. Static queries are now supported, not yet very much testing done as there can be surprising chains of things (static query adds a module which has static query and this can repeat over and over again)
• Hot data updates in gatsby develop are not working properly when new modules are added to a page - user does need to manually refresh browser to load new modules ( To handle this we will need feat(gatsby): Instrument partial writes to page data #24808 + some additional refactor on top ). EDIT: with state machine in place this is almost done, there are few more timing issues to solve (making long running process rewrite modules at proper time slots, we will need to put bit more things into state machine) this should be handled now. Due to various timing difficulties it might still not be 100% reliable (there are 2 levels - timing in node process, which is now nicely handled by state machine, but there is also timing in runtime - we receive webpack HMR updates and data updates without sync, so there is a code that tries to account for that (dev-loader changes that tries to wait for webpack updates)
• This PR doesn't implement public facing registerModule action that would allow users/plugins to stabilize webpack builds (add module to bundle even if not used currently so we don't have to rewrite whole lot of html files because of webpack hash changes)

Trying it out

I published gatsby@query-webpack-modules that includes those changes - feel free to take it for a spin and let me know what you think

Documentation

❌ This draft PR lacks any documentation related to added new APIs.

Related Issues

Related to #18689

[imarenco]

Thanks for contribution, We test that, and work :)

[gatsby-cloud-staging]

Gatsby Cloud Build Report

gatsby-master

🎉 Your build was successful! See the Deploy preview here.

Build Details

View the build logs here.

🕐 Build time: 2m

Performance

Lighthouse report

Metric	Score
Performance	💚 97
Accessibility	🔶 87
Best Practices	💚 93
SEO	🔶 73

🔗 View full report

[rasmuswoelk]

This is awesome 🚀

Tested and working! 💪🏼 Looking forward for this to be merged in!

Check it out on https://samhaeng.com (using wordpress-source-gatsby, advanced custom fields and flexible content)

[gatsby-cloud-staging]

Gatsby Cloud Build Report

gatsby
🎉 Your build was successful! See the Deploy preview here.

Build Details

View the build logs here.

🕐 Build time: 17m

Performance

Lighthouse report

Metric	Score
Performance	💚 91
Accessibility	💚 100
Best Practices	💚 100
SEO	🔶 76

🔗 View full report

[gatsby-cloud]

Gatsby Cloud Build Report

client-only-paths

🎉 Your build was successful! See the Deploy preview here.

Build Details

View the build logs here.

🕐 Build time: 19s

Performance

Lighthouse report

Metric	Score
Performance	💚 100
Accessibility	🔶 85
Best Practices	💚 100
SEO	🔶 70

🔗 View full report

[gatsby-cloud]

Gatsby Cloud Build Report

using-reach-skip-nav

🎉 Your build was successful! See the Deploy preview here.

Build Details

View the build logs here.

🕐 Build time: 20s

Performance

Lighthouse report

Metric	Score
Performance	💚 100
Accessibility	💚 100
Best Practices	💚 100
SEO	🔶 82

🔗 View full report

[rasmuswoelk]

Any news? This feature is all I want for Christmas 😆🤞🏼🎅🏼

[pieh]

@rasmuswoelk Sorry, no news on this :(

This is unfortunately postponed for now due to refactors that aim to speed up time time it takes to start up development server and/or improve development experience taking priority. Some of those would be much harder to do if we got this feature in first (this feature does make a lot of things more complicated which would make it much harder for us to do those refactors).

[rasmuswoelk]

Hey @pieh 👋

No worries – Thanks for the update! 🙌🏼

I'm just eager to be able to use this feature 👍 (I'm building a large page builder based site that could benefit a lot from this feature)

Do you think it would be possible to merge the changes from the latest Gatsby version into this draft?

Thank you so much for your work. It's greatly appreciated!

[lunelson]

I am also excited for this feature to be in Gatsby core, but @rasmuswoelk if you need a page-builder capability now, this package—based on my experiments so far—works very well right now (it is a part of the WP GraphQL Gutenberg project):

https://github.com/pristas-peter/gatsby-plugin-graphql-component

[rasmuswoelk]

@lunelson This looks awesome! I will try it out soon 👌🏼

Thank you for sharing this! 🙌🏼

[Grsmto]

development server and/or improve development experience taking priority

dev experience taking priority over end user experience? 😕 This should be merged!

[manuelJung]

@pieh any updates on this?

[wardpeet]

This has been on hold. We've been focusing on the development experience with lazy images, query on-demand and ssr in develop. Currently, our focus is to speed up builds.

This feature has been moved to the backlog.

[lnikell]

Just out of curiosity, is there are any ETA when that PR could reach the stable version of Gatsby?

[rshackleton]

This has been on hold. We've been focusing on the development experience with lazy images, query on-demand and ssr in develop. Currently, our focus is to speed up builds.

This feature has been moved to the backlog.

Faster builds are nice, but shipping too much code to the end-user is a serious issue for client-side performance and PSI scores.

How likely is it that this PR or others related to lazy loading components, better code splitting, etc, will be brought in scope in the near future?

[SimonKlausLudwig]

Is there any update on this issue ?

[coreyward]

Hey @wardpeet + @pieh, any chance that this might get some time again now that SSR has shipped? Maintaining performance with localization without having to remove static queries is tough, and this would help a lot.

[phil-lgr]

Now that this is closed, is there another issue/discussion we can follow for this topic? @LekoArts Thanks

[LekoArts]

We won't continue work on this feature, but you can expect an RFC about Partial Hydration support in the future which will help y'all ship less JS to the client.