Add an API to load some data at runtime

[mathieudutour]

Summary

Add an API to load some specific queries from graphql at runtime.

I can imagine a new plugin API to expose a query with an id:

createStaticQuery('my query id', graphql`
  load {
    something
  }
`)

and a client-side method to fetch it:

import { fetchStaticQuery } from 'gatsby'

const data = await fetchStaticQuery('my query id')

This is different from useStaticQuery because the client can dynamically fetch any static query, effectively creating a "static API".

Expose window.___loader as an API so that React code can leverage its functionality (or perhaps only part of it)

Motivation

I wanted to create a "stacked" page navigation like this: https://twitter.com/aravindballa/status/1263112947670245377?s=20

That means loading data from notes at runtime (because it'd be too complex to generate all possible permutations at build time).

Workaround

I had a stupid crazy idea but had an epiphany that the data I wanted was already accessible: it's the pages data. So I found window.___loader and managed to dynamically load the data from a page (like gatsby is doing with Link). The code is here: https://github.com/mathieudutour/gatsby-n-roamresearch/tree/master/packages/react-stacked-pages-hook.

While this works (and a quick and dirty first step could be to expose window.___loader as an official API), I believe my proposal is more generic, powerful, and unlocks more use cases.

References

https://twitter.com/kylemathews/status/1263254949816709120?s=20

[pieh]

FIrst of all - the stacked pages is pretty slick! Nice job!

I can imagine a new plugin API to expose a query with an id:

createStaticQuery('my query id', graphql`
  load {
    something
  }
`)

I'd like to clarify - what's the environemnt/context where this would be used - would this be in frontend code or in gatsby-node?

[mathieudutour]

In gatsby-node. We define the “static API” beforehand, the id being its “endpoint”

[pieh]

If that's the case - I do wonder if this need to be in gatsby core, because I think we already have tools needed in userland / plugin API to make it work without cramming this to core:

• createStaticQuery is really just a wrapper around graphql function that you can already execute in gatsby-node (granted in not all API hooks - but it's available in onPostBuild / createPages). Other part of it is saving result to some json file in public dir - this can all be done in userland code
• fetchStaticQuery is really just a fetch so it can be implemented in userland as well - thing that it does over fetch is converting query id to actual path (it could be smart to keep track of previous fetches to not even request same static query result again)

[pieh]

I'm not saying we completely wouldn't want it in core (eventually) - we are working on something that while not exactly touch on this subject, can be somewhat related - #18689 (comment) - particularly

how this could be used to avoid some semi-global data from either being duplicated in data for all pages or ending up as true global

for above case - we could actually be solved with using createStaticQuery there as well - just consume it in different way - not "lazily" / load on demand, but rather let loader load those as part of page loading (I know that it's not what you want here - you actually want it lazy, but there can be multiple ways to consume it)

What I'm getting on here is - we want stuff like this to have coherent API and not duplicate very similar APIs multiple times - with potential solution being done in user land today (at least in meantime) I think there is no "rush" to implement this in core.

What are your thoughts on that?

[mathieudutour]

Oh yeah for sure a consistent API is key.

Agree there is no rush, I’m happy with my workaround for now (as I don’t expect ___loader to go anywhere any time soon anyway).

Feel free to consider this as a use case for the API you have in mind then 🙂
I do believe being able to lazily load some data could be really powerful

[pieh]

I do believe being able to lazily load some data could be really powerful

For sure! There are also other possibilities here (but those might not fit the use case for stacked pages):

Like you could have @lazy directive in page/static queries that would chunk part of query result to separate files and let users do something like this in react:

export const pageQuery = graphql`
  {
    someCriticalData {
      fieldsForCriticalData
    }
    someNotCriticalData @lazy {
      fields
    }
  }
`

and inside component you could do

data.someNotCriticalData().then(lazyData => { /* do stuff */})

We are exploring lot of options like this. They might not fit this use case because powerful part of your proposal is running them in gatsby-node which means you have a lot more flexibility when it comes to query construction or even deciding how to chunk those (creating results for each possible page that could be "stacked" might not really be possible with just @lazy apporoach).

[github-actions]

Hiya!

This issue has gone quiet. Spooky quiet. 👻

We get a lot of issues, so we currently close issues after 30 days of inactivity. It’s been at least 20 days since the last update here.
If we missed this issue or if you want to keep it open, please reply here. You can also add the label "not stale" to keep this issue open!
As a friendly reminder: the best way to see this issue, or any other, fixed is to open a Pull Request. Check out gatsby.dev/contribute for more information about opening PRs, triaging issues, and contributing!

Thanks for being a part of the Gatsby community! 💪💜

[github-actions]

Hey again!

It’s been 30 days since anything happened on this issue, so our friendly neighborhood robot (that’s me!) is going to close it.
Please keep in mind that I’m only a robot, so if I’ve closed this issue in error, I’m HUMAN_EMOTION_SORRY. Please feel free to reopen this issue or create a new one if you need anything else.
As a friendly reminder: the best way to see this issue, or any other, fixed is to open a Pull Request. Check out gatsby.dev/contribute for more information about opening PRs, triaging issues, and contributing!

Thanks again for being part of the Gatsby community! 💪💜