Extending third-party schemas via directives

[zcei]

Rendered format

In short, this RFC allows third-party GraphQL APIs to be extended by Gatsby plugins. This will bring native GraphQL APIs on feature parity with custom sources.

Input on the open questions is very welcome. I'm hacking on a PoC already, but would like to get early feedback as you all are more familiar with the internal architecture of Gatsby.

I'm pretty sure there is a better approach to hide the complexity from the end-user. (e.g. whitelisting root query types via graphql-tools)

[pieh]

Hey @zcei, this looks very cool! We plan on refactoring grapqhl schema creation (probably will start next month) and most likely this we be led by @freiksenet . Of course it's ok to hack on PoC to catch any potential blockers before we start designing APIs.

Here's some links to discussion we had already - it focused mostly on gatsby generated schema and not 3rd party stitched schemas, but it's worth checking out nonetheless:

• [EPIC] Graphql schema refactor gatsby#4261
• [Discussion] Feature: GraphQL schema snapshots for all data sources to solve undefined/empty data issues gatsby#3344

Right now we are on v2 release mode, so we have lag on exciting new feature possibilities / discussions, but I'll definitely get back to You and this RFC once v2 dust settles.

[zcei]

Hej @pieh,

sure thing, I'm eager to see v2 landing soon.

I will definitely check out the discussion and contribute with my findings from building the PoC. At some point I was like "Phew, a different design to source the graph is needed, but I cannot build v3 now 😂" (In a kinda hacky way it currently works for me, would just need some more abstractions.)
So reading that you're on the same page is lovely. 💪

[freiksenet]

This aligns very well to what we discussed when I was working on graphql-source. Eg we had similar concept of to shadow nodes that are there for the processors, but don't exists for eg global node cache.

In longer term we discussed that a better control over Gatsby schema creation as a whole would be desired. However this is a good first step that doesn't change too much about how Gatsby works, while adding lots of benefits and flexibility to third-party schemas.

[zcei]

Hej @freiksenet, great to hear! I did proceed coding in the meantime and have a prototype where the third-party schema can basically only receives the set FieldsOnNodeType fields from plugins, and no Gatsby internal fields.

I think the refactoring approach that was mentioned in one of the two linked issues makes a lot of sense there, as I'm currently short-circuiting for the shadow nodes.
If you start the refactoring, I'd be happy to discuss my findings.

So if I understood you correctly this means I should proceed with this RFC, as it already adds value even before the refactoring?

[pieh]

@zcei Does your prototype add or modify any public facing gatsby APIs (setFieldsOnNodeType changes to support 3rd party types)? Only blocker I would see would be if it would need API changes (which we would then need to support even after schema refactor). Do you have your prototype available somewhere just for quick looksie, how did you approach and implemented that?

[zcei]

I fear we would need to add another hook so that directive plugin authors could properly modify the types. Currently I only have one field hardcoded extended with remark, so it's more a PoC than a real prototype.

I would be happy to discuss the refactoring first and see what APIs you would want to support there. Probably we can come up with better places to hook into than just "after types are built, before schema is merged".

Gatsby branch
Contentful starter with gatsby-transformer-remark directive imlpementation

This is super rough, so don't judge me at the code ;)

[freiksenet]

One thing I've thought off. Instead of overriding or updating types, maybe we add types?

type BlogPost {
   // blog post has field `body` with type String
   bodyMarkdown: Markdown @markdownFromField('body')
}

Alternaitvely, in resolver format.

type BlogPost {
   // blog post has field `body` with type String
   bodyMarkdown: Markdown
}

Somewhere in Gatsby we allow providing resolvers

{
   body(parent, args, context, info) {
      return GatsbyMarkdownPlugin.markdownFromString(parent.body)
   }
}

[zcei]

Yep, currently the PoC is actually extending the types, so I'm more "annotating" what the field is, and based on that it generates new fields.

[djfarly]

Hi,
just to make sure I understand all of this correctly.

Would those directives mean, that it would also be possible to add a way to download images from external graphql sources into the gatsby filesystem? (Similar to your example with a markdown transform).

If so, I don't see how this isn't much more popular 😄 Is there some way to help with anything?

[stefanoverna]

I second this. We would love to deprecate our gatsby-source-datocms plugin and just use gatsby-source-graphql without having to reinvent the wheel. The only thing that's preventing us from doing that is that gatsby-source-graphql is not compatibile with gatsby-image transformations.

We're here to help too!

[Khaledgarbaya]

Same for gatsby-contentful-source users we'll be able to share the same queries from the native GraphQL endpoint also they will get all of the Contentful goodness for free Whenever we ship a new feature in the API

[Khaledgarbaya]

Hey Folks,
Any updates on this RFC?

[freiksenet]

@Khaledgarbaya I've joined Gatsby full-time this week so my plan is to figure out a good long-term solution for modifying the gatsby schema.

[mblode]

@freiksenet Thanks for the update! I'm relatively new to Gatsby and I'm experimenting with porting my Craft CMS portfolio website to Gatsby using gatsby-source-graphql and CraftQL. I'm currently trying to figure out the optimal way of getting images from the Craft CMS server into Gatsby. Is the only way to use something like: <img src={data.featuredImage[0].thumbnail} alt={data.title} />? Or is it at all possible to use GraphQL along with gatsby-plugin-sharp and gatsby-image to get all those magic features and optimisations? Thanks!

[freiksenet]

@mblode We want to make it possible to use gatsby-plugins with data coming from 3rd party servers, currently it isn't possible, because the way data comes from there is quite different from our usual data model.

[mblode]

@freiksenet I've decided to use https://github.com/stereobooster/react-ideal-image instead with Craft's image transforms and it seems to work very well. However, what is the most common setup between a front-end and a back-end to avoid CORS policy if they are hosted on different servers? I believe React Ideal Image fetches JSON to dynamically load the image which gave me issues like: "No 'Access-Control-Allow-Origin' header is present on the requested resource". I had to use NGINX to allow all this but that seems insecure. Thanks again

[Khaledgarbaya]

Hi @mblode,

would you mind creating an issue in the Gatsby repo, to keep the discussion here focused about the RFC