Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Expose document representation of sub-query request within GraphQLDataSourceProcessOptions so that it is available to RemoteGraphQLDataSource.process and RemoteGraphQLDataSource.willSendRequest #1878

Merged
merged 2 commits into from
Jul 1, 2022

Conversation

theJC
Copy link
Contributor

@theJC theJC commented May 21, 2022

Fixes #1352

Provides access to the AST so that customer code processing a subgraph query no longer has to waste cycles re-parsing it back into a document when needed.

@netlify
Copy link

netlify bot commented May 21, 2022

👷 Deploy request for apollo-federation-docs pending review.

Visit the deploys page to approve it

Name Link
🔨 Latest commit be04a21

@codesandbox-ci
Copy link

codesandbox-ci bot commented May 21, 2022

This pull request is automatically built and testable in CodeSandbox.

To see build info of the built libraries, click here or the icon next to each commit SHA.

@theJC theJC changed the title feat: Expose sub-query DocumentNode to RemoteGraphQLDataSource#willSendRequest hook feat: Expose sub-query DocumentNode to RemoteGraphQLDataSource#willSendRequest and RemoteGraphQLDataSource.process May 21, 2022
@theJC theJC changed the title feat: Expose sub-query DocumentNode to RemoteGraphQLDataSource#willSendRequest and RemoteGraphQLDataSource.process feat: Expose sub-query DocumentNode to RemoteGraphQLDataSource.process (and RemoteGraphQLDataSource#willSendRequest) May 21, 2022
@pcmanus
Copy link
Contributor

pcmanus commented May 23, 2022

I'm personally ok with adding this. The only thing I'd maybe suggest would be to make operationDocumentNode optional within FetchNode, so as to make it clear it is redundant information that is provided mostly opportunistically. I could see some other tool around query plans not wanting to bother filling in that field for instance (this will also make plans bigger in memory, and this is probably not an issue, but keeping it easy to later add an option to disable this doesn't seem horrible either).

@theJC theJC force-pushed the exposeSubqueryDocumentNode branch from 179fc9e to fc5ccc1 Compare May 29, 2022 03:54
@theJC
Copy link
Contributor Author

theJC commented May 29, 2022

Thanks for the feedback, it has been adjusted to be optional.

@pcmanus -- Do you have any guidance on the best way to approach updating the tests? Are we best served by adding the representation of the documentNode into all the query plan results in build-query-plan.feature?

https://app.circleci.com/pipelines/github/apollographql/federation/8610/workflows/bf18dcef-c4cf-45e4-8ec0-b592a80f8e24/jobs/52912?invite=true#step-107-8795

@pcmanus
Copy link
Contributor

pcmanus commented Jun 2, 2022

@theJC As the patch stand, I don't know of other option than adding the representations all over in build-query-plan.feature (but to be honest, my knowledge of cucumber, used by those test, is weak). I do get it's annoying, both now and frankly moving forward since any future change will now have to be done in 2 places; and the value is unclear.

That said, following on my previous suggestion, one option might be to do "add an option to disable this" now. This would mean adding some new options argument to the QueryPlanner ctor, which currently has no options (but we'll likely have need for other options soon enough, so that's fine imo). If populating the new field can be disabled, then it could be used that to disable it for all the cucumber tests. It'd make sense in that case to have a dedicated non-cucumber unit-test that target this new addition, but at least the existing cucumber tests won't have to be updated.

Personally, I kind of prefer that later option because as said before, I suspect we'll have some use case at some point where generated this somewhat redundant info is not desirable. But I won't object to just updating all the cucumber tests either.

@theJC theJC force-pushed the exposeSubqueryDocumentNode branch from a891c2e to a416627 Compare June 17, 2022 21:24
@theJC
Copy link
Contributor Author

theJC commented Jun 18, 2022

@pcmanus I believe I've addressed your valuable input, let me know if there is anything else needed to make this PR mergeable.

Thanks again for your help,
Jon

Copy link
Contributor

@pcmanus pcmanus left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry the delay. Patch looks great, thanks, though I'm afraid it needs rebasing (one of the reason is that the buildPlan.ts file has been reorganised a bit, which confuses the diff algorithm). It'd be awesome if you could:

  1. rebase it.
  2. add a quick line in query-planner/CHANGELOG.md, and copy-paste it also in gateway-js/CHANGELOG.md.

and then we can ship it.

@theJC theJC force-pushed the exposeSubqueryDocumentNode branch from 7c96e8e to fa69b13 Compare July 1, 2022 04:03
@theJC theJC changed the title feat: Expose sub-query DocumentNode to RemoteGraphQLDataSource.process (and RemoteGraphQLDataSource#willSendRequest) feat: Expose document representation of sub-query request within GraphQLDataSourceProcessOptions so that it is available to RemoteGraphQLDataSource.process and RemoteGraphQLDataSource.willSendRequest Jul 1, 2022
@theJC
Copy link
Contributor Author

theJC commented Jul 1, 2022

Should be all set @pcmanus

@pcmanus pcmanus merged commit 77f6d48 into apollographql:main Jul 1, 2022
@pcmanus
Copy link
Contributor

pcmanus commented Jul 1, 2022

Perfect, thanks a lot of the contribution!

pcmanus pushed a commit to pcmanus/federation that referenced this pull request Nov 2, 2022
Since apollographql#1878 (added in 2.1.0), the full AST of subgraph fetch operations
has been store in query plans. However, this is not used by the gateway
by default, while it noticeably increases the size of query plans,
leading to either a less efficient query plan cache (if its size is
limited) or increased memory consumption/pressue (if the cache is given
plenty of size).

This patch disable this option by default. Users that explicitely
need to access the subgraph fetch ASTs can still make the trade-off
of additional memory pressure by manually using the
`GatewayConfig.queryPlannerConfig.exposeDocumentNodeInFetchNode` config
option.

Fixes apollographql#2227.
pcmanus pushed a commit that referenced this pull request Nov 2, 2022
Since #1878 (added in 2.1.0), the full AST of subgraph fetch operations
has been store in query plans. However, this is not used by the gateway
by default, while it noticeably increases the size of query plans,
leading to either a less efficient query plan cache (if its size is
limited) or increased memory consumption/pressue (if the cache is given
plenty of size).

This patch disable this option by default. Users that explicitely
need to access the subgraph fetch ASTs can still make the trade-off
of additional memory pressure by manually using the
`GatewayConfig.queryPlannerConfig.exposeDocumentNodeInFetchNode` config
option.

Fixes #2227
@pcmanus
Copy link
Contributor

pcmanus commented Nov 2, 2022

Hi @theJC. I wanted to let you know that we're planning to switch this option to false by default in the upcoming 2.2 release (see #2227/#2230), so you would have to manually opt-in as you upgrade if you still rely on this. Sorry for the inconvenience.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Expose sub-query DocumentNode to RemoteGraphQLDataSource#willSendRequest hook
2 participants