[graphql] query costing #14463
[graphql] query costing #14463
Conversation
|
The latest updates on your projects. Learn more about Vercel for Git ↗︎
4 Ignored Deployments
|
wlmyng
force-pushed
the
graphql-query-costing
branch
from
1a2c95f to
1be43e1
Compare
wlmyng
force-pushed
the
graphql-query-costing
branch
from
1be43e1 to
45b69af
Compare
| let re = Regex::new(r"(LIMIT\s+)\$(\d+)") | ||
| .map_err(|e| crate::error::Error::Internal(format!("Failed create valid regex: {}", e)))?; |
There was a problem hiding this comment.
- If we need Regexes like this, we should be able to create them once (statically) and then re-use them.
- Setting the limit to
DEFAULT_PAGE_SIZEkind of defeats the point of this check because it means that someone can create a query that requests 1000x the default page size, and we'll think it costs as much as a default page. I'm fine with us incrementally building towards the costing algorithm, but do we have a plan for how to get the query with all the parameters in place? Without that, this form of costing is not going to be effective.
There was a problem hiding this comment.
- ah thanks, I'm not super familiar with regex but will look into that
- and yes follow up PR to apply the correct bindings. Although on this note, does it make sense for us to limit to default page size or allow users to provide an arbitrary limit? I assumed the former (altho the code doesn't do the capping today)
There was a problem hiding this comment.
But yes the true cost and estimated cost here are a bit off rn due to replacing the placeholder value with these arbitrary values
| E: From<diesel::result::Error> + std::error::Error + Send + 'static, | ||
| T: Send + 'static, | ||
| { | ||
| let max_db_query_cost = self.limits.max_db_query_cost; |
There was a problem hiding this comment.
This limit applies to all queries that are run as part of a single request, but currently it looks like we're judging the cost of each transaction individually.
There was a problem hiding this comment.
This was a q I had posed to @oxade , and I think I was convinced by the explanation (unless I misunderstood it) that the node and complexity limiting already cover the whole request, so the query cost limiting is just to make sure that each individual db operation is not overly complex
There was a problem hiding this comment.
Ah okay, my recollection of past discussions was that this step was also intended to cover the whole request, so that each successive cost check (node + depth, explain, timeout) is successively more rigorous, because otherwise, a bad actor can take advantage of a discrepancy between our least and most expensive node to compute:
If we set a local explain limit (like in this PR), it has to be generous enough to run a sensible query involving our cheaper nodes, but then the bad actor can fill up a query with really expensive nodes to query (like the analytics queries for explorer, or the APY query), and overload the system. If we set a global limit, we don't have that problem.
Description
Now that we've split out query building from execution, we can run a cost analysis through db explain, and stop early if it exceeds query cost limits set in config
Test Plan
Manual testing, making sure existing tests pass
If your changes are not user-facing and not a breaking change, you can skip the following section. Otherwise, please indicate what changed, and then add to the Release Notes section as highlighted during the release process.
Type of Change (Check all that apply)
Release notes