Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Slow response with GetEntitlement requests #1259

Closed
jrschumacher opened this issue Aug 6, 2024 · 0 comments · Fixed by #1271
Closed

Slow response with GetEntitlement requests #1259

jrschumacher opened this issue Aug 6, 2024 · 0 comments · Fixed by #1271
Assignees
@alkalescent

Comments

@jrschumacher
Copy link
Member

When making a request to GetEntitlements the response is extremely slow. Additionally, the payload sent via the IPC exceeds the default 4Mb gRPC message size. A workaround was implemented in #1207 to enable setting a message size via configuration.

It was discovered that the root of the response time was split between overlogging and non-performant db queries [respectively 1/3 and 2/3 of the cost].

Acceptance criteria

  • remove the excessive logging in favor of a payload size or count, etc
  • investigate SQL issue and optimize performance
@alkalescent alkalescent linked a pull request Aug 7, 2024 that will close this issue
feat(core): improve entitlements performance #1271
Merged
@alkalescent alkalescent mentioned this issue Aug 9, 2024
Merged
github-merge-queue bot pushed a commit that referenced this issue Aug 12, 2024
@alkalescent @strantalis
feat(core): improve entitlements performance (#1271)
f6a1b26 
### Context:

While developing COP, we found that GetEntitlements could take around
5.5 seconds to return a response for the federal dataset. This forced us
to increase the server timeout, increase the grpc message size, and
implement caches.

After investigating the latency, we determined the two main causes:
[excess
logs](https://github.com/opentdf/platform/blob/f275e25e4f986455fc536d2c93f7e9535f8519ab/service/authorization/authorization.go#L393)
(33%) and [excess database
queries](https://github.com/opentdf/platform/blob/cc15f25af2c3e839d7ad45283b7bd298a80e8728/service/policy/db/attribute_fqn.go#L188-L195)
(66%).

### Proposed Solution:

#### Primary
1. In the case of logging subject mappings, we now log their count
instead of their content.
2. In the case of database calls, we now list attributes, list subject
mappings, and match them based on their values. The database calls were
O(n) time complexity because they were dependent on the number of
attribute values. Now the database calls are constant time. We still
loop through all the values to match their subject mappings; however we
were already doing that in the `prepareValues` values function, so the
new approach is strictly better (especially due to our ubiquitous use of
maps).

#### Rego Query Optimization:
Yet if we simply match subject mappings and attribute values, the rego
query becomes massive (65 mb). It takes 3 seconds to build (20%) and
evaluate (80%). To optimize for not only time but also space, we remove
unrelated values for each fqn/attribute pair in the rego input (unless
the attribute rule is hierarchical).

After all the optimizations, fetching entitlements using the federal
dataset now takes about 125 ms. This is a **_latency reduction of
98%_**.

resolves: #1259

---------

Co-authored-by: Sean Trantalis <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@alkalescent alkalescent

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

feat(core): improve entitlements performance opentdf/platform
2 participants
@jrschumacher @alkalescent