Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

refactor(query): optimized passwords and secrets query #3059

Merged
merged 3 commits into from
May 4, 2021
Merged

refactor(query): optimized passwords and secrets query #3059

merged 3 commits into from
May 4, 2021

Conversation

felipe-avelar
Copy link
Contributor

Signed-off-by: Felipe Avelar [email protected]

Closes #3058

Proposed Changes

  • Optimize query

I submit this contribution under the Apache-2.0 license.

@felipe-avelar
Changed query to use walk (v1)
606192c 
Signed-off-by: Felipe Avelar <[email protected]>
@felipe-avelar felipe-avelar added bug Something isn't working query New query feature labels Apr 30, 2021
@felipe-avelar felipe-avelar added this to the Queries Support milestone Apr 30, 2021
@felipe-avelar felipe-avelar self-assigned this Apr 30, 2021
@felipe-avelar felipe-avelar marked this pull request as draft April 30, 2021 16:32
@felipe-avelar
Finished query update
53b8c76 
Signed-off-by: Felipe Avelar <[email protected]>
@felipe-avelar
Corrected bug on docker search key
7866f31 
Signed-off-by: Felipe Avelar <[email protected]>
@kicsbot
Copy link
Contributor

kicsbot commented May 4, 2021

Scan submitted to Checkmarx

@kicsbot
Copy link
Contributor

kicsbot commented May 4, 2021

Logo
Checkmarx SAST - Scan Summary & Details

Cx-SAST Summary

Total of 2 vulnerabilities
High 0 High
Medium 0 Medium
Low 2 Low
Info 0 Info

Violation Summary

No policy violation found

@felipe-avelar
Copy link
Contributor Author

Using opa bench to compare, we can see the following improvement on performance:

Running kics scan -p . (kics folder)

--> New query

+-------------------------------------------+---------------+
| samples                                   |             1 |
| ns/op                                     |    7730055000 | 7.73s
| B/op                                      |    3986459192 |
| allocs/op                                 |      68875751 |
| histogram_timer_rego_query_eval_ns_75%    |    7730017400 |
| histogram_timer_rego_query_eval_ns_90%    |    7730017400 |
| histogram_timer_rego_query_eval_ns_95%    |    7730017400 |
| histogram_timer_rego_query_eval_ns_99%    |    7730017400 |
| histogram_timer_rego_query_eval_ns_99.9%  |    7730017400 |
| histogram_timer_rego_query_eval_ns_99.99% |    7730017400 |
| histogram_timer_rego_query_eval_ns_count  |          1.00 |
| histogram_timer_rego_query_eval_ns_max    |    7730017400 |
| histogram_timer_rego_query_eval_ns_mean   |    7730017400 |
| histogram_timer_rego_query_eval_ns_median |    7730017400 |
| histogram_timer_rego_query_eval_ns_min    |    7730017400 |
| histogram_timer_rego_query_eval_ns_stddev |             0 |
+-------------------------------------------+---------------+

--> Old query

+-------------------------------------------+---------------+
| samples                                   |             1 |
| ns/op                                     |   53299056500 | 53.30s
| B/op                                      |   18989655952 |
| allocs/op                                 |     329803520 |
| histogram_timer_rego_query_eval_ns_75%    |   53299000800 |
| histogram_timer_rego_query_eval_ns_90%    |   53299000800 |
| histogram_timer_rego_query_eval_ns_95%    |   53299000800 |
| histogram_timer_rego_query_eval_ns_99%    |   53299000800 |
| histogram_timer_rego_query_eval_ns_99.9%  |   53299000800 |
| histogram_timer_rego_query_eval_ns_99.99% |   53299000800 |
| histogram_timer_rego_query_eval_ns_count  |          1.00 |
| histogram_timer_rego_query_eval_ns_max    |   53299000800 |
| histogram_timer_rego_query_eval_ns_mean   |   53299000800 |
| histogram_timer_rego_query_eval_ns_median |   53299000800 |
| histogram_timer_rego_query_eval_ns_min    |   53299000800 |
| histogram_timer_rego_query_eval_ns_stddev |             0 |
+-------------------------------------------+---------------+

-> Larger Repo
-->New query

+-------------------------------------------+---------------+
| samples                                   |             1 |
| ns/op                                     |   92831469700 |92.83s
| B/op                                      |   33928882208 |
| allocs/op                                 |     593658279 |
| histogram_timer_rego_query_eval_ns_75%    |   92831384400 |
| histogram_timer_rego_query_eval_ns_90%    |   92831384400 |
| histogram_timer_rego_query_eval_ns_95%    |   92831384400 |
| histogram_timer_rego_query_eval_ns_99%    |   92831384400 |
| histogram_timer_rego_query_eval_ns_99.9%  |   92831384400 |
| histogram_timer_rego_query_eval_ns_99.99% |   92831384400 |
| histogram_timer_rego_query_eval_ns_count  |          1.00 |
| histogram_timer_rego_query_eval_ns_max    |   92831384400 |
| histogram_timer_rego_query_eval_ns_mean   |   92831384400 |
| histogram_timer_rego_query_eval_ns_median |   92831384400 |
| histogram_timer_rego_query_eval_ns_min    |   92831384400 |
| histogram_timer_rego_query_eval_ns_stddev |             0 |
+-------------------------------------------+---------------+

--> Old query

+-------------------------------------------+---------------+
| samples                                   |             1 |
| ns/op                                     |  777429701200 |777.429s
| B/op                                      |  260060757176 |
| allocs/op                                 |    4518867209 |
| histogram_timer_rego_query_eval_ns_75%    |  777429622100 |
| histogram_timer_rego_query_eval_ns_90%    |  777429622100 |
| histogram_timer_rego_query_eval_ns_95%    |  777429622100 |
| histogram_timer_rego_query_eval_ns_99%    |  777429622100 |
| histogram_timer_rego_query_eval_ns_99.9%  |  777429622100 |
| histogram_timer_rego_query_eval_ns_99.99% |  777429622100 |
| histogram_timer_rego_query_eval_ns_count  |          1.00 |
| histogram_timer_rego_query_eval_ns_max    |  777429622100 |
| histogram_timer_rego_query_eval_ns_mean   |  777429622100 |
| histogram_timer_rego_query_eval_ns_median |  777429622100 |
| histogram_timer_rego_query_eval_ns_min    |  777429622100 |
| histogram_timer_rego_query_eval_ns_stddev |             0 |
+-------------------------------------------+---------------+

@felipe-avelar felipe-avelar marked this pull request as ready for review May 4, 2021 09:31
rogeriopeixotocx
rogeriopeixotocx approved these changes May 4, 2021
View reviewed changes
Copy link
Contributor

@rogeriopeixotocx rogeriopeixotocx left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LCTM

@rogeriopeixotocx rogeriopeixotocx merged commit f134433 into master May 4, 2021
@rogeriopeixotocx rogeriopeixotocx deleted the bugfix/optimize-secrets-query branch May 4, 2021 10:16
@felipe-avelar felipe-avelar changed the title fix(query): optimized passwords and secrets query enhancement(query): optimized passwords and secrets query May 5, 2021
@felipe-avelar felipe-avelar changed the title enhancement(query): optimized passwords and secrets query refactor(query): optimized passwords and secrets query May 5, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@rogeriopeixotocx rogeriopeixotocx rogeriopeixotocx approved these changes

@joaoReigota1 joaoReigota1 Awaiting requested review from joaoReigota1

@rafaela-soares rafaela-soares Awaiting requested review from rafaela-soares

Assignees

@felipe-avelar felipe-avelar

Labels
bug Something isn't working query New query feature
Projects
None yet
Milestone
Queries Support
Development

Successfully merging this pull request may close these issues.

Passwords and Secrets timeout
3 participants
@felipe-avelar @kicsbot @rogeriopeixotocx