Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make search for vulnerabilities faster #955

Merged
merged 10 commits into from
Oct 20, 2022
Merged

Make search for vulnerabilities faster #955

merged 10 commits into from
Oct 20, 2022

Conversation

pombredanne
Copy link
Member

@pombredanne pombredanne commented Oct 18, 2022

This is a PR for #954 and attempts to make some small and critically needed adjustments for performance in the web UI.

Some approaches:

  1. Cascade queries from exact to approximate searches to avoid full table scans in all cases. This is a band-aid for now. The proper solution will likely require using full text search instead.
  2. Avoid iceberg queries with "prefetch related" to limit the number of queries that are needed in the UI
  3. Do not recreate querysets from scratch but instead allow these to be chained for simpler and correct code.
  4. Remove extra details from the vulnerability pacge: each package was further listing its related vulnerabilities creating an iceberg query.
  5. Enable the django-debug-toolbar with a setting to easily profile queries on demand.

Signed-off-by: Philippe Ombredanne [email protected]

@pombredanne pombredanne marked this pull request as draft October 18, 2022 14:50
tdruez and others added 8 commits October 18, 2022 21:36
Signed-off-by: Philippe Ombredanne <[email protected]>
Signed-off-by: Philippe Ombredanne <[email protected]>
Signed-off-by: Philippe Ombredanne <[email protected]>
And chain querysets.

Signed-off-by: Philippe Ombredanne <[email protected]>
And streamline template

Signed-off-by: Philippe Ombredanne <[email protected]>
Signed-off-by: Philippe Ombredanne <[email protected]>
@pombredanne pombredanne marked this pull request as ready for review October 20, 2022 11:47
@pombredanne pombredanne requested a review from TG1999 October 20, 2022 11:48
Copy link
Contributor

@TG1999 TG1999 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@pombredanne pombredanne merged commit 9db8b12 into main Oct 20, 2022
@pombredanne pombredanne changed the title [WIP] Make search for vulnerabilities faster Make search for vulnerabilities faster Oct 20, 2022
@pombredanne pombredanne deleted the faster-vuln-search branch October 20, 2022 11:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants