Backport: Refactor VulnerabilityAnalysisTask to be more efficient
#4625
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
* For portfolio analysis, don't fetch *all* projects at once, but in batches of 100. * Regularly evict objects from the ORM's L1 cache to prevent excessive cache size. * For project analysis, don't fetch *all* components at once, but in batches of 1000. * Limit the fields being fetched from the database to only those needed for vulnerability analysis. * When triggering a project analysis, only transmit the project's UUID via events, instead of all of its components. * Deprecate `QueryManager#getAllProjects` for removal. This method is never the right choice, but is still used in a few places. In addition, as byproducts of this refactoring: * Add MDC variables to provide more context in logs emitted during vulnerability analysis. * Remove event subscription for specific scanner events. Scanners are only ever invoked directly, never through the event system. * Gracefully handle thread interrupts. * Fix `CacheableScanTask#applyAnalysisFromCache` never being invoked. Signed-off-by: nscuro <[email protected]>
nscuro
force-pushed
the
backport-pr-4623
branch
from
7cad65e to
cbd6405
Compare
Coverage summary from CodacySee diff coverage on Codacy
Coverage variation details
Coverage variation is the difference between the coverage for the head and common ancestor commits of the pull request branch: Diff coverage details
Diff coverage is the percentage of lines that are covered by tests out of the coverable lines that the pull request added or modified: See your quality gate settings Change summary preferencesCodacy stopped sending the deprecated coverage status on June 5th, 2024. Learn more |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
Refactors vuln analysis task to be more efficient.
Addressed Issue
Backports #4623
Additional Details
See #4623
Checklist
This PR implements an enhancement, and I have provided tests to verify that it works as intendedThis PR introduces changes to the database model, and I have added corresponding update logicThis PR introduces new or alters existing behavior, and I have updated the documentation accordingly