Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Perf improvement: process '-has' flag at the end #37

Merged
merged 2 commits into from
May 6, 2024
Merged

Conversation

fridim
Copy link
Contributor

@fridim fridim commented May 6, 2024

The --has flag is costly because it forces variables to be merged. When looping over a lot of catalog items, it can slow down the command a lot.

Move its processing at the last position when filtering catalog items, so it's executed less often when used in combination of other flags.

This will help in agnosticv-operator.
Here is the improvement from 5s to 0.5s of execution time for a command:

agnosticv on  master [$?] via 🐍 v3.11.8
❯ time agnosticv.v0.8.0 --list --has=__meta__ --dir $PWD --related sandboxes-gpte/ANS_BU_WKSP_JUST_SATELLITE/common.yaml --or-related sandboxes-gpte/ANS_BU_WKSP_JUST_SATELLITE/description.html --or-related sandboxes-gpte/ANS_BU_WKSP_JUST_SATELLITE/dev.yaml --or-related sandboxes-gpte/ANS_BU_WKSP_JUST_SATELLITE/info-message-template.adoc

real    0m5.222s
user    0m5.411s
sys     0m0.427s

❯ time agnosticv_dev --list --has=__meta__ --dir $PWD --related sandboxes-gpte/ANS_BU_WKSP_JUST_SATELLITE/common.yaml --or-related sandboxes-gpte/ANS_BU_WKSP_JUST_SATELLITE/description.html --or-related sandboxes-gpte/ANS_BU_WKSP_JUST_SATELLITE/dev.yaml --or-related sandboxes-gpte/ANS_BU_WKSP_JUST_SATELLITE/info-message-template.adoc

real    0m0.499s
user    0m0.404s
sys     0m0.125s

@fridim fridim added the enhancement New feature or request label May 6, 2024
The '--has' flag is costly because it forces variables to be merged.
When looping over a lot of catalog items, it can slow down the command a
lot.

Move its processing at the last position when filtering catalog items, so it's
executed less often when used in combination of other flags.
@fridim
Copy link
Contributor Author

fridim commented May 6, 2024

Tested successfully against our full agnosticv RHDP repo using test_new_version.sh

@fridim fridim marked this pull request as draft May 6, 2024 09:45
@fridim fridim force-pushed the GPTEINFRA-9625 branch from 3ae3fd5 to a5d7424 Compare May 6, 2024 09:51
@fridim fridim marked this pull request as ready for review May 6, 2024 09:52
@fridim fridim merged commit f752710 into master May 6, 2024
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants