Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

perf(git): optimize git scan when exclude but no include filter is set #4823

Merged
merged 1 commit into from
Jul 20, 2023

Conversation

edvald
Copy link
Collaborator

@edvald edvald commented Jul 13, 2023

This can be a major optimization in certain cases.

The reason this wasn't done previously was due to a git ls-files issue but we missed out on this path when no includes are used, in which case the issue doesn't come up.

Usually includes are much more specific than excludes in configs, so this approach should cover a lot of user scenarios.

It is still more optimal to use .gardenignore files over exclude patterns where possible, but using excludes in configs is obviously a common and supported use case, and not always possible to use the ignore files (say, if many actions/modules share a directory and need different exclusions).

Fixes #4763

This can be a major optimization in certain cases.

The reason this wasn't done previously was due to a git ls-files issue
but we missed out on this path when no includes are used, in which case
the issue doesn't come up.

Usually includes are much more specific than excludes in configs, so this
approach should cover a lot of user scenarios.

It is still more optimal to use `.gardenignore` files over exclude patterns
where possible, but using excludes in configs is obviously  a common and
supported use case, and not always possible to use the ignore files (say,
if many actions/modules share a directory and need different exclusions).
@Orzelius
Copy link
Contributor

I think this closes #4763?

@edvald
Copy link
Collaborator Author

edvald commented Jul 13, 2023

I believe so, yes.

@stefreak
Copy link
Member

Does it also fix #4783? If yes it would be great if we add a test case that catches the issue described in #4783 (comment) to make this code more safe to change in the future 👍

@edvald
Copy link
Collaborator Author

edvald commented Jul 13, 2023

It does not. Would be great to address that but it's a separate issue, and I wasn't able to repro it. That one might have something to do with submodules or at least some specific case that I'm not quite capturing.

@edvald edvald requested review from thsig and shumailxyz July 13, 2023 15:42
Copy link
Contributor

@shumailxyz shumailxyz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good, thanks

@vvagaytsev vvagaytsev merged commit 7361fc9 into main Jul 20, 2023
@vvagaytsev vvagaytsev deleted the git-exclude-optimization branch July 20, 2023 08:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

0.13: [Bug]: stack graph resolution performance degradation
5 participants