Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: ensure --values settings override internal defaults #64

Merged
merged 5 commits into from
Jul 30, 2024

Conversation

colesnodgrass
Copy link
Member

Context

abctl defines certain helm values differently then their default values in Airbyte's helm charts. Specifically increasing the global.jobs.resource.limits values, as increasing these values improves sync performance significantly in comparison to docker-compose on resource rich machines.

The idea was that users could override any value defined in the helm charts or in abctl by specifying a --values flag. Unfortunately this was not the case, the values within abctl were overriding any values provided by the --values flag.

To fix this, this PR now treats these internal overrides as a separate values.yml file (that only exists in memory) and will now overlay the user defined --values on top of it, ensuring that the user defined --values always takes higher priority.

Summary

@colesnodgrass colesnodgrass requested review from bgroff and a team July 29, 2024 23:48
@colesnodgrass colesnodgrass changed the title fix: ensure --values settings override internal defaults fix: ensure --values settings override internal defaults Jul 29, 2024
@colesnodgrass colesnodgrass requested a review from perangel July 29, 2024 23:51
@bgroff
Copy link
Contributor

bgroff commented Jul 30, 2024

Tested the merge changes with a values file overriding the auth.enabled field and things look good.

for _, s := range slice {
// s is going to be in the format of
// a.b.c=xyz
parts := strings.SplitN(s, "=", 2)
Copy link
Collaborator

@perangel perangel Jul 30, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this going to cause any issues if the value contains = as in a base64 encoded string (e.g. DEADBEEF==)? Would you mind adding a test case for that

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, this is why I am using SplitN with 2, so it will only split on the first =.

If the key was base64 encoded, then that would be a problem.

Copy link
Collaborator

@perangel perangel left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks good to me, but I would like to verify my concern re: = in the value

@colesnodgrass colesnodgrass merged commit e2ecb2e into main Jul 30, 2024
2 checks passed
@colesnodgrass colesnodgrass deleted the cole/prioritized-values.yml branch July 30, 2024 16:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants