Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Set helm releases history size. #44

Merged
merged 2 commits into from
Apr 28, 2021
Merged

Set helm releases history size. #44

merged 2 commits into from
Apr 28, 2021

Conversation

mglotov
Copy link
Contributor

@mglotov mglotov commented Apr 26, 2021

Helm keeps releases history as k8s secrets. If we don't set max_history it will be unlimited.
There is an opened issue related to that helm/helm#7997.
If count of secrets is huge then helm list/istall can be failed due to:

request.go:924] Unexpected error when reading response body: net/http: request canceled (Client.Timeout exceeded while reading body)
Error: unable to build kubernetes objects from release manifest: unexpected error when reading response body. Please retry. Original error: net/http: request canceled (Client.Timeout exceeded while reading body

Helm keeps releases history as k8s secrets. If we don't set max_history it will be unlimited.
There is an opened issue related to that helm/helm#7997.
If count of secrets is huge then helm list/istall can be failed due to:
```
request.go:924] Unexpected error when reading response body: net/http: request canceled (Client.Timeout exceeded while reading body)
Error: unable to build kubernetes objects from release manifest: unexpected error when reading response body. Please retry. Original error: net/http: request canceled (Client.Timeout exceeded while reading body
```
@mglotov mglotov requested a review from a team April 26, 2021 17:21
@halfb00t
Copy link
Member

@mglotov wouldn't it better to set this limit using variable?

@mglotov
Copy link
Contributor Author

mglotov commented Apr 27, 2021

@mglotov wouldn't it better to set this limit using variable?

@mglotov mglotov closed this Apr 27, 2021
@mglotov mglotov reopened this Apr 27, 2021
@mglotov
Copy link
Contributor Author

mglotov commented Apr 27, 2021

@halfb00t Do you want to set them as we do for helm chart version?

@halfb00t
Copy link
Member

@halfb00t Do you want to set them as we do for helm chart version?

@mglotov i think single var for all system releases is enough =)

@mglotov
Copy link
Contributor Author

mglotov commented Apr 27, 2021

@p1gmale0n @halfb00t hmm, is it really OK to have one variable for all releases?

@p1gmale0n
Copy link
Contributor

@p1gmale0n hmm, is it really OK to have one variable for all releases?

i think this is ok for as the default behavior.
you can still change the specific value for a specific release, if necessary

@arthur-c
Copy link

Does it cover failed releases?

In helm/helm#7997 (comment) we ended with lots of release versions due to a deployment bug that went for few weeks. We had a loop deploying helm releases that always failed (failure unrelated to helm), then retried... So we had thousands of failed versions.

@mglotov
Copy link
Contributor Author

mglotov commented Apr 28, 2021

Does it cover failed releases?

In helm/helm#7997 (comment) we ended with lots of release versions due to a deployment bug that went for few weeks. We had a loop deploying helm releases that always failed (failure unrelated to helm), then retried... So we had thousands of failed versions.

Yes, it does. It covers all release versions (failed, deployed, ..). There are some changes in helm v3.5.3 and terraform-helm-provider v2.1.1. Starting from those if you don't have at least one deployed release, then you are limited by max_history size and can't deploy any new release version, even if it's a working release version. You will need to increase max_history value or delete a k8s secret with the release version.

@mglotov mglotov merged commit dbb3e95 into main Apr 28, 2021
@mglotov mglotov deleted the count_of_helm_releases branch April 28, 2021 10:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants