Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

helm chart - Allow cloud config path for AWS provider #5105

Merged
merged 1 commit into from
Aug 25, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion charts/cluster-autoscaler/Chart.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -11,4 +11,4 @@ name: cluster-autoscaler
sources:
- https://github.com/kubernetes/autoscaler/tree/master/cluster-autoscaler
type: application
version: 9.19.4
version: 9.20.0
3 changes: 3 additions & 0 deletions charts/cluster-autoscaler/templates/deployment.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -63,6 +63,9 @@ spec:
{{- if .Values.autoDiscovery.clusterName }}
- --node-group-auto-discovery=asg:tag={{ tpl (join "," .Values.autoDiscovery.tags) . }}
{{- end }}
{{- if .Values.cloudConfigPath }}
Copy link

@SergiySavarin SergiySavarin Aug 29, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@khizunov could you consider to change cloudConfigPath default value here https://github.com/kubernetes/autoscaler/blob/master/charts/cluster-autoscaler/values.yaml#L128.
Currently, it affects autoscaler start up if file not present:

I0829 12:07:52.348973       1 reflector.go:255] Listing and watching *v1.Job from k8s.io/autoscaler/cluster-autoscaler/utils/kubernetes/listers.go:338
I0829 12:07:52.348994       1 reflector.go:219] Starting reflector *v1.StatefulSet (1h0m0s) from k8s.io/autoscaler/cluster-autoscaler/utils/kubernetes/listers.go:356
I0829 12:07:52.349001       1 reflector.go:255] Listing and watching *v1.StatefulSet from k8s.io/autoscaler/cluster-autoscaler/utils/kubernetes/listers.go:356
W0829 12:07:52.364627       1 warnings.go:70] policy/v1beta1 PodDisruptionBudget is deprecated in v1.21+, unavailable in v1.25+; use policy/v1 PodDisruptionBudget
I0829 12:07:52.364817       1 cloud_provider_builder.go:29] Building aws cloud provider.
F0829 12:07:52.364869       1 aws_cloud_provider.go:352] Couldn't open cloud provider configuration /etc/gce.conf: &fs.PathError{Op:"open", Path:"/etc/gce.conf", Err:0x2}
goroutine 30 [running]:
k8s.io/klog/v2.stacks(0x1)
    /gopath/src/k8s.io/autoscaler/cluster-autoscaler/vendor/k8s.io/klog/v2/klog.go:1038 +0x8a
k8s.io/klog/v2.(*loggingT).output(0x611e4e0, 0x3, 0x0, 0xc000565730, 0x0, {0x4d2e584, 0x2}, 0xc0009ea840, 0x0)
    /gopath/src/k8s.io/autoscaler/cluster-autoscaler/vendor/k8s.io/klog/v2/klog.go:987 +0x5fd
k8s.io/klog/v2.(*loggingT).printf(0...

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@SergiySavarin Thanks for the comment, the fix is in PR #5141

- --cloud-config={{ .Values.cloudConfigPath }}
{{- end }}
{{- else if eq .Values.cloudProvider "gce" }}
{{- if .Values.autoscalingGroupsnamePrefix }}
{{- range .Values.autoscalingGroupsnamePrefix }}
Expand Down
36 changes: 36 additions & 0 deletions cluster-autoscaler/cloudprovider/aws/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -421,6 +421,42 @@ If you want to update the vendored AWS SDK to a newer version, please make sure
2. Update the import statements within the newly-copied AWS SDK to reference the new paths (e.g., `github.com/aws/aws-sdk-go/aws/awsutil` -> `k8s.io/autoscaler/cluster-autoscaler/cloudprovider/aws/aws-sdk-go/aws/awsutil`).
3. Update the version number above to indicate the new vendored version.

## Using cloud config with helm

If you want to use custom AWS cloud config e.g. endpoint urls

1. Create ConfigMap with cloud config file definition (see [example](examples/configmap-cloudconfig-example.yaml)):
```shell
kubectl apply -f examples/configmap-cloudconfig-example.yaml
```
2. Add the following in your `values.yaml`:
```yaml
cloudConfigPath: config/cloud.conf

extraVolumes:
- name: cloud-config
configMap:
name: cloud-config

extraVolumeMounts:
- name: cloud-config
mountPath: config
```
3. Install (or upgrade) helm chart with updated values (see [example](examples/values-cloudconfig-example.yaml))

Please note: it is also possible to mount the cloud config file from host:
```yaml
extraVolumes:
- name: cloud-config
hostPath:
path: /path/to/file/on/host

extraVolumeMounts:
- name: cloud-config
mountPath: config/cloud.conf
readOnly: true
```

## Common Notes and Gotchas:

- The `/etc/ssl/certs/ca-bundle.crt` should exist by default on ec2 instance in
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
apiVersion: v1
kind: ConfigMap
metadata:
name: cloud-config
data:
cloud.conf: |
[Global]
Zone=<region><az>
[ServiceOverride "ec2"]
Service=ec2
Region=<region>
URL=<endpoint_url>
SigningRegion=<region>
[ServiceOverride "autoscaling"]
Service=autoscaling
Region=<region>
URL=<endpoint_url>
SigningRegion=<region>
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
cloudProvider: aws

autoscalingGroups:
- name: aws-asg-name
minSize: 1
maxSize: 2

awsRegion: eu-west-1
awsAccessKeyID: "******"
awsSecretAccessKey: "******"

cloudConfigPath: config/cloud.conf

extraVolumes:
- name: cloud-config
configMap:
name: cloud-config

extraVolumeMounts:
- name: cloud-config
mountPath: config