Use Kustomize to build elastic-agent manifests for both managed and standalone more #2104
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,2 @@ | ||
| overlays/**/.env | ||
| *.yaml |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,31 @@ | ||
| # https://taskfile.dev | ||
|
|
||
| version: '3' | ||
|
|
||
| tasks: | ||
| default: | ||
| cmds: | ||
| - task: managed | ||
| # - task: standalone | ||
|
|
||
| build: | ||
| cmds: | ||
| # note: create overlay | ||
| - kustomize build overlays/elastic-agent-managed > elastic-agent-managed-kubernetes.yaml | ||
| - kustomize build overlays/elastic-agent-standalone > elastic-agent-standalone-kubernetes.yaml | ||
|
|
||
| managed: | ||
| cmds: | ||
| - kubectl apply -k overlays/elastic-agent-managed | ||
|
|
||
| managed-delete: | ||
| cmds: | ||
| - kubectl delete -k overlays/elastic-agent-managed | ||
|
|
||
| standalone: | ||
| cmds: | ||
| - kubectl apply -k overlays/elastic-agent-standalone | ||
|
|
||
| standalone-delete: | ||
| cmds: | ||
| - kubectl delete -k overlays/elastic-agent-standalone |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,11 @@ | ||
| apiVersion: rbac.authorization.k8s.io/v1 | ||
| kind: ClusterRoleBinding | ||
| metadata: | ||
| name: cluster-role-binding | ||
| subjects: | ||
| - kind: ServiceAccount | ||
| name: service-account | ||
| roleRef: | ||
| kind: ClusterRole | ||
| name: cluster-role | ||
| apiGroup: rbac.authorization.k8s.io |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,67 @@ | ||
| apiVersion: rbac.authorization.k8s.io/v1 | ||
| kind: ClusterRole | ||
| metadata: | ||
| name: cluster-role | ||
| rules: | ||
| - apiGroups: [""] | ||
| resources: | ||
| - nodes | ||
| - namespaces | ||
| - events | ||
| - pods | ||
| - services | ||
| - configmaps | ||
| # Needed for cloudbeat | ||
| - serviceaccounts | ||
| - persistentvolumes | ||
| - persistentvolumeclaims | ||
| verbs: ["get", "list", "watch"] | ||
| # Enable this rule only if planing to use kubernetes_secrets provider | ||
| #- apiGroups: [""] | ||
| # resources: | ||
| # - secrets | ||
| # verbs: ["get"] | ||
| - apiGroups: ["extensions"] | ||
| resources: | ||
| - replicasets | ||
| verbs: ["get", "list", "watch"] | ||
| - apiGroups: ["apps"] | ||
| resources: | ||
| - statefulsets | ||
| - deployments | ||
| - replicasets | ||
| - daemonsets | ||
| verbs: ["get", "list", "watch"] | ||
| - apiGroups: | ||
| - "" | ||
| resources: | ||
| - nodes/stats | ||
| verbs: | ||
| - get | ||
| - apiGroups: [ "batch" ] | ||
| resources: | ||
| - jobs | ||
| - cronjobs | ||
| verbs: [ "get", "list", "watch" ] | ||
| # Needed for apiserver | ||
| - nonResourceURLs: | ||
| - "/metrics" | ||
| verbs: | ||
| - get | ||
| # Needed for cloudbeat | ||
| - apiGroups: ["rbac.authorization.k8s.io"] | ||
| resources: | ||
| - clusterrolebindings | ||
| - clusterroles | ||
| - rolebindings | ||
| - roles | ||
| verbs: ["get", "list", "watch"] | ||
| # Needed for cloudbeat | ||
| - apiGroups: ["policy"] | ||
| resources: | ||
| - podsecuritypolicies | ||
| verbs: ["get", "list", "watch"] | ||
| - apiGroups: [ "storage.k8s.io" ] | ||
| resources: | ||
| - storageclasses | ||
| verbs: [ "get", "list", "watch" ] |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,6 @@ | ||
| resources: | ||
| - service-account.yaml | ||
| - role-leases.yaml | ||
| - cluster-role.yaml | ||
| - role-binding-leases.yaml | ||
| - cluster-role-binding.yaml |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,11 @@ | ||
| apiVersion: rbac.authorization.k8s.io/v1 | ||
| kind: RoleBinding | ||
| metadata: | ||
| name: role-binding-leases | ||
| subjects: | ||
| - kind: ServiceAccount | ||
| name: service-account | ||
| roleRef: | ||
| kind: Role | ||
| name: role-leases | ||
| apiGroup: rbac.authorization.k8s.io |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,10 @@ | ||
| apiVersion: rbac.authorization.k8s.io/v1 | ||
| kind: Role | ||
| metadata: | ||
| name: role-leases | ||
| rules: | ||
| - apiGroups: | ||
| - coordination.k8s.io | ||
| resources: | ||
| - leases | ||
| verbs: ["get", "create", "update"] |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,4 @@ | ||
| apiVersion: v1 | ||
| kind: ServiceAccount | ||
| metadata: | ||
| name: service-account |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,6 @@ | ||
| FLEET_URL=https://fleet-server:8220 | ||
| FLEET_ENROLLMENT_TOKEN=token-id | ||
| FLEET_INSECURE=true | ||
| KIBANA_HOST=http://kibana:5601 | ||
| KIBANA_FLEET_USERNAME=elastic | ||
| KIBANA_FLEET_PASSWORD=changeme |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,116 @@ | ||
| # For more information https://www.elastic.co/guide/en/fleet/current/running-on-kubernetes-managed-by-fleet.html | ||
| apiVersion: apps/v1 | ||
| kind: DaemonSet | ||
| metadata: | ||
| name: daemonset | ||
| spec: | ||
| template: | ||
| spec: | ||
| # Tolerations are needed to run Elastic Agent on Kubernetes control-plane nodes. | ||
| # Agents running on control-plane nodes collect metrics from the control plane components (scheduler, controller manager) of Kubernetes | ||
| tolerations: | ||
| - key: node-role.kubernetes.io/control-plane | ||
| effect: NoSchedule | ||
| - key: node-role.kubernetes.io/master | ||
| effect: NoSchedule | ||
| serviceAccountName: service-account | ||
| hostNetwork: true | ||
| # 'hostPID: true' enables the Elastic Security integration to observe all process exec events on the host. | ||
| # Sharing the host process ID namespace gives visibility of all processes running on the same host. | ||
| hostPID: true | ||
| dnsPolicy: ClusterFirstWithHostNet | ||
| containers: | ||
| - name: elastic-agent | ||
| image: docker.elastic.co/beats/elastic-agent:8.1.0 | ||
| env: | ||
| # Set to 1 for enrollment into Fleet server. If not set, Elastic Agent is run in standalone mode | ||
| - name: FLEET_ENROLL | ||
| value: "1" | ||
| - name: NODE_NAME | ||
| valueFrom: | ||
| fieldRef: | ||
| fieldPath: spec.nodeName | ||
| - name: POD_NAME | ||
| valueFrom: | ||
| fieldRef: | ||
| fieldPath: metadata.name | ||
| envFrom: | ||
| - configMapRef: | ||
| # Fleet Server URL to enroll the Elastic Agent into | ||
| # FLEET_URL can be found in Kibana, go to Management > Fleet > Settings | ||
| # - name: FLEET_URL | ||
| # value: "https://fleet-server:8220" | ||
| # Elasticsearch API key used to enroll Elastic Agents in Fleet (https://www.elastic.co/guide/en/fleet/current/fleet-enrollment-tokens.html#fleet-enrollment-tokens) | ||
| # If FLEET_ENROLLMENT_TOKEN is empty then KIBANA_HOST, KIBANA_FLEET_USERNAME, KIBANA_FLEET_PASSWORD are needed | ||
| # - name: FLEET_ENROLLMENT_TOKEN | ||
| # value: "token-id" | ||
| # Set to true to communicate with Fleet with either insecure HTTP or unverified HTTPS | ||
| # - name: FLEET_INSECURE | ||
| # value: "true" | ||
| # - name: KIBANA_HOST | ||
| # value: "http://kibana:5601" | ||
| # # The basic authentication username used to connect to Kibana and retrieve a service_token to enable Fleet | ||
| # - name: KIBANA_FLEET_USERNAME | ||
| # value: "elastic" | ||
| # # The basic authentication password used to connect to Kibana and retrieve a service_token to enable Fleet | ||
| # - name: KIBANA_FLEET_PASSWORD | ||
| # value: "changeme" | ||
| name: configs | ||
| securityContext: | ||
| runAsUser: 0 | ||
| resources: | ||
| limits: | ||
| memory: 500Mi | ||
| requests: | ||
| cpu: 100m | ||
| memory: 200Mi | ||
| volumeMounts: | ||
|
There was a problem hiding this comment. I think following part can also be splitted to multilple overalys. We want to have needed mounts for observability and extra ones that imposed from security There was a problem hiding this comment. do you mind elaborate a bit more on the topic. I vaguely recollect some suggestions from the security team but I wouldn't know how to change this part of the manifest |
||
| - name: proc | ||
| mountPath: /hostfs/proc | ||
| readOnly: true | ||
| - name: cgroup | ||
| mountPath: /hostfs/sys/fs/cgroup | ||
| readOnly: true | ||
| - name: varlibdockercontainers | ||
| mountPath: /var/lib/docker/containers | ||
| readOnly: true | ||
| - name: varlog | ||
| mountPath: /var/log | ||
| readOnly: true | ||
| - name: etc-full | ||
| mountPath: /hostfs/etc | ||
| readOnly: true | ||
| - name: var-lib | ||
| mountPath: /hostfs/var/lib | ||
| readOnly: true | ||
| - name: etc-mid | ||
| mountPath: /etc/machine-id | ||
| readOnly: true | ||
| volumes: | ||
| - name: proc | ||
| hostPath: | ||
| path: /proc | ||
| - name: cgroup | ||
| hostPath: | ||
| path: /sys/fs/cgroup | ||
| - name: varlibdockercontainers | ||
| hostPath: | ||
| path: /var/lib/docker/containers | ||
| - name: varlog | ||
| hostPath: | ||
| path: /var/log | ||
| # The following volumes are needed for Cloud Security Posture integration (cloudbeat) | ||
| # If you are not using this integration, then these volumes and the corresponding | ||
| # mounts can be removed. | ||
| - name: etc-full | ||
| hostPath: | ||
| path: /etc | ||
| - name: var-lib | ||
| hostPath: | ||
| path: /var/lib | ||
| # Mount /etc/machine-id from the host to determine host ID | ||
| # Needed for Elastic Security integration | ||
| - name: etc-mid | ||
| hostPath: | ||
| path: /etc/machine-id | ||
| type: File | ||
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,20 @@ | ||
| namespace: kube-system | ||
| commonLabels: | ||
| k8s-app: elastic-agent | ||
| namePrefix: elastic-agent-managed- | ||
|
|
||
| images: | ||
| - name: docker.elastic.co/beats/elastic-agent | ||
| newTag: "8.6.0" | ||
|
|
||
| resources: | ||
| - ../../base/common | ||
| - daemonset.yaml | ||
|
|
||
| configMapGenerator: | ||
| - name: configs | ||
| envs: | ||
| - .env | ||
|
|
||
| generatorOptions: | ||
| disableNameSuffixHash: true |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,5 @@ | ||
| ES_USERNAME=elastic | ||
| ES_PASSWORD=changeme | ||
| ES_HOST=https://elasticsearch:9200 | ||
|
There was a problem hiding this comment. I would suggest to put namespace as variable here There was a problem hiding this comment. I don't think you can use a variable from an .env file in the kustomization.yml file (which is where I set the namespace for all the resources) There was a problem hiding this comment. Can we have as boolean on/off options the ability to install kube state metrics here? There was a problem hiding this comment. I don't think so. This env file is only used to create a configmap and I don't think we can use an env variable from a configmap to decide if we want to add a manifest based on that. We could use an overlay with an extra resource to install or not the kube-state-metrics There was a problem hiding this comment. But then we will need to edit the kustomize.yaml. Maybe we can think that we can provide diffrent kustomzation files ? There was a problem hiding this comment. yes, I believe we have to go that way. Different overlays for different behaviours There was a problem hiding this comment. 1st Option: 2nd Option: There was a problem hiding this comment. I heard about ytt but it seemed way too complicated compared to kustomize. Should we create another issue with the list of requirements before diving into ytt? |
||
| ES_SSL_VERIFICATION_MODE=full | ||
| ES_ALLOW_OLDER_VERSIONS=false | ||
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Just out of curiosity, why all these settings are commented out?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
they are mostly left there for documentation. We should address them (one way or another) in the next update about on this topic.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
But when those vars should be uncommented?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I have created this PR to fix this