Migrate helm charts from opensearch-devops repo (opensearch-project#7)

* Adding a new folder to host Helm related code


* Helm Chart for OpenSearch (opensearch-project#4)

* Create basic structure of OpenSearch helm chart


* Add templates and change values


* Change statefulset and configmap to resolve indentation issue


* Fix issues in templates


* Fix typos in statefulset.yaml

* Add multinode deployment feature


* Update version to reflect the OpenSearch version

* Add explicit security configuration

* Update values.yaml

* Create placeholder README.md


* Minimum masters should be 3

* Add YAML support for config. sysctl vm.mem fix.

* Fixing PSP. Adding better sysctl logic.

* Adding ref for systctl

* PSP False by default

* Disable HTTP SSL by default for Demo.

* Fix Chart version to sync with OpenSearch Version


* Change cluster name and enable SSL by default


Co-authored-by: Aaron Layfield <[email protected]>

* fix: give networkpolicy objects a unique name (opensearch-project#16)

This fixes the problem of installing this chart multiple times in the
same namespace and having the network policy name conflict.


* fix: use the stable chart appVersion as image tag by default (opensearch-project#17)

Using :latest by default is going to lead to clusters with version skew
as pods schedule onto new nodes. So use a stable tag instead.


* OpenSearch Dashboards Helm Chart (opensearch-project#10)

* Scaffold OpenSearch Dashboards Helm Chart


* Fix error for connection refused


* Add RBAC functionality


* Add security configurations in the chart


* Address issues and comments


* Fix templates


* Disable SSL by default

* Address comments for beautification

* Address comments


* chore: update demo config section (opensearch-project#24)

This snippet doesn't make sense in a kubernetes statefulset.


* added secretMounts to values.yaml w/ example config (opensearch-project#29)


Co-authored-by: johannes.reppin <[email protected]>

* Change persistence config to make it more coherent w/ other helm charts (opensearch-project#33)


Co-authored-by: johannes.reppin <[email protected]>

* add Volumes and change broken (!) yaml indentation (opensearch-project#31)


Co-authored-by: johannes.reppin <[email protected]>

* support for current ingress apiVersion (opensearch-project#47)

* Helm Chart Fixes for Env variables and volumes (opensearch-project#35)

* Helm Chart Fixes for Env variables and volumes

The opensearch-dashboards chart failed to render correctly when
utilizing the extraEnvs flag, caused by incorrect indentation.

The opensearch chart failed to render when utlizing the secrets for the
security config, this was due to them being in the env section.

This pull request reqolves both issues, verified via running helm
template with the minumal values files included here:

```yaml
envFrom:
  - secretRef:
      name: kibana-secrets
extraEnvs:
  - name: TENANT_ID
    valueFrom:
      secretKeyRef:
        name: kibana-secrets
        key: tenantID
```

```yaml
securityConfig:
  enabled: true
  configSecret: "security-config"
  internalUsersSecret: "internal-users-config"
  rolesMappingSecret: "roles-mapping-config"
  rolesSecret: "roles-config"
```


* Updating paths in sts to be dynamic

Updating the paths specified in the sts for opensearch to utilize
.Values.opensearchHome to allow for dynamic paths, with a default of
`/usr/share/opensearch` which should be sufficient for most users.


* Fixing config path in opensearch-dashboards (opensearch-project#38)

* Fixing config path in opensearch-dashboards

The manifests rendered by the Helm chart place the user provided config
into the incorrect directory. This simply updates that location to the
correct path and updates the values.yaml file to use the correct default
config file so that the user provided setting override the defaults.


* Updating cert paths to opensearch-dashboards

Cert paths also need to utilize new filesystem location for
opensearch-dashboards config.


* Resolves issue with securityConfig path (opensearch-project#41)

* Resolves issue with securityConfig path

Issue opensearch-project#39

This updates the securityConfig path in values to use the correct value
for opensearch.


* Fixing bad auto formatting

Removing unneeded indentation/newlines.


* Fixing missed auto formatting errors


* resolve issue about .Values.opensearchHome (opensearch-project#52)

refer to this:
opensearch-project/opensearch-devops@fe831db#commitcomment-55395428

Error Msg: nil pointer evaluating interface {}.opensearchHome

* Fix helm chart can not be deployed without ssl (opensearch-project#56)

* Fixing issue exposed by changes in opensearch-project#38

After switching the name of the config file, and removing the shadowing
between the default (from the docker container opensearch-dashbaords.yaml) and the default from the helm chart (dashboards.yaml) there is an issue with the certs that are attempting to be used.

In order for this to work with the defaults, disabled TLS verification
will be needed, and then disabling TLS to remain in line with the
defaults.

I added a commented out section showing what could potentially be used
as TLS config if the user chooses to enable it.


* Using conventional yaml formatting for ssl config

Moving comments around to follow relevant code and utilizing nested yaml
format rather than dot format.


* Changing Folder name to Charts

* Change deafult configuration for dashboards


* Update securityconfig.yaml to remove extra spaces


Co-authored-by: Barani <[email protected]>
Co-authored-by: Aaron Layfield <[email protected]>
Co-authored-by: Scott Leggett <[email protected]>
Co-authored-by: Johannes Reppin <[email protected]>
Co-authored-by: johannes.reppin <[email protected]>
Co-authored-by: paltryeffort <[email protected]>
Co-authored-by: hgoscenski-imanage <[email protected]>
Co-authored-by: Nagle Zhang <[email protected]>

charts/README.md

@@ -0,0 +1,44 @@
+
+This Helm chart is a lightweight way to configure and run the official [OpenSearch Docker image](https://hub.docker.com/r/opensearchproject/opensearch).
+
+
+- [Installing](#installing)
+- [Usage notes](#usage-notes)
+- [Configuration](#configuration)
+- [Future](#Future)
+
+
+
+## Installing
+
+This chart is tested with the latest 1.0.0-rc1 version.
+
+* Clone this repo
+* Install it:
+    - with Helm 3: `helm install oss ./Helm/opensearch`
+
+## Usage notes
+
+* The chart deploys a StatefulSet and by default will do an automated rolling
+  update of your cluster. It does this by waiting for the cluster health to become
+  green after each instance is updated.
+* It is important to verify that the JVM heap size in `esJavaOpts` and to set
+  the CPU/Memory `resources` to something suitable for your cluster.
+* To simplify chart and maintenance each set of node groups is deployed as a
+  separate Helm release. Without doing this it isn't possible to resize persistent
+  volumes in a StatefulSet. By setting it up this way it makes it possible to add
+  more nodes with a new storage size then drain the old ones. It also solves the
+  problem of allowing the user to determine which node groups to update first when
+  doing upgrades or changes.
+* We have designed this chart to be very un-opinionated about how to configure
+  OpenSearch. It exposes ways to set environment variables and mount secrets
+  inside of the container. Doing this makes it much easier for this chart to
+  support multiple versions with minimal changes.
+
+## Configuration
+TODO : Write about all the parameters used
+
+
+## Future
+* Create example for different types of configurations for different K8S providers.
+* Have a chart for deploying the OpenSearch stack comprising OpenSearch and OpenSearch Dashboards

charts/opensearch-dashboards/.helmignore

@@ -0,0 +1,23 @@
+# Patterns to ignore when building packages.
+# This supports shell glob matching, relative path matching, and
+# negation (prefixed with !). Only one pattern per line.
+.DS_Store
+# Common VCS dirs
+.git/
+.gitignore
+.bzr/
+.bzrignore
+.hg/
+.hgignore
+.svn/
+# Common backup files
+*.swp
+*.bak
+*.tmp
+*.orig
+*~
+# Various IDEs
+.project
+.idea/
+*.tmproj
+.vscode/

charts/opensearch-dashboards/Chart.yaml

@@ -0,0 +1,24 @@
+apiVersion: v2
+name: opensearch-dashboards
+description: A Helm chart for OpenSearch Dashboards
+
+# A chart can be either an 'application' or a 'library' chart.
+#
+# Application charts are a collection of templates that can be packaged into versioned archives
+# to be deployed.
+#
+# Library charts provide useful utilities or functions for the chart developer. They're included as
+# a dependency of application charts to inject those utilities and functions into the rendering
+# pipeline. Library charts do not define any templates and therefore cannot be deployed.
+type: application
+
+# This is the chart version. This version number should be incremented each time you make changes
+# to the chart and its templates, including the app version.
+# Versions are expected to follow Semantic Versioning (https://semver.org/)
+version: 1.0.0
+
+# This is the version number of the application being deployed. This version number should be
+# incremented each time you make changes to the application. Versions are not expected to
+# follow Semantic Versioning. They should reflect the version the application is using.
+# It is recommended to use it with quotes.
+appVersion: "1.0.0"

charts/opensearch-dashboards/templates/NOTES.txt

@@ -0,0 +1,22 @@
+1. Get the application URL by running these commands:
+{{- if .Values.ingress.enabled }}
+{{- range $host := .Values.ingress.hosts }}
+  {{- range .paths }}
+  http{{ if $.Values.ingress.tls }}s{{ end }}://{{ $host.host }}{{ .path }}
+  {{- end }}
+{{- end }}
+{{- else if contains "NodePort" .Values.service.type }}
+  export NODE_PORT=$(kubectl get --namespace {{ .Release.Namespace }} -o jsonpath="{.spec.ports[0].nodePort}" services {{ include "opensearch-dashboards.fullname" . }})
+  export NODE_IP=$(kubectl get nodes --namespace {{ .Release.Namespace }} -o jsonpath="{.items[0].status.addresses[0].address}")
+  echo http://$NODE_IP:$NODE_PORT
+{{- else if contains "LoadBalancer" .Values.service.type }}
+     NOTE: It may take a few minutes for the LoadBalancer IP to be available.
+           You can watch the status of by running 'kubectl get --namespace {{ .Release.Namespace }} svc -w {{ include "opensearch-dashboards.fullname" . }}'
+  export SERVICE_IP=$(kubectl get svc --namespace {{ .Release.Namespace }} {{ include "opensearch-dashboards.fullname" . }} --template "{{"{{ range (index .status.loadBalancer.ingress 0) }}{{.}}{{ end }}"}}")
+  echo http://$SERVICE_IP:{{ .Values.service.port }}
+{{- else if contains "ClusterIP" .Values.service.type }}
+  export POD_NAME=$(kubectl get pods --namespace {{ .Release.Namespace }} -l "app.kubernetes.io/name={{ include "opensearch-dashboards.name" . }},app.kubernetes.io/instance={{ .Release.Name }}" -o jsonpath="{.items[0].metadata.name}")
+  export CONTAINER_PORT=$(kubectl get pod --namespace {{ .Release.Namespace }} $POD_NAME -o jsonpath="{.spec.containers[0].ports[0].containerPort}")
+  echo "Visit http://127.0.0.1:8080 to use your application"
+  kubectl --namespace {{ .Release.Namespace }} port-forward $POD_NAME 8080:$CONTAINER_PORT
+{{- end }}

charts/opensearch-dashboards/templates/_helpers.tpl

@@ -0,0 +1,73 @@
+{{/*
+Expand the name of the chart.
+*/}}
+{{- define "opensearch-dashboards.name" -}}
+{{- default .Chart.Name .Values.nameOverride | trunc 63 | trimSuffix "-" }}
+{{- end }}
+
+{{/*
+Create a default fully qualified app name.
+We truncate at 63 chars because some Kubernetes name fields are limited to this (by the DNS naming spec).
+If release name contains chart name it will be used as a full name.
+*/}}
+{{- define "opensearch-dashboards.fullname" -}}
+{{- if .Values.fullnameOverride }}
+{{- .Values.fullnameOverride | trunc 63 | trimSuffix "-" }}
+{{- else }}
+{{- $name := default .Chart.Name .Values.nameOverride }}
+{{- if contains $name .Release.Name }}
+{{- .Release.Name | trunc 63 | trimSuffix "-" }}
+{{- else }}
+{{- printf "%s-%s" .Release.Name $name | trunc 63 | trimSuffix "-" }}
+{{- end }}
+{{- end }}
+{{- end }}
+
+{{/*
+Create chart name and version as used by the chart label.
+*/}}
+{{- define "opensearch-dashboards.chart" -}}
+{{- printf "%s-%s" .Chart.Name .Chart.Version | replace "+" "_" | trunc 63 | trimSuffix "-" }}
+{{- end }}
+
+{{/*
+Common labels
+*/}}
+{{- define "opensearch-dashboards.labels" -}}
+helm.sh/chart: {{ include "opensearch-dashboards.chart" . }}
+{{ include "opensearch-dashboards.selectorLabels" . }}
+{{- if .Chart.AppVersion }}
+app.kubernetes.io/version: {{ .Chart.AppVersion | quote }}
+{{- end }}
+app.kubernetes.io/managed-by: {{ .Release.Service }}
+{{- end }}
+
+{{/*
+Selector labels
+*/}}
+{{- define "opensearch-dashboards.selectorLabels" -}}
+app.kubernetes.io/name: {{ include "opensearch-dashboards.name" . }}
+app.kubernetes.io/instance: {{ .Release.Name }}
+{{- end }}
+
+{{/*
+Create the name of the service account to use
+*/}}
+{{- define "opensearch-dashboards.serviceAccountName" -}}
+{{- if .Values.serviceAccount.create }}
+    {{- default (include "opensearch-dashboards.fullname" .) .Values.serviceAccount.name }}-dashboards
+{{- else }}
+    {{- default "default" .Values.serviceAccount.name }}
+{{- end }}
+{{- end }}
+
+{{/*
+Define standard labels for frequently used metadata.
+*/}}
+{{- define "opensearch-dashboards.standard" -}}
+app: {{ template "opensearch-dashboards.fullname" . }}
+chart: "{{ .Chart.Name }}-{{ .Chart.Version }}"
+release: "{{ .Release.Name }}"
+heritage: "{{ .Release.Service }}"
+{{- end -}}
+

charts/opensearch-dashboards/templates/configmap.yaml

@@ -0,0 +1,12 @@
+{{- if .Values.config -}}
+apiVersion: v1
+kind: ConfigMap
+metadata:
+  name: {{ template "opensearch-dashboards.fullname" . }}-config
+  labels: {{ include "opensearch-dashboards.labels" . | nindent 4 }}
+data:
+{{- range $configName, $configYaml := .Values.config }}
+  {{ $configName }}: |
+    {{- toYaml $configYaml | nindent 4 }}
+{{- end }}
+{{- end -}}

charts/opensearch-dashboards/templates/deployment.yaml

@@ -0,0 +1,158 @@
+apiVersion: apps/v1
+kind: Deployment
+metadata:
+  name: {{ template "opensearch-dashboards.fullname" . }}
+  labels: {{- include "opensearch-dashboards.labels" . | nindent 4 }}
+spec:
+  replicas: {{ .Values.replicaCount }}
+  strategy:
+{{ toYaml .Values.updateStrategy | indent 4 }}
+  selector:
+    matchLabels:
+      app: {{ .Chart.Name }}
+      release: {{ .Release.Name | quote }}
+  template:
+    metadata:
+      labels:
+        app: {{ .Chart.Name }}
+        release: {{ .Release.Name | quote }}
+        {{- range $key, $value := .Values.labels }}
+        {{ $key }}: {{ $value | quote }}
+        {{- end }}
+      annotations:
+        {{- range $key, $value := .Values.podAnnotations }}
+        {{ $key }}: {{ $value | quote }}
+        {{- end }}
+        {{- /* This forces a restart if the configmap has changed */}}
+        {{- if .Values.config }}
+        configchecksum: {{ include (print .Template.BasePath "/configmap.yaml") . | sha256sum | trunc 63 }}
+        {{- end }}
+    spec:
+{{- if .Values.priorityClassName }}
+      priorityClassName: {{ .Values.priorityClassName }}
+{{- end }}
+      securityContext:
+{{ toYaml .Values.podSecurityContext | indent 8 }}
+      serviceAccountName: {{ template "opensearch-dashboards.serviceAccountName" . }}
+      {{- if .Values.hostAliases }}
+      hostAliases: {{ toYaml .Values.hostAliases | nindent 6 }}
+      {{- end }}
+      volumes:
+        {{- range .Values.secretMounts }}
+        - name: {{ .name }}
+          secret:
+            secretName: {{ .secretName }}
+        {{- end }}
+        {{- if .Values.config }}
+        - name: config
+          configMap:
+            name: {{ template "opensearch-dashboards.fullname" . }}-config
+        {{- end }}
+    {{- with .Values.nodeSelector }}
+      nodeSelector:
+{{ toYaml . | indent 8 }}
+    {{- end }}
+    {{- with .Values.affinity }}
+      affinity:
+{{ toYaml . | indent 8 }}
+    {{- end }}
+    {{- with .Values.tolerations }}
+      tolerations:
+{{ toYaml . | indent 8 }}
+    {{- end }}
+    {{- if .Values.imagePullSecrets }}
+      imagePullSecrets:
+{{ toYaml .Values.imagePullSecrets | indent 8 }}
+    {{- end }}
+      {{- if .Values.extraInitContainers }}
+      # Currently some extra blocks accept strings
+      # to continue with backwards compatibility this is being kept
+      # whilst also allowing for yaml to be specified too.
+      initContainers:
+      {{- if eq "string" (printf "%T" .Values.extraInitContainers) }}
+{{ tpl .Values.extraInitContainers . | indent 6 }}
+      {{- else }}
+{{ toYaml .Values.extraInitContainers | indent 6 }}
+      {{- end }}
+      {{- end }}
+      containers:
+      - name: dashboards
+        securityContext:
+{{ toYaml .Values.securityContext | indent 10 }}
+        image: "{{ .Values.image }}:{{ .Values.imageTag | default .Chart.AppVersion }}"
+        imagePullPolicy: "{{ .Values.imagePullPolicy }}"
+        env:
+        {{- if .Values.opensearchURL }}
+        - name: OPENSEARCH_URL
+          value: "{{ .Values.opensearchURL }}"
+        {{- else if .Values.opensearchHosts }}
+        - name: OPENSEARCH_HOSTS
+          value: "{{ .Values.opensearchHosts }}"
+        {{- end }}
+        - name: SERVER_HOST
+          value: "{{ .Values.serverHost }}"
+        {{- if .Values.opensearchAccount.secret }}
+        - name: OPENSEARCH_USERNAME
+          valueFrom:
+            secretKeyRef:
+              name: {{ .Values.opensearchAccount.secret }}
+              key: username
+        - name: OPENSEARCH_PASSWORD
+          valueFrom:
+            secretKeyRef:
+              name: {{ .Values.opensearchAccount.secret }}
+              key: password
+        {{- if and .Values.opensearchAccount.keyPassphrase.enabled }}
+        - name: KEY_PASSPHRASE
+          valueFrom:
+            secretKeyRef:
+              name: {{ .Values.opensearchAccount.secret }}
+              key: keypassphrase
+            # 32-character random string to be used as cookie password by security plugin
+        {{- end }}
+        - name: COOKIE_PASS
+          valueFrom:
+            secretKeyRef:
+              name: {{ .Values.opensearchAccount.secret }}
+              key: cookie
+            {{- end }}
+{{- if .Values.extraEnvs }}
+{{ toYaml .Values.extraEnvs | indent 8 }}
+{{- end }}
+{{- if .Values.envFrom }}
+        envFrom:
+{{ toYaml .Values.envFrom | indent 10 }}
+{{- end }}
+        ports:
+        - containerPort: {{ .Values.service.port }}
+          name: {{ .Values.service.httpPortName | default "http" }}
+          protocol: TCP
+{{- if .Values.lifecycle }}
+        lifecycle:
+{{ toYaml .Values.lifecycle | indent 10 }}
+{{- end }}
+        resources:
+{{ toYaml .Values.resources | indent 10 }}
+        volumeMounts:
+          {{- range .Values.secretMounts }}
+          - name: {{ .name }}
+            mountPath: {{ .path }}
+            {{- if .subPath }}
+            subPath: {{ .subPath }}
+            {{- end }}
+          {{- end }}
+          {{- range $path, $config := .Values.config }}
+          - name: config
+            mountPath: /usr/share/opensearch-dashboards/config/{{ $path }}
+            subPath: {{ $path }}
+          {{- end }}
+      {{- if .Values.extraContainers }}
+      # Currently some extra blocks accept strings
+      # to continue with backwards compatibility this is being kept
+      # whilst also allowing for yaml to be specified too.
+      {{- if eq "string" (printf "%T" .Values.extraContainers) }}
+{{ tpl .Values.extraContainers . | indent 6 }}
+      {{- else }}
+{{ toYaml .Values.extraContainers | indent 6 }}
+      {{- end }}
+      {{- end }}