Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Update the helm chart #333

Closed
wants to merge 3 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion docs/architecture/decisions/0004-documentation.md
Original file line number Diff line number Diff line change
Expand Up @@ -110,7 +110,7 @@ following implementation of this ADR proposal:
┃ ┣ carbon-aware-webapi.md
┃ ┗ carbon-aware-cli.md
┣ samples/
┃ ┣ helmexample/
┃ ┣ helm-deploy/
┃ ┃ ┗ README.md
┃ ┗ python-proxy-server/
┃ ┗ README.md
Expand Down
File renamed without changes.
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
apiVersion: v2
name: helmexample
name: carbon-aware-sdk
description: A Helm chart for Kubernetes

# A chart can be either an 'application' or a 'library' chart.
Expand Down
162 changes: 162 additions & 0 deletions samples/helm-deploy/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,162 @@
# How-To: Deploy to Kubernetes using Helm

## Setup Dev Environment (optional)

The easiest way to setup your environment is to install [Kind](https://kind.sigs.k8s.io/) or [Minikube](https://minikube.sigs.k8s.io/docs/start/). It is also valid to setup a cloud provider kubernetes cluster (i.e. AKS, EKS, GKE, ...)

## Push an image to the Docker registry

The following steps illustrates how to push a webservice image in order to be
deployed in Kubernetes clsuter.

1. Build an image using the following
[Dockerfile](.../../../../src/CarbonAware.WebApi/src/Dockerfile)

```sh
cd <Dockerfile path>
docker build -t <registry>/myapp:v1 .
```

1. Login into docker using the username and password credentials that are needed in
order to push.

```sh
docker login
```

1. Push to Docker registry

```sh
docker push <registry>/myapp:v1
```
Comment on lines +7 to +31
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This section is not needed because default values.yaml refers container image from GitHub Container Registry.


Comment on lines +7 to +32
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
## Push an image to the Docker registry
The following steps illustrates how to push a webservice image in order to be
deployed in Kubernetes clsuter.
1. Build an image using the following
[Dockerfile](.../../../../src/CarbonAware.WebApi/src/Dockerfile)
```sh
cd <Dockerfile path>
docker build -t <registry>/myapp:v1 .
```
1. Login into docker using the username and password credentials that are needed in
order to push.
```sh
docker login
```
1. Push to Docker registry
```sh
docker push <registry>/myapp:v1
```


## Create a new Helm chart (optional)

Run `helm` to ensure you have the helm CLI running properly. Then you can create
a new helm chart by running

```bash
helm create <chart-name>
```

## Setting up the Helm chart

Once you've got your helm chart open (whether from scratch or existing), the
main files you will likely be working with are `Chart.yaml` and `values.yaml`.

### Chart.yaml

In `Chart.yaml`, we won't need to make any changes but make note of the chart
`name`, as you will need to reference it in commands later on.

### Values.yaml
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should guide account settings for data source.


In `values.yaml`, we need to change a couple fields. In the `image` section, you
will need to set

- The `repository` field to be `<image-repository>`
- The `pullPolicy` field to be `IfNotPresent` (pulls a new image if not present)
or `Always` (pulls a new image every time).
- The `tag` field if you need a particuar tag version

Set the `nameOverride` and `fullNameOverride` fields to make it easier to
reference your helm chart, and ensure they are not the same.

In the `serviceAccount` section, ensure that
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

serviceAccount is no longer needed.


- The `name` field is set to the name of the helm chart from `Chart.yaml`.

Comment on lines +66 to +69
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
In the `serviceAccount` section, ensure that
- The `name` field is set to the name of the helm chart from `Chart.yaml`.

In the `monitorConfig` section, you will need to adjust the `liveness` and
`readiness` that the helm chart will ping to ensure the sdk is ready. As a
default, you should set the path to `/health`.
Comment on lines +70 to +72
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Where is monitorConfig? Liveness / Startup probe should be set into the deployment yaml.



### Installing your helm chart

To install your helm chart, you should run

```bash
helm install carbon-aware ./samples/helm-deploy/
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe it is better to good with -n <namespace> and --create-namespace. IMHO carbon-aware-sdk is good for namespace.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
helm install carbon-aware ./samples/helm-deploy/
helm install -n carbon-aware-sdk --create-namespace carbon-aware ./samples/helm-deploy/

```

(If you run into an error, see the [troubleshooting](#troubleshooting) section.
below.)

### Deploying your helm chart

If the installation was successful, helm should give you a console out message
for your to copy and paste to deploy the chart. We've replicated below for quick
reference (you will need to fill in `nameOverride` and `fullNameOverride`):

```bash
NAME: carbon-aware
LAST DEPLOYED: Thu Apr 13 17:39:51 2023
NAMESPACE: default
STATUS: deployed
REVISION: 1
TEST SUITE: None
NOTES:
1. Get the application URL by running these commands:
export POD_NAME=$(kubectl get pods --namespace default -l "app.kubernetes.io/name=carbon-aware-sdk,app.kubernetes.io/instance=carbon-aware" -o jsonpath="{.items[0].metadata.name}")
export CONTAINER_PORT=$(kubectl get pod --namespace default $POD_NAME -o jsonpath="{.spec.containers[0].ports[0].containerPort}")
echo "Visit http://127.0.0.1:8080 to use your application"
kubectl --namespace default port-forward $POD_NAME 8080:$CONTAINER_PORT
```

If the deployment works properly, you should be able to visit the link they
provide and use it to query the SDK

## Troubleshooting

### Error connecting to the kubernetes service

If you get

```text
Error: INSTALLATION FAILED: Kubernetes cluster unreachable: Get "http://localhost:8080/version": dial tcp 127.0.0.1:8080: connect: connection refused
```

that means that helm cannot connect to kubernetes. Helm connects to kuberentes
either via the $KUBECONFIG env variable, or (if it's not set) looks for the
default location kubectl files are location (`~/.kube/config`). This may occur
the first time to try connecting the helm chart or if you clear the
files/variables.

### Error installing the helm chart

If you get `Error: INSTALLATION FAILED: cannot re-use a name that is still in use` when you
tried to install the helm chart, it means there is still an instance of that
chart installed. If you started this instance, you can simply skip the install
step and continue. If you're unsure that it's the right installation, or you've
made changes, first run `helm uninstall <fullNameOverride>`. Once it's
uninstalled, you can redo the helm install step.


### Error deploying the helm chart

If you get an error deploying the helm chart and have ensured the image is
pulling properly, one possible error may be with the liveliness and readiness
probes. If those are failing, the deployment will fail to start properly. Ensure
that the paths provided in the `deployment.yaml` file are valid and that the sdk
can actual spin up correctly.

### Useful kubectl commands to check on cluster

- List all deployments in all namespaces:
`kubectl get deployments --all-namespaces=true`
- List details about a specific deployment:
`kubectl describe deployment <deployment-name> --namespace <namespace-name>`
- Delete a specific deployment: `kubectl delete deployment <deployment-name>`
- List pods: `kubectl get pods`
- Check on a specific pod: `kubectl describe pod <pod-name>`
- Get logs on a specific pod: `kubectl logs <pod-name>`
- Delete a specific pod: `kubectl delete pod <pod-name>`

## References

- Helm 3: [docs](https://helm.sh/docs/),
[image docs](https://helm.sh/docs/chart_best_practices/pods/#images)

- ContainIQ:
[Troubleshooting ImagePullBackOff Error](https://www.containiq.com/post/kubernetes-imagepullbackoff)
Original file line number Diff line number Diff line change
@@ -1,21 +1,15 @@
1. Get the application URL by running these commands:
{{- if .Values.ingress.enabled }}
{{- range $host := .Values.ingress.hosts }}
{{- range .paths }}
http{{ if $.Values.ingress.tls }}s{{ end }}://{{ $host.host }}{{ .path }}
{{- end }}
{{- end }}
{{- else if contains "NodePort" .Values.service.type }}
export NODE_PORT=$(kubectl get --namespace {{ .Release.Namespace }} -o jsonpath="{.spec.ports[0].nodePort}" services {{ include "helmexample.fullname" . }})
{{- if contains "NodePort" .Values.service.type }}
export NODE_PORT=$(kubectl get --namespace {{ .Release.Namespace }} -o jsonpath="{.spec.ports[0].nodePort}" services {{ include "carbon-aware-sdk.fullname" . }})
export NODE_IP=$(kubectl get nodes --namespace {{ .Release.Namespace }} -o jsonpath="{.items[0].status.addresses[0].address}")
echo http://$NODE_IP:$NODE_PORT
{{- else if contains "LoadBalancer" .Values.service.type }}
NOTE: It may take a few minutes for the LoadBalancer IP to be available.
You can watch the status of by running 'kubectl get --namespace {{ .Release.Namespace }} svc -w {{ include "helmexample.fullname" . }}'
export SERVICE_IP=$(kubectl get svc --namespace {{ .Release.Namespace }} {{ include "helmexample.fullname" . }} --template "{{"{{ range (index .status.loadBalancer.ingress 0) }}{{.}}{{ end }}"}}")
You can watch the status of by running 'kubectl get --namespace {{ .Release.Namespace }} svc -w {{ include "carbon-aware-sdk.fullname" . }}'
export SERVICE_IP=$(kubectl get svc --namespace {{ .Release.Namespace }} {{ include "carbon-aware-sdk.fullname" . }} --template "{{"{{ range (index .status.loadBalancer.ingress 0) }}{{.}}{{ end }}"}}")
echo http://$SERVICE_IP:{{ .Values.service.port }}
{{- else if contains "ClusterIP" .Values.service.type }}
export POD_NAME=$(kubectl get pods --namespace {{ .Release.Namespace }} -l "app.kubernetes.io/name={{ include "helmexample.name" . }},app.kubernetes.io/instance={{ .Release.Name }}" -o jsonpath="{.items[0].metadata.name}")
export POD_NAME=$(kubectl get pods --namespace {{ .Release.Namespace }} -l "app.kubernetes.io/name={{ include "carbon-aware-sdk.name" . }},app.kubernetes.io/instance={{ .Release.Name }}" -o jsonpath="{.items[0].metadata.name}")
export CONTAINER_PORT=$(kubectl get pod --namespace {{ .Release.Namespace }} $POD_NAME -o jsonpath="{.spec.containers[0].ports[0].containerPort}")
echo "Visit http://127.0.0.1:8080 to use your application"
kubectl --namespace {{ .Release.Namespace }} port-forward $POD_NAME 8080:$CONTAINER_PORT
Expand Down
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
{{/*
Expand the name of the chart.
*/}}
{{- define "helmexample.name" -}}
{{- define "carbon-aware-sdk.name" -}}
{{- default .Chart.Name .Values.nameOverride | trunc 63 | trimSuffix "-" }}
{{- end }}

Expand All @@ -10,7 +10,7 @@ Create a default fully qualified app name.
We truncate at 63 chars because some Kubernetes name fields are limited to this (by the DNS naming spec).
If release name contains chart name it will be used as a full name.
*/}}
{{- define "helmexample.fullname" -}}
{{- define "carbon-aware-sdk.fullname" -}}
{{- if .Values.fullnameOverride }}
{{- .Values.fullnameOverride | trunc 63 | trimSuffix "-" }}
{{- else }}
Expand All @@ -26,16 +26,16 @@ If release name contains chart name it will be used as a full name.
{{/*
Create chart name and version as used by the chart label.
*/}}
{{- define "helmexample.chart" -}}
{{- define "carbon-aware-sdk.chart" -}}
{{- printf "%s-%s" .Chart.Name .Chart.Version | replace "+" "_" | trunc 63 | trimSuffix "-" }}
{{- end }}

{{/*
Common labels
*/}}
{{- define "helmexample.labels" -}}
helm.sh/chart: {{ include "helmexample.chart" . }}
{{ include "helmexample.selectorLabels" . }}
{{- define "carbon-aware-sdk.labels" -}}
helm.sh/chart: {{ include "carbon-aware-sdk.chart" . }}
{{ include "carbon-aware-sdk.selectorLabels" . }}
{{- if .Chart.AppVersion }}
app.kubernetes.io/version: {{ .Chart.AppVersion | quote }}
{{- end }}
Expand All @@ -45,17 +45,18 @@ app.kubernetes.io/managed-by: {{ .Release.Service }}
{{/*
Selector labels
*/}}
{{- define "helmexample.selectorLabels" -}}
app.kubernetes.io/name: {{ include "helmexample.name" . }}
{{- define "carbon-aware-sdk.selectorLabels" -}}
app.kubernetes.io/name: {{ include "carbon-aware-sdk.name" . }}
app.kubernetes.io/instance: {{ .Release.Name }}
name: carbon-aware-sdk
{{- end }}

{{/*
Create the name of the service account to use
*/}}
{{- define "helmexample.serviceAccountName" -}}
{{- define "carbon-aware-sdk.serviceAccountName" -}}
{{- if .Values.serviceAccount.create }}
{{- default (include "helmexample.fullname" .) .Values.serviceAccount.name }}
{{- default (include "carbon-aware-sdk.fullname" .) .Values.serviceAccount.name }}
{{- else }}
{{- default "default" .Values.serviceAccount.name }}
{{- end }}
Expand Down
45 changes: 45 additions & 0 deletions samples/helm-deploy/templates/deployment.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,45 @@
apiVersion: apps/v1
kind: Deployment
metadata:
name: {{ include "carbon-aware-sdk.fullname" . }}
namespace: {{ .Values.namespace }}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should use .Release.namespace rather than the value from values.yaml. https://helm.sh/docs/chart_template_guide/builtin_objects/

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
namespace: {{ .Values.namespace }}
namespace: {{ .Release.namespace }}

labels:
{{- include "carbon-aware-sdk.labels" . | nindent 4 }}
spec:
replicas: {{ .Values.replicaCount }}
selector:
matchLabels:
{{- include "carbon-aware-sdk.selectorLabels" . | nindent 6 }}
template:
metadata:
{{- with .Values.podAnnotations }}
annotations:
{{- toYaml . | nindent 8 }}
{{- end }}
labels:
{{- include "carbon-aware-sdk.selectorLabels" . | nindent 8 }}
spec:
containers:
- name: {{ .Values.image.name }}
image: "{{ .Values.image.repository }}:{{ .Values.image.tag }}"
imagePullPolicy: {{ .Values.image.pullPolicy }}
command:
- sh
- -c
- cd /app && sed "s/username/{{ .Values.image.auth.username }}/" -i appsettings.json && sed "s/password/{{ .Values.image.auth.password }}/" -i appsettings.json && dotnet CarbonAware.WebApi.dll
Comment on lines +26 to +29
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

command is needed? WebAPI container has ENTRYPOINT.

For configuration, we should support environment value and/or JSON in deployment.yaml.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This command to inject the username and password into appsettings.json. This file doesn't support env variable afaik.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can set username and account through environment variable: https://github.com/Green-Software-Foundation/carbon-aware-sdk/blob/dev/docs/quickstart.md#setting-up-the-web-api

Otherwise we should use ConfigMap here.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It would be good to use Secret to store credentials. So it is better approach to have both secrets and configmap (as JSON).

ports:
- name: api-server-port
containerPort: 7031
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should it be 80?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

not necessary

Copy link
Member

@YaSuenag YaSuenag Jun 17, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is 7031 for? I believe WebAPI container will expose 80, and it should be set as containerPort.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
containerPort: 7031
containerPort: 80


Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
startupProbe:
httpGet:
path: /health
port: 80
failureThreshold: 10
periodSeconds: 5
livenessProbe:
httpGet:
path: /health
port: 80
periodSeconds: 30

{{- with .Values.nodeSelector }}
nodeSelector:
{{- toYaml . | nindent 8 }}
{{- end }}
{{- with .Values.affinity }}
affinity:
{{- toYaml . | nindent 8 }}
{{- end }}
{{- with .Values.tolerations }}
tolerations:
{{- toYaml . | nindent 8 }}
{{- end }}
15 changes: 15 additions & 0 deletions samples/helm-deploy/templates/service.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
apiVersion: v1
kind: Service
metadata:
name: carbon-aware-sdk
namespace: {{ .Values.namespace }}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should use .Release.namespace rather than the value from values.yaml. https://helm.sh/docs/chart_template_guide/builtin_objects/

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
namespace: {{ .Values.namespace }}
namespace: {{ .Release.namespace }}

labels:
{{- include "carbon-aware-sdk.labels" . | nindent 4 }}
spec:
type: {{ .Values.service.type }}
ports:
- port: {{ .Values.service.port }}
targetPort: api-server-port
Comment on lines +11 to +12
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess you want to specify the port what the user want to expose in values.yaml, and also it should be so. Thus api-server-port should be set to port, and also .Values.service,.port should be set to targetPort?

protocol: TCP
selector:
{{- include "carbon-aware-sdk.selectorLabels" . | nindent 4 }}
5 changes: 5 additions & 0 deletions samples/helm-deploy/templates/serviceaccount.yaml
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This YAML is no longer needed.

Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
apiVersion: v1
kind: ServiceAccount
metadata:
name: {{ .Values.rbac.serviceAccountName }}
namespace: {{ .Values.namespace }}
27 changes: 27 additions & 0 deletions samples/helm-deploy/values.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
# Default values for carbon-aware-sdk.
# This is a YAML-formatted file.
# Declare variables to be passed into your templates.

replicaCount: 1
namespace: default

image:
name: carbon-aware-sdk
repository: ghcr.io/green-software-foundation/carbon-aware-sdk
pullPolicy: IfNotPresent
tag: "latest"
auth:
username: username
password: password
Comment on lines +13 to +15
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should consider other parameters (e.g. data source) if we can set them via environment variables and/or ConfigMap.


service:
type: ClusterIP
port: 80


nameOverride: ""
fullnameOverride: ""
podAnnotations: {}
nodeSelector: {}
tolerations: []
affinity: {}
Loading