Add helm test for chart tgi

- Added helm test for chart tgi - Update the chart README Signed-off-by: Lianhao Lu <[email protected]>
opea-project · Jul 17, 2024 · 9b5def0 · 9b5def0
1 parent 2363819
commit 9b5def0
Show file tree

Hide file tree

Showing 3 changed files with 48 additions and 24 deletions.
diff --git a/helm-charts/common/tgi/README.md b/helm-charts/common/tgi/README.md
@@ -20,16 +20,31 @@ By default, the tgi service will downloading the "bigscience/bloom-560m" which i
 
 If you already cached the model locally, you can pass it to container like this example:
 
-MODELDIR=/home/ubuntu/hfmodels
+MODELDIR=/mnt/opea-models
 
 MODELNAME="/data/models--bigscience--bloom-560m"
 
+## Verify
+
+To verify the installation, run the command `kubectl get pod` to make sure all pods are runinng.
+
+Then run the command `kubectl port-forward svc/tgi 2080:80` to expose the tgi service for access.
+
+Open another terminal and run the following command to verify the service if working:
+
+```console
+curl http://localhost:2080/generate \
+    -X POST \
+    -d '{"inputs":"What is Deep Learning?","parameters":{"max_new_tokens":17, "do_sample": true}}' \
+    -H 'Content-Type: application/json'
+```
+
 ## Values
 
-| Key                     | Type   | Default                                           | Description                                                                                                                                                  |
-| ----------------------- | ------ | ------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------ |
-| LLM_MODEL_ID            | string | `"bigscience/bloom-560m"`                         | Models id from https://huggingface.co/, or predownloaded model directory                                                                                     |
-| port                    | string | `2080`                                            | Hugging Face Text Generation Inference service port                                                                                                          |
-| global.modelUseHostPath | string | `"/mnt/opea-models"`                              | Cached models directory, tgi will not download if the model is cached here. The host path "modelUseHostPath" will be mounted to container as /data directory |
-| image.repository        | string | `"ghcr.io/huggingface/text-generation-inference"` |                                                                                                                                                              |
-| image.tag               | string | `"1.4"`                                           |                                                                                                                                                              |
+| Key                             | Type   | Default                                           | Description                                                                                                                                                                                                           |
+| ------------------------------- | ------ | ------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
+| LLM_MODEL_ID                    | string | `"bigscience/bloom-560m"`                         | Models id from https://huggingface.co/, or predownloaded model directory                                                                                                                                              |
+| global.HUGGINGFACEHUB_API_TOKEN | string | `insert-your-huggingface-token-here`              | Hugging Face API token                                                                                                                                                                                                |
+| global.modelUseHostPath         | string | `"/mnt/opea-models"`                              | Cached models directory, tgi will not download if the model is cached here. The host path "modelUseHostPath" will be mounted to container as /data directory. Set this to null/empty will force it to download model. |
+| image.repository                | string | `"ghcr.io/huggingface/text-generation-inference"` |                                                                                                                                                                                                                       |
+| image.tag                       | string | `"1.4"`                                           |                                                                                                                                                                                                                       |
diff --git a/helm-charts/common/tgi/templates/NOTES.txt b/helm-charts/common/tgi/templates/NOTES.txt
diff --git a/helm-charts/common/tgi/templates/tests/test-pod.yaml b/helm-charts/common/tgi/templates/tests/test-pod.yaml
@@ -0,0 +1,25 @@
+# Copyright (C) 2024 Intel Corporation
+# SPDX-License-Identifier: Apache-2.0
+
+apiVersion: v1
+kind: Pod
+metadata:
+  name: "{{ include "tgi.fullname" . }}-testpod"
+  labels:
+    {{- include "tgi.labels" . | nindent 4 }}
+  annotations:
+    "helm.sh/hook": test
+    #"helm.sh/hook-delete-policy": "hook-succeeded, hook-failure"
+spec:
+  containers:
+    - name: curl
+      image: alpine/curl
+      #image: python:3.10.14
+      command: ['sh', '-c']
+      args:
+        - |
+          curl http://{{ include "tgi.fullname" . }}/generate -sS --fail-with-body \
+            -X POST \
+            -d '{"inputs":"What is Deep Learning?","parameters":{"max_new_tokens":17, "do_sample": true}}' \
+            -H 'Content-Type: application/json'
+  restartPolicy: Never