Skip to content

Commit

Permalink
add k8s support for audioqna (#583)
Browse files Browse the repository at this point in the history
* add k8s support for audioqna

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
  • Loading branch information
Spycsh and pre-commit-ci[bot] authored Aug 13, 2024
1 parent 4f7fc39 commit 0a6bad0
Show file tree
Hide file tree
Showing 3 changed files with 866 additions and 0 deletions.
32 changes: 32 additions & 0 deletions AudioQnA/kubernetes/manifests/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
# Deploy VisualQnA in a Kubernetes Cluster

> [NOTE]
> The following values must be set before you can deploy:
> HUGGINGFACEHUB_API_TOKEN
> You can also customize the "MODEL_ID" and "model-volume"
## Deploy On Xeon
```
cd GenAIExamples/AudioQnA/kubernetes/manifests/xeon
export HUGGINGFACEHUB_API_TOKEN="YourOwnToken"
sed -i "s/insert-your-huggingface-token-here/${HUGGINGFACEHUB_API_TOKEN}/g" audioqna.yaml
kubectl apply -f audioqna.yaml
```
## Deploy On Gaudi
```
cd GenAIExamples/AudioQnA/kubernetes/manifests/gaudi
export HUGGINGFACEHUB_API_TOKEN="YourOwnToken"
sed -i "s/insert-your-huggingface-token-here/${HUGGINGFACEHUB_API_TOKEN}/g" audioqna.yaml
kubectl apply -f audioqna.yaml
```


## Verify Services

Make sure all the pods are running, and restart the audioqna-xxxx pod if necessary.

```bash
kubectl get pods

curl http://${host_ip}:3008/v1/audioqna -X POST -d '{"audio": "UklGRigAAABXQVZFZm10IBIAAAABAAEARKwAAIhYAQACABAAAABkYXRhAgAAAAEA", "max_tokens":64}' -H 'Content-Type: application/json'
```
Loading

0 comments on commit 0a6bad0

Please sign in to comment.