Skip to content

Commit

Permalink
doc: fix headings and indenting (#404)
Browse files Browse the repository at this point in the history
* remove trailing punctuation on headings
* fix indenting within lists
* catch some misspellings and remove use of please

Signed-off-by: David B. Kinder <[email protected]>
  • Loading branch information
dbkinder authored Sep 6, 2024
1 parent 6f7a24e commit c10bca1
Show file tree
Hide file tree
Showing 13 changed files with 63 additions and 67 deletions.
12 changes: 6 additions & 6 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,31 +16,31 @@ The GenAIInfra repository is organized under four main directories, which includ

## Prerequisite

GenAIInfra uses Kubernetes as the cloud native infrastructure. Please follow the steps below to prepare the Kubernetes environment.
GenAIInfra uses Kubernetes as the cloud native infrastructure. Follow the steps below to prepare the Kubernetes environment.

### Setup Kubernetes cluster

Please follow [Kubernetes official setup guide](https://kubernetes.io/docs/setup/) to setup Kubernetes. We recommend to use Kubernetes with version >= 1.27.
Follow [Kubernetes official setup guide](https://kubernetes.io/docs/setup/) to setup Kubernetes. We recommend to use Kubernetes with version >= 1.27.

There are different methods to setup Kubernetes production cluster, such as [kubeadm](https://kubernetes.io/docs/setup/production-environment/tools/kubeadm/), [kubespray](https://kubespray.io/), and [more](https://kubernetes.io/docs/setup/production-environment/tools/).

NOTE: We recommend to use containerd when choosing the container runtime during Kubernetes setup. Docker engine is also verified on Ubuntu 22.04 and above.

### (Optional) To run GenAIInfra on [Intel Gaudi](https://habana.ai/products/) product:
### (Optional) To run GenAIInfra on [Intel Gaudi](https://habana.ai/products/) product

The following steps are optional. They're only required if you want to run the workloads on Intel Gaudi product.

1. Please check the [support matrix](https://docs.habana.ai/en/latest/Support_Matrix/Support_Matrix.html) to make sure that environment meets the requirements.
1. Check the [support matrix](https://docs.habana.ai/en/latest/Support_Matrix/Support_Matrix.html) to make sure that environment meets the requirements.

2. [Install Intel Gaudi software stack](https://docs.habana.ai/en/latest/Installation_Guide/Bare_Metal_Fresh_OS.html#driver-fw-install-bare).

3. [Install and setup container runtime](https://docs.habana.ai/en/latest/Installation_Guide/Bare_Metal_Fresh_OS.html#set-up-container-usage), based on the container runtime used by Kubernetes.

NOTE: Please make sure you configure the appropriate container runtime based on the type of container runtime you installed during Kubernetes setup.
NOTE: Make sure you configure the appropriate container runtime based on the type of container runtime you installed during Kubernetes setup.

4. [Install Intel Gaudi device plugin for Kubernetes](https://docs.habana.ai/en/latest/Orchestration/Gaudi_Kubernetes/Device_Plugin_for_Kubernetes.html).

Alternatively, Intel provides a base operator to manage the Gaudi software stack. Please refer to [this file](kubernetes-addons/Intel-Gaudi-Base-Operator/README.md) for details.
Alternatively, Intel provides a base operator to manage the Gaudi software stack. Refer to [this file](kubernetes-addons/Intel-Gaudi-Base-Operator/README.md) for details.

## Usages

Expand Down
4 changes: 2 additions & 2 deletions helm-charts/common/asr/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ Helm chart for deploying asr microservice.

asr depends on whisper, you should set ASR_ENDPOINT endpoints before start.

## (Option1): Installing the chart separately:
## (Option1): Installing the chart separately

First, you need to install the whisper chart, please refer to the [whisper](../whisper) chart for more information.

Expand All @@ -19,7 +19,7 @@ helm dependency update
helm install asr . --set ASR_ENDPOINT=${ASR_ENDPOINT}
```

## (Option2): Installing the chart with dependencies automatically:
## (Option2): Installing the chart with dependencies automatically

```console
cd GenAIInfra/helm-charts/common/asr
Expand Down
4 changes: 2 additions & 2 deletions helm-charts/common/data-prep/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ Helm chart for deploying data-prep microservice.

data-prep will use redis and tei service, please specify the endpoints.

## (Option1): Installing the chart separately:
## (Option1): Installing the chart separately

First, you need to install the tei and redis-vector-db chart, please refer to the [tei](../tei) and [redis-vector-db](../redis-vector-db) for more information.

Expand All @@ -20,7 +20,7 @@ helm dependency update
helm install data-prep . --set REDIS_URL=${REDIS_URL} --set TEI_EMBEDDING_ENDPOINT=${TEI_EMBEDDING_ENDPOINT}
```

## (Option2): Installing the chart with dependencies automatically:
## (Option2): Installing the chart with dependencies automatically

```console
cd GenAIInfra/helm-charts/common/data-prep
Expand Down
4 changes: 2 additions & 2 deletions helm-charts/common/embedding-usvc/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ Helm chart for deploying embedding microservice.

embedding-usvc depends on TEI, set TEI_EMBEDDING_ENDPOINT.

## (Option1): Installing the chart separately:
## (Option1): Installing the chart separately

First, you need to install the tei chart, please refer to the [tei](../tei) chart for more information.

Expand All @@ -19,7 +19,7 @@ helm dependency update
helm install embedding-usvc . --set TEI_EMBEDDING_ENDPOINT=${TEI_EMBEDDING_ENDPOINT}
```

## (Option2): Installing the chart with dependencies automatically:
## (Option2): Installing the chart with dependencies automatically

```console
cd GenAIInfra/helm-charts/common/embedding-usvc
Expand Down
4 changes: 2 additions & 2 deletions helm-charts/common/guardrails-usvc/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ Helm chart for deploying LLM microservice.

guardrails-usvc depends on TGI, you should set TGI_LLM_ENDPOINT as tgi endpoint.

## (Option1): Installing the chart separately:
## (Option1): Installing the chart separately

First, you need to install the tgi chart, please refer to the [tgi](../tgi) chart for more information. Please use model `meta-llama/Meta-Llama-Guard-2-8B` during installation.

Expand All @@ -21,7 +21,7 @@ helm dependency update
helm install guardrails-usvc . --set global.HUGGINGFACEHUB_API_TOKEN=${HFTOKEN} --set SAFETY_GUARD_ENDPOINT=${SAFETY_GUARD_ENDPOINT} --set SAFETY_GUARD_MODEL_ID=${SAFETY_GUARD_MODEL_ID} --wait
```

## (Option2): Installing the chart with dependencies automatically:
## (Option2): Installing the chart with dependencies automatically

```console
cd GenAIInfra/helm-charts/common/guardrails-usvc
Expand Down
4 changes: 2 additions & 2 deletions helm-charts/common/llm-uservice/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ Helm chart for deploying LLM microservice.

llm-uservice depends on TGI, you should set TGI_LLM_ENDPOINT as tgi endpoint.

## (Option1): Installing the chart separately:
## (Option1): Installing the chart separately

First, you need to install the tgi chart, please refer to the [tgi](../tgi) chart for more information.

Expand All @@ -20,7 +20,7 @@ helm dependency update
helm install llm-uservice . --set global.HUGGINGFACEHUB_API_TOKEN=${HFTOKEN} --set TGI_LLM_ENDPOINT=${TGI_LLM_ENDPOINT} --wait
```

## (Option2): Installing the chart with dependencies automatically:
## (Option2): Installing the chart with dependencies automatically

```console
cd GenAIInfra/helm-charts/common/llm-uservice
Expand Down
4 changes: 2 additions & 2 deletions helm-charts/common/reranking-usvc/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ Helm chart for deploying reranking microservice.

reranking-usvc depends on teirerank, set the TEI_RERANKING_ENDPOINT as teirerank endpoint.

## (Option1): Installing the chart separately:
## (Option1): Installing the chart separately

First, you need to install the teirerank chart, please refer to the [teirerank](../teirerank) chart for more information.

Expand All @@ -19,7 +19,7 @@ helm dependency update
helm install reranking-usvc . --set TEI_RERANKING_ENDPOINT=${TEI_RERANKING_ENDPOINT}
```

## (Option2): Installing the chart with dependencies automatically:
## (Option2): Installing the chart with dependencies automatically

```console
cd GenAIInfra/helm-charts/common/reranking-usvc
Expand Down
8 changes: 4 additions & 4 deletions helm-charts/common/retriever-usvc/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,11 +4,11 @@ Helm chart for deploying Retriever microservice.

retriever-usvc depends on redis and tei, you should set these endpoints before start.

## (Option1): Installing the chart separately:
## (Option1): Installing the chart separately

First, you need to install the tei and redis-vector-db chart, please refer to the [tei](../tei) and [redis-vector-db](../redis-vector-db) for more information.
First, you need to install the tei and redis-vector-db chart, refer to the [tei](../tei) and [redis-vector-db](../redis-vector-db) for more information.

After you've deployted the tei and redis-vector-db chart successfully, please run `kubectl get svc` to get the service endpoint and URL respectively, i.e. `http://tei`, `redis://redis-vector-db:6379`.
After you've deployed the tei and redis-vector-db chart successfully, run `kubectl get svc` to get the service endpoint and URL respectively, i.e. `http://tei`, `redis://redis-vector-db:6379`.

To install retriever-usvc chart, run the following:

Expand All @@ -20,7 +20,7 @@ helm dependency update
helm install retriever-usvc . --set REDIS_URL=${REDIS_URL} --set TEI_EMBEDDING_ENDPOINT=${TEI_EMBEDDING_ENDPOINT}
```

## (Option2): Installing the chart with dependencies automatically:
## (Option2): Installing the chart with dependencies automatically

```console
cd GenAIInfra/helm-charts/common/retriever-usvc
Expand Down
4 changes: 2 additions & 2 deletions helm-charts/common/tts/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ Helm chart for deploying tts microservice.

tts depends on speecht5, you should set TTS_ENDPOINT endpoints before start.

## (Option1): Installing the chart separately:
## (Option1): Installing the chart separately

First, you need to install the speecht5 chart, please refer to the [speecht5](../speecht5) chart for more information.

Expand All @@ -19,7 +19,7 @@ helm dependency update
helm install tts . --set TTS_ENDPOINT=${TTS_ENDPOINT}
```

## (Option2): Installing the chart with dependencies automatically:
## (Option2): Installing the chart with dependencies automatically

```console
cd GenAIInfra/helm-charts/common/tts
Expand Down
4 changes: 2 additions & 2 deletions helm-charts/common/web-retriever/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ Helm chart for deploying Web Retriever microservice.

Web retriever depends on tei, you should set TEI_EMBEDDING_ENDPOINT endpoints before start.

## (Option1): Installing the chart separately:
## (Option1): Installing the chart separately

First, you need to install the tei chart, please refer to the [tei](../tei) chart for more information.

Expand All @@ -21,7 +21,7 @@ export GOOGLE_CSE_ID="yourownid"
helm install web-retriever . --set TEI_EMBEDDING_ENDPOINT=${TEI_EMBEDDING_ENDPOINT} --set GOOGLE_API_KEY=${GOOGLE_API_KEY} --set GOOGLE_CSE_ID=${GOOGLE_CSE_ID}
```

## (Option2): Installing the chart with dependencies automatically:
## (Option2): Installing the chart with dependencies automatically

```console
cd GenAIInfra/helm-charts/common/web-retriever
Expand Down
24 changes: 12 additions & 12 deletions microservices-connector/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,8 +3,8 @@
This repo defines the GenAI Microservice Connector(GMC) for OPEA projects. GMC can be used to compose and adjust GenAI pipelines dynamically
on kubernetes. It can leverage the microservices provided by [GenAIComps](https://github.com/opea-project/GenAIComps) and external services to compose GenAI pipelines. External services might be running in a public cloud or on-prem by providing an URL and access details such as an API key and ensuring there is network connectivity. It also allows users to adjust the pipeline on the fly like switching to a different Large language Model(LLM), adding new functions into the chain(like adding guardrails),etc. GMC supports different types of steps in the pipeline, like sequential, parallel and conditional.

Please refer to [usage_guide](./usage_guide.md) for sample use cases.
Please refer to [chatqna_use_cases](./config/samples/ChatQnA/use_cases.md) for more ChatQnA use cases.
Refer to [usage_guide](./usage_guide.md) for sample use cases.
Refer to [chatqna_use_cases](./config/samples/ChatQnA/use_cases.md) for more ChatQnA use cases.

## Description

Expand All @@ -31,10 +31,10 @@ Istio Service Mesh can also be leveraged to facilicate communication between mic

### Introduction

There are `2` components in this repo:
There are two components in this repo:

- 1. `manager`: controller manager to handle GMC CRD
- 2. `router`: route the traffic among the microservices defined in GMC
- `manager`: controller manager to handle GMC CRD
- `router`: route the traffic among the microservices defined in GMC

### GMC build

Expand Down Expand Up @@ -75,10 +75,10 @@ There are 3 steps for deploying GMC on K8s cluster as below:

**Deploy GMC NOTE:**

- Before installting the manifests, please replace your own huggingface tokens
- `MOUNT_DIR` is the `hostPath` to save the models on you cluster hosts, so please make sure it is ready on every node of the k8s nodes and the microservices have enough permission to access it.
- Before installting the manifests, replace your own huggingface tokens
- `MOUNT_DIR` is the `hostPath` to save the models on you cluster hosts, so make sure it is ready on every node of the k8s nodes and the microservices have enough permission to access it.
- The `SYSTEM_NAMESPACE` should keep the same with the namespace defined in `gmc-manager.yaml` and `gmc-manager-rbac.yaml`
- The configmap name `gmcyaml` is defined in gmcmanager deployment Spec. Please modify accordingly if you want
- The configmap name `gmcyaml` is defined in gmcmanager deployment Spec. Modify accordingly if you want
use a different name for the configmap

```sh
Expand Down Expand Up @@ -119,13 +119,13 @@ make undeploy

#### Deploy via helm chart

Please refer to [helm chart README](./helm/README.md) for deploying GMC using helm chart.
Refer to [helm chart README](./helm/README.md) for deploying GMC using helm chart.

### Next Step

Please refer to [usage_guide](./usage_guide.md) for sample use cases.
Please refer to [advanced_use_cases](./config/samples/ChatQnA/use_cases.md) for more use cases based on ChatQnA example.
Refer to [usage_guide](./usage_guide.md) for sample use cases.
Refer to [advanced_use_cases](./config/samples/ChatQnA/use_cases.md) for more use cases based on ChatQnA example.

### Troubleshooting guide

Please refer to this [troubleshooting_guide](./troubleshooting_guide.md) for identifying GMC Custom Resource issues.
Refer to this [troubleshooting_guide](./troubleshooting_guide.md) for identifying GMC Custom Resource issues.
22 changes: 9 additions & 13 deletions microservices-connector/config/samples/ChatQnA/use_cases.md
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@ For Gaudi:
- tgi-service: ghcr.io/huggingface/tgi-gaudi:1.2.1

> [NOTE]
> Please refer to [Xeon README](https://github.com/opea-project/GenAIExamples/blob/main/ChatQnA/docker/xeon/README.md) or [Gaudi README](https://github.com/opea-project/GenAIExamples/blob/main/ChatQnA/docker/gaudi/README.md) to build the OPEA images. These too will be available on Docker Hub soon to simplify use.
> Refer to [Xeon README](https://github.com/opea-project/GenAIExamples/blob/main/ChatQnA/docker/xeon/README.md) or [Gaudi README](https://github.com/opea-project/GenAIExamples/blob/main/ChatQnA/docker/gaudi/README.md) to build the OPEA images. These too will be available on Docker Hub soon to simplify use.
## Deploy ChatQnA pipeline

Expand Down Expand Up @@ -138,9 +138,8 @@ This involves deploying the ChatQnA custom resource. You can use `chatQnA_datapr
chatqa http://router-service.chatqa.svc.cluster.local:8080 10/0/10 3m
```

> [NOTE]
Comparing with `General ChatQnA with preset RAG data`, there should be `10` microservices, the extra one is the microservice of `dataprep`.
> [NOTE]
> Comparing with `General ChatQnA with preset RAG data`, there should be `10` microservices, the extra one is the microservice of `dataprep`.
4. Deploy a client pod to test the application

Expand All @@ -162,9 +161,8 @@ Comparing with `General ChatQnA with preset RAG data`, there should be `10` micr
kubectl exec "$CLIENT_POD" -n chatqa -- curl -s --no-buffer $accessUrl -X POST '{"text":"What are the key features of Intel Gaudi?","parameters":{"max_new_tokens":100, "do_sample": true}}' -H 'Content-Type: application/json'
```

> [NOTE]
You can remove your ChatQnA pipeline by executing standard Kubernetes kubectl commands to remove a custom resource. Verify it was removed by executing kubectl get pods in the chatqa namespace.
> [NOTE]
> You can remove your ChatQnA pipeline by executing standard Kubernetes kubectl commands to remove a custom resource. Verify it was removed by executing kubectl get pods in the chatqa namespace.
### ChatQnA supports multiple LLM models

Expand All @@ -191,9 +189,8 @@ This involves deploying the ChatQnA custom resource. You can use `chatQnA_switch
switch http://router-service.switch.svc.cluster.local:8080 15/0/15 83s
```

> [NOTE]
Comparing with `General ChatQnA with preset RAG data`, there should be `15` microservices, the extra are the microservices for different embedding models and LLM models.
> [NOTE]
> Comparing with `General ChatQnA with preset RAG data`, there should be `15` microservices, the extra are the microservices for different embedding models and LLM models.
4. Deploy a client pod to test the application

Expand All @@ -217,6 +214,5 @@ Comparing with `General ChatQnA with preset RAG data`, there should be `15` micr
kubectl exec "$CLIENT_POD" -n switch -- curl -s --no-buffer $accessUrl -X POST -d '{"text":"What are the key features of Intel Gaudi?", "model-id":"llama", "embedding-model-id":"small", "parameters":{"max_new_tokens":50, "do_sample": true}}' -H 'Content-Type: application/json'
```

> [NOTE]
Showing as above, user can switch the LLM models in runtime by changing the request body, such as adding `"model-id":"llama"` in request body to use the Llama model or changing it into `"model-id":"intel"` to use the Intel model.
> [NOTE]
> Showing as above, user can switch the LLM models in runtime by changing the request body, such as adding `"model-id":"llama"` in request body to use the Llama model or changing it into `"model-id":"intel"` to use the Intel model.
Loading

0 comments on commit c10bca1

Please sign in to comment.