Skip to content

Commit

Permalink
Update the GMC README based on current changes. (#396)
Browse files Browse the repository at this point in the history
* update the GMC README based on current changes.
Signed-off-by: zhlsunshine <[email protected]>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* change the doc based on comments.
Signed-off-by: zhlsunshine <[email protected]>

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
  • Loading branch information
zhlsunshine and pre-commit-ci[bot] authored Sep 5, 2024
1 parent 0094f52 commit 6f7a24e
Showing 1 changed file with 38 additions and 65 deletions.
103 changes: 38 additions & 65 deletions microservices-connector/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -36,83 +36,64 @@ There are `2` components in this repo:
- 1. `manager`: controller manager to handle GMC CRD
- 2. `router`: route the traffic among the microservices defined in GMC

#### How to build these binaries?
### GMC build

#### Binaries building

```sh
make build
```

#### How to build docker images for these 2 components?
#### Docker images building

```sh
make docker.build
```

#### How to delete these components' binaries?
#### Binaries deleting

```sh
make clean
```

### To Deploy on the cluster
### GMC Deployment on K8s cluster

**Build and push your image to the location specified by `CTR_IMG`:**
**GMC Images NOTE:** This image ought to be published in [OPEA docker hub](https://hub.docker.com/u/opea), including [gmcmanager](https://hub.docker.com/r/opea/gmcmanager) and [gmcrouter](https://hub.docker.com/r/opea/gmcrouter). Make sure you have the proper permission to the registry and use the latest images.

```sh
make docker.build docker.push CTR_IMG=<some-registry>/gmcmanager:tag
```
There are 2 methods for deploying GMC on K8s cluster:

**NOTE:** This image ought to be published in the personal registry you specified.
And it is required to have access to pull the image from the working environment.
Make sure you have the proper permission to the registry if the above commands don’t work.
- Deploy via native kubectl
- Deploy via helm chart

**Install GMC CRD**
#### Deploy using native kubectl

```sh
kubectl apply -f config/crd/bases/gmc.opea.io_gmconnectors.yaml
```
There are 3 steps for deploying GMC on K8s cluster as below:

**Copy GMC router manifest**
- Install GMC CRD
- Prepare GenAI Components and GMC Router manifests
- Deploy GMC Manager

```sh
cp $(pwd)/config/gmcrouter/gmc-router.yaml -p $(pwd)/config/manifests/
```
**Deploy GMC NOTE:**

**NOTE:**
before apply the manifests, please replace your own huggingface tokens in the manifests
- Before installting the manifests, please replace your own huggingface tokens
- `MOUNT_DIR` is the `hostPath` to save the models on you cluster hosts, so please make sure it is ready on every node of the k8s nodes and the microservices have enough permission to access it.
- The `SYSTEM_NAMESPACE` should keep the same with the namespace defined in `gmc-manager.yaml` and `gmc-manager-rbac.yaml`
- The configmap name `gmcyaml` is defined in gmcmanager deployment Spec. Please modify accordingly if you want
use a different name for the configmap

```sh
# Install GMC CRD
kubectl apply -f config/crd/bases/gmc.opea.io_gmconnectors.yaml
# Prepare GenAI Components and GMC Router manifests
cp $(pwd)/config/gmcrouter/gmc-router.yaml -p $(pwd)/config/manifests/
export YOUR_HF_TOKEN=<your hugging facetoken>
export MOUNT_DIR=<your model path>
find . -name '*.yaml' -type f -exec sed -i "s#insert-your-huggingface-token-here#$YOURTOKEN#g" {} \;
```

if you have pre-defined directory to save the models on you cluster hosts, please set the path to the manifests

```sh
find . -name '*.yaml' -type f -exec sed -i "s#path: /mnt/opea-models#path: $MOUNT_DIR#g" {} \;
```

**Create Namespace for gmcmanager deployment**
\*\*Create Namespace for gmcmanager deployment

```sh
# Deploy GMC Manager
export SYSTEM_NAMESPACE=system
kubectl create namespace $SYSTEM_NAMESPACE
```

**NOTE:** The `SYSTEM_NAMESPACE` should keep the same with the namespace defined in gmc-manager.yaml and gmc-manager-rbac.yaml.

**Create ConfigMap for GMC to hold GenAI Components and GMC Router manifests**

```sh
kubectl create configmap gmcyaml -n $SYSTEM_NAMESPACE --from-file $(pwd)/config/manifests
```

**NOTE:** The configmap name `gmcyaml' is defined in gmcmanager deployment Spec. Please modify accordingly if you want
use a different name for the configmap.

**Install GMC manager**

```sh
kubectl apply -f $(pwd)/config/rbac/gmc-manager-rbac.yaml
kubectl apply -f $(pwd)/config/manager/gmc-manager.yaml
```
Expand All @@ -125,33 +106,25 @@ NAME READY STATUS RESTARTS AGE
gmc-controller-78f9c748cb-ltcdv 1/1 Running 0 3m
```

### Next Step

Please refer to this [usage_guide](./usage_guide.md) for sample use cases.

### To Uninstall

**Delete the instances (CRs) from the cluster:**
**To Uninstall via kubectl**

```sh
# Delete the instances (CRs) from the cluster
kubectl delete -k config/samples/
```

**Delete the APIs(CRDs) from the cluster:**

```sh
# Delete the APIs(CRDs) from the cluster
make uninstall
# UnDeploy the controller from the cluster
make undeploy
```

**UnDeploy the controller from the cluster:**
#### Deploy via helm chart

```sh
make undeploy
```
Please refer to [helm chart README](./helm/README.md) for deploying GMC using helm chart.

### Deploy using helm chart
### Next Step

Please refer to the [helm chart README](./helm/README.md) for deploying GMC using helm chart.
Please refer to [usage_guide](./usage_guide.md) for sample use cases.
Please refer to [advanced_use_cases](./config/samples/ChatQnA/use_cases.md) for more use cases based on ChatQnA example.

### Troubleshooting guide

Expand Down

0 comments on commit 6f7a24e

Please sign in to comment.