Skip to content

Commit

Permalink
Fix wrong endpoint for tei embedding gaudi wrapper (#1043)
Browse files Browse the repository at this point in the history
* fix doc index retriever embed issue on gaudi

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* align test router with examples

* align readme

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
  • Loading branch information
Spycsh and pre-commit-ci[bot] authored Dec 18, 2024
1 parent a6cdd17 commit 70c151d
Show file tree
Hide file tree
Showing 3 changed files with 6 additions and 4 deletions.
4 changes: 2 additions & 2 deletions comps/embeddings/tei/langchain/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -42,7 +42,7 @@ curl localhost:$your_port/v1/embeddings \
Start the embedding service with the TEI_EMBEDDING_ENDPOINT.

```bash
export TEI_EMBEDDING_ENDPOINT="http://localhost:$yourport/v1/embeddings"
export TEI_EMBEDDING_ENDPOINT="http://localhost:$yourport"
export TEI_EMBEDDING_MODEL_NAME="BAAI/bge-large-en-v1.5"
python embedding_tei.py
```
Expand Down Expand Up @@ -71,7 +71,7 @@ curl localhost:$your_port/embed/v1/embeddings \
Export the `TEI_EMBEDDING_ENDPOINT` for later usage:

```bash
export TEI_EMBEDDING_ENDPOINT="http://localhost:$yourport/v1/embeddings"
export TEI_EMBEDDING_ENDPOINT="http://localhost:$yourport"
export TEI_EMBEDDING_MODEL_NAME="BAAI/bge-large-en-v1.5"
```

Expand Down
4 changes: 3 additions & 1 deletion comps/embeddings/tei/langchain/embedding_tei.py
Original file line number Diff line number Diff line change
Expand Up @@ -74,7 +74,9 @@ async def aembed_query(request: Dict, async_client: AsyncInferenceClient) -> Uni

def get_async_inference_client(access_token: str) -> AsyncInferenceClient:
headers = {"Authorization": f"Bearer {access_token}"} if access_token else {}
return AsyncInferenceClient(model=TEI_EMBEDDING_ENDPOINT, token=HUGGINGFACEHUB_API_TOKEN, headers=headers)
return AsyncInferenceClient(
model=f"{TEI_EMBEDDING_ENDPOINT}/v1/embeddings", token=HUGGINGFACEHUB_API_TOKEN, headers=headers
)


if __name__ == "__main__":
Expand Down
2 changes: 1 addition & 1 deletion tests/embeddings/test_embeddings_tei_langchain.sh
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ function start_service() {
model="BAAI/bge-base-en-v1.5"
unset http_proxy
docker run -d --name="test-comps-embedding-tei-endpoint" -p $tei_endpoint:80 -v ./data:/data --pull always ghcr.io/huggingface/text-embeddings-inference:cpu-1.5 --model-id $model
export TEI_EMBEDDING_ENDPOINT="http://${ip_address}:${tei_endpoint}/v1/embeddings"
export TEI_EMBEDDING_ENDPOINT="http://${ip_address}:${tei_endpoint}"
tei_service_port=5002
docker run -d --name="test-comps-embedding-tei-server" -e LOGFLAG=True -e http_proxy=$http_proxy -e https_proxy=$https_proxy -p ${tei_service_port}:6000 --ipc=host -e TEI_EMBEDDING_ENDPOINT=$TEI_EMBEDDING_ENDPOINT opea/embedding-tei:comps
sleep 3m
Expand Down

0 comments on commit 70c151d

Please sign in to comment.