Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add no_proxy env for embedding-tei-server container #140

Merged
merged 2 commits into from
Jun 21, 2024

Conversation

moting9
Copy link
Contributor

@moting9 moting9 commented Jun 6, 2024

Description

Add no_proxy env for embedding-tei-server container

Issues

In a proxy r&d or deployment environment ( need proxy to access external network, no proxy to access internal network)
we need to pass no_proxy setting to embedding-tei-server, otherwise by following https://github.com/opea-project/GenAIExamples/blob/main/ChatQnA/docker/xeon/README.md

"TEI embedding service" access is OK, but inside "Embedding Microservice", it failed to access "Embedding Microservice".
some discussion in opea-project/GenAIExamples#251

TEI Embedding Service
curl ${host_ip}:6006/embed
-X POST
-d '{"inputs":"What is Deep Learning?"}'
-H 'Content-Type: application/json'
Embedding Microservice
curl http://${host_ip}:6000/v1/embeddings
-X POST
-d '{"text":"hello"}'
-H 'Content-Type: application/json'

Type of change

List the type of change like below. Please delete options that are not relevant.

  • [x ] Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds new functionality)
  • Breaking change (fix or feature that would break existing design and interface)

Dependencies

No dependency.

Tests

[Describe the tests that you ran to verify your changes.]
Follow (https://github.com/opea-project/GenAIExamples/blob/main/ChatQnA/docker/xeon/README.md) to set up ChatQnA in a proxy environment

TEI Embedding Service ok
curl ${host_ip}:6006/embed
-X POST
-d '{"inputs":"What is Deep Learning?"}'
-H 'Content-Type: application/json'
Embedding Microservice ko
curl http://${host_ip}:6000/v1/embeddings
-X POST
-d '{"text":"hello"}'
-H 'Content-Type: application/json'

@moting9
Copy link
Contributor Author

moting9 commented Jun 6, 2024

After discussion with @huiyan2021 rerank container has the same issue, add no_proxy for both embedding and rerank

@lvliang-intel
Copy link
Collaborator

@moting9

please also help to add no_proxy env for other microservices? thanks.

@moting9
Copy link
Contributor Author

moting9 commented Jun 12, 2024

@moting9

please also help to add no_proxy env for other microservices? thanks.

Hello, @lvliang-intel, changes in GenAIComps docker_compose.yaml has no impact on GenAIExamples execution because each GenAIExample has its own docker_compose.yaml, no_proxy here is NOT necessary change for GenAIExample execution. To keep alignment, I will try to add no_proxy along with http/https proxy setting in GenAIComps.

@moting9 moting9 force-pushed the mydev branch 3 times, most recently from ae7c2b7 to 934efd2 Compare June 17, 2024 02:13
Signed-off-by: Zhou Ting <[email protected]>
Singed-off-by: Cao Huiyan <[email protected]>
@moting9
Copy link
Contributor Author

moting9 commented Jun 17, 2024

@moting9

please also help to add no_proxy env for other microservices? thanks.

@lvliang-intel Done. Please review. Thanks!

@lvliang-intel lvliang-intel merged commit df0c119 into opea-project:main Jun 21, 2024
19 checks passed
sharanshirodkar7 pushed a commit to sharanshirodkar7/GenAIComps that referenced this pull request Jul 9, 2024
Singed-off-by: Cao Huiyan <[email protected]>

Signed-off-by: Zhou Ting <[email protected]>
Co-authored-by: lvliang-intel <[email protected]>
Signed-off-by: sharanshirodkar7 <[email protected]>
yogeshmpandey pushed a commit to yogeshmpandey/GenAIComps that referenced this pull request Jul 10, 2024
Singed-off-by: Cao Huiyan <[email protected]>

Signed-off-by: Zhou Ting <[email protected]>
Co-authored-by: lvliang-intel <[email protected]>
Signed-off-by: Yogesh Pandey <[email protected]>
dwhitena pushed a commit to predictionguard/GenAIComps that referenced this pull request Jul 24, 2024
Singed-off-by: Cao Huiyan <[email protected]>

Signed-off-by: Zhou Ting <[email protected]>
Co-authored-by: lvliang-intel <[email protected]>
Signed-off-by: Daniel Whitenack <[email protected]>
lkk12014402 pushed a commit that referenced this pull request Aug 8, 2024
Signed-off-by: chensuyue <[email protected]>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants