[Bug] vllm microservice not working not xeon

[srinarayan-srikanthan]

Priority

Undecided

OS type

Ubuntu

Hardware type

Xeon-SPR

Installation method

• Pull docker images from hub.docker.com
• Build docker images from source

Deploy method

• Docker compose
• Docker
• Kubernetes
• Helm

Running nodes

Single Node

What's the version?

1.0 version

Description

The vllm LLM backend Service is running but the
vllm LLM microservice is failing doing the sanity check. Receiving internal server error. And docker logs show connection timeout

Reproduce steps

followed steps here : https://github.com/opea-project/GenAIExamples/tree/main/ChatQnA/docker_compose/intel/cpu/xeon#1-build-embedding-image

Raw log

No response

[srinarayan-srikanthan]

created a PR with fix : #882