Added the microservice of vLLM

[tianyil1]

Description

This PR mainly updates the microservice wrapper for the vllm to align with the tgi, rename the ray backend service, and adds the vllm and ray introduction in the readme.

Issues

n/a.

Type of change

List the type of change like below. Please delete options that are not relevant.

• New feature (non-breaking change which adds new functionality)
  • wrapped the vllm with the langchain microservice for aligning with tgi microservice.
  • updated the llms readme with the vllm and ray introduction.
  • fixed the vllm readme and docker script with huggingface API token useage.
• Breaking change (fix or feature that would break existing design and interface
  • rename the ray serve from "rayllm" to "ray_serve" to avoid the confusion of the ray implement based on the "ray serve" instead of "ray llm"

Dependencies

No newly introduced 3rd party dependency.

Tests

This PR is tested in the Gaudi2 server with:
2 sockers Intel(R) Xeon(R) Platinum 8368 CPU @ 2.40GHz
8 Gaudi nodes, HL-SMI Version: hl-1.14.0-fw-48.0.1.0 Driver Version: 1.14.0-9e8ecf8

which is tested well in the above env:

• vLLM backend serving
  

• Langchain vLLM serving
  

[tianyil1]

Please help review, and I expect your invaluable feedback. Thanks. @Jian-Zhang @xuechendi

[tianyil1]

The PR has fixed the above comments and is ready to merge. Please help review. Thanks. @lvliang-intel @hshen14 @Jian-Zhang

[ftian1]

from the PR title, only LLM microservice based on vLLM was added. but in the code, you in fact included Ray Serve version, right? and it's better to move 'text-generation/ray_serve/docker' to upper level like what we did in vLLM and other examples.

another question is why there is a requirements.txt in 'text-generation/ray_serve/docker' folder? moving it to dockerfile is feasible?

[tianyil1]

from the PR title, only LLM microservice based on vLLM was added. but in the code, you in fact included Ray Serve version, right? and it's better to move 'text-generation/ray_serve/docker' to upper level like what we did in vLLM and other examples.

another question is why there is a requirements.txt in 'text-generation/ray_serve/docker' folder? moving it to dockerfile is feasible?

@ftian1 Thanks for your comments. I exactly renamed the ray serve name instead of changing its version, did not implement the ray serve microservice in this PR, and will implement it in the next PR. The code in "docker" folder of the ray_serve was self-implemented docker image and just aimed to launch the ray serve llm engine unlike that the vLLM/TGI can build the docker from official github.

The requirements.txt of ray serve was the necessary file to build the ray serve engine image instead of the microservice of the ray langchain implementation, so I put it under the docker folder as before.

[tianyil1]

This PR is ready to merge. Would you please help check this PR? Thanks. @ftian1 @lvliang-intel

[ftian1]

looks good to me