align VLLM micro-service output format with UI (#392)

VLLM misco-service output format align with UI as well as that of LLM micro-service TGI implementation. Signed-off-by: Wang, Xigui <[email protected]>
opea-project · Aug 6, 2024 · c1887ed · c1887ed
1 parent b61f61b
commit c1887ed
Showing 1 changed file with 2 additions and 8 deletions.
diff --git a/comps/llms/text-generation/vllm/llm.py b/comps/llms/text-generation/vllm/llm.py
@@ -49,14 +49,8 @@ def stream_generator():
             chat_response = ""
             for text in llm.stream(input.query):
                 chat_response += text
-                processed_text = post_process_text(text)
-                if text and processed_text:
-                    if "</s>" in text:
-                        res = text.split("</s>")[0]
-                        if res != "":
-                            yield res
-                        break
-                    yield processed_text
+                chunk_repr = repr(text.encode("utf-8"))
+                yield f"data: {chunk_repr}\n\n"
             print(f"[llm - chat_stream] stream response: {chat_response}")
             yield "data: [DONE]\n\n"