Skip to content

Commit

Permalink
fix mismatched response format w/wo streaming guardrails (#568)
Browse files Browse the repository at this point in the history
* fix mismatched response format w/wo streaming  guardrails

* fix & debug

* fix & rm debug

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
  • Loading branch information
Spycsh and pre-commit-ci[bot] authored Aug 28, 2024
1 parent 25174c0 commit b6c0785
Showing 1 changed file with 12 additions and 0 deletions.
12 changes: 12 additions & 0 deletions comps/cores/mega/orchestrator.py
Original file line number Diff line number Diff line change
Expand Up @@ -72,6 +72,18 @@ async def schedule(self, initial_inputs: Dict, llm_parameters: LLMParams = LLMPa
downstreams.remove(downstream)
except re.error as e:
print("Pattern invalid! Operation cancelled.")
if len(downstreams) == 0 and llm_parameters.streaming:
# turn the response to a StreamingResponse
# to make the response uniform to UI
def fake_stream(text):
yield "data: b'" + text + "'\n\n"
yield "data: [DONE]\n\n"

self.dump_outputs(
node,
StreamingResponse(fake_stream(response["text"]), media_type="text/event-stream"),
result_dict,
)

for d_node in downstreams:
if all(i in result_dict for i in runtime_graph.predecessors(d_node)):
Expand Down

0 comments on commit b6c0785

Please sign in to comment.