[Question]: How can I improve the concurrency performance of the ragflow_stream_output api？

[Weishaoya]

Describe your problem

For ragflow_streaming_output api, when I set the number of concurrent requests to 1, 10, and 100, the first token latency was 0.6719s, 4.7593s, and 41.9158s, respectively. Due to the existence of the retrieval link, the concurrency performance of ragflow_stream_output api is weak, which is not conducive to large-scale applications. How can I improve the concurrency performance of ragflow_stream_output api?

[KevinHuSh]

Change the run_simple function in api/ragflow_server.py.