Fix JSON Return Format in getReqData Function #145
Merged
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
The getReqData function was incorrectly returning a tuple containing a dictionary, which is not a valid JSON format.
Description
Fix JSON Return Format in getReqData Function
Issues
n/a
Type of change
List the type of change like below. Please delete options that are not relevant.
Dependencies
NA
Tests
Retriever benchmark now works with no failures:
===Starting test: benchmark
Running test: locust --locustfile /home/ronglei/rag/GenAIEval/evals/benchmark/stresscli/locust/aistress.py --host http://10.112.120.165:7000 --run-time 60m --load-shape constant --processes 2 --users 1 --spawn-rate 1 --max-request 4 --bench-target retrieverfixed --llm-model Intel/neural-chat-7b-v3-3 --stop-timeout 120 --csv /home/ronglei/benchmark_output/retrieverfixed_20240929_025009/1 --headless --only-summary --loglevel WARNING --json
Concurrency : 1
Max request count : 4
Http timeout : 120000
Benchmark target : retrieverfixed
Load shape : constant
=================Total statistics=====================
Succeed Response: 4 (Total 4, 100.0% Success), Duration: 2.94s, RPS: 1.36
End to End latency(ms), P50: 4.07, P90: 7.59, P99: 8.90, Avg: 5.20
[]
[]
[
{
"name": "/v1/retrieval",
"method": "POST",
"last_request_timestamp": 1727592612.8506937,
"start_time": 1727592612.8384776,
"num_requests": 4,
"num_none_requests": 0,
"num_failures": 0,
"total_response_time": 20.651484606787562,
"max_response_time": 8.988250978291035,
"min_response_time": 3.587363986298442,
"total_content_length": 448,
"response_times": {
"9": 1,
"4": 3
},
"num_reqs_per_sec": {
"1727592612": 4
},
"num_fail_per_sec": {}
}
]