From f9d17e95c212c12785c417ae2772dbcc1ee98020 Mon Sep 17 00:00:00 2001 From: keshavkowshik Date: Fri, 16 Aug 2024 09:17:43 -0700 Subject: [PATCH] Update README.md --- benchmarks/inference/deepspeedometer/README.md | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/benchmarks/inference/deepspeedometer/README.md b/benchmarks/inference/deepspeedometer/README.md index 7c165c57d..b327916c5 100644 --- a/benchmarks/inference/deepspeedometer/README.md +++ b/benchmarks/inference/deepspeedometer/README.md @@ -1,5 +1,8 @@ # DeepSpeedometer +NOTE: This is an experimental tool and is not currently being supported since it's not fully functional. Please use the MII benchmark which can be found here: +https://github.com/microsoft/DeepSpeedExamples/tree/master/benchmarks/inference/mii + This benchmark is designed to measure performance of LLM serving solutions. Using a different number of parallel clients sending requests to an inference server, we gather data to plot throughput-latency curves and find the saturation point of an inference server that demonstrates the maximum performance. ## Installation @@ -82,4 +85,4 @@ The DeepSpeedometer benchmark was designed to allow easily adding support for ne 3. Define a `*ClientConfig` class that inherits from the `BaseConfigModel` class. Place any configuration options (i.e., user-passed command line arguments) necessary for your defined `*Client` class in here. 4. Import the newly added `*Client` and `*ClientConfig` into `clients/__init__.py` and add them to the `client_config_classes` and `client_classes` dictionaries. -For the simplest example of adding a new client, take a look at the `clients/dummy_client.py` file where we have defined a client that does not stand up a server and only returns a sample of the input prompt after a short sleep cycle. We use this as a light-weight class for unit testing. \ No newline at end of file +For the simplest example of adding a new client, take a look at the `clients/dummy_client.py` file where we have defined a client that does not stand up a server and only returns a sample of the input prompt after a short sleep cycle. We use this as a light-weight class for unit testing.