Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update throughput-latency plot script #881

Merged
merged 10 commits into from
Apr 9, 2024
Merged

Conversation

lekurile
Copy link
Contributor

@lekurile lekurile commented Mar 28, 2024

This PR updates the plot_th_lat.py throughput-latency plot generation script to remove the concept of a backend (aml, fastgen, vllm) and generalize for any result output directory, irrespective of where it was run.

The PR also introduces the concept of an optional plot_config.yaml that resides within each result directory and allows for overrides in the plot formatting. An example config file may look like this:

label: "vLLM"
color: "purple"
marker: "o"
linestyle: "--"
polyfit_degree: 0
x_max : 30
y_max : 10

Each of the config parameters is optional, allowing for override of only the specific plot aspects required, however all parameters may also be provided.

A few nuances for the polyfit_degree and x/y_max parameters:

  • polyfit_degree: Specifies the polynomial degree for the 'best fit line'. Specifying 0 removes the best fit line and simply connects the scatter plot points.
  • x/y_max: Clips the x or y axis data using the specified value as the upper bound.

An example command executing the script may look something like this:

python3 src/plot_th_lat.py --data_dirs ./results/results-* --model_name <plot_model_title>

Or each result directory can be enumerated explicitly:

python3 src/plot_th_lat.py --data_dirs ./results/results-1 ./results/results-2 ./results/results-3 --model_name <plot_model_title>

Copy link
Contributor

@mrwyattii mrwyattii left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please address my comments before merging, but LGTM!

benchmarks/inference/mii/README.md Outdated Show resolved Hide resolved
benchmarks/inference/mii/plot_config.yaml Outdated Show resolved Hide resolved
@lekurile lekurile merged commit fab5d06 into master Apr 9, 2024
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants