-
Notifications
You must be signed in to change notification settings - Fork 27.4k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Exclude torch.compile time from metrics computation #31443
Exclude torch.compile time from metrics computation #31443
Conversation
b30517e
to
fb37d01
Compare
The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for the PR @zxd1997066 ! I think it would be better if we can do similar to what we did with f"{metric_key_prefix}_jit_compilation_time".
if f"{metric_key_prefix}_jit_compilation_time" in output.metrics:
start_time += output.metrics[f"{metric_key_prefix}_jit_compilation_time"]
Hi @SunMarc , thanks for the review, I will take a look at it. |
0e09f5c
to
5f105f4
Compare
5f105f4
to
28151ba
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is a lot better @zxd1997066 , thanks for iterating ! Could you fix the quality issue by typing make style
?
Fixed the quality issue, thanks! |
Hi @SunMarc , could you please help merge this PR if all good? I do not have access. Thanks! |
Hey @zxd1997066, I've pinged @amyeroberts for a final review ! |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for your PR!
When using torch.compile in accelerator.prepare_model, compilation time is being included while computing statistics. And this may result in incorrect performance metrics calculations. This PR measures time after the torch.compile called in accelerate to exclude the compilation time.