Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[conv] Optimize performance of GWSS #1300

Merged
merged 4 commits into from
Dec 22, 2021
Merged

[conv] Optimize performance of GWSS #1300

merged 4 commits into from
Dec 22, 2021

Conversation

atamazov
Copy link
Contributor

@atamazov atamazov commented Nov 19, 2021 •
edited
Loading

  • Optimizes GWSS for DYNAMIC_HYBRID mode
  • Optimize out IsApplicable() calls when workspace is not necessary for a Solver
    • MayNeedWorkspace() is introduced in the Solver API for that.
  • By-products
    • Refactor GetWorkspaceSize -> GetWorkspaceSizes for clarity

Average acceleration of GWSS calls

  • Immediate mode API (miopenConvolutionForwardGetSolutionWorkspaceSize etc): 15...115%
  • Find API (miopenConvolutionForwardGetWorkSpaceSize etc): 2...17%

However the overall effect on the total Aux Wall Time (that includes GWSS time) is relatively small and does not exceed 2%.

Side effects

The total GWSS time for the use case described at #1297 is progressed from ~24 to ~7 ms.

⚠️ This may camouflage the visibility of #1297 during GWSS calls. So engineers who work on #1297 may wish to revert this PR in their development trees.

@codecov

This comment has been minimized.

Sign in to view
@atamazov
Copy link
Contributor Author

Please review.

DrizztDoUrden
DrizztDoUrden reviewed Nov 25, 2021
View reviewed changes
Copy link
Contributor

@DrizztDoUrden DrizztDoUrden left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@atamazov
Copy link
Contributor Author

LGTM

Hey, I need also a green mark ;)

@DrizztDoUrden
Copy link
Contributor

My bad, misclicked.

@atamazov
Copy link
Contributor Author

Results of testing added. @junliume Does the positive effect worth the slightly increased complexity of the code, what do you think?

junliume
junliume approved these changes Dec 21, 2021
View reviewed changes
Copy link
Contributor

@junliume junliume left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As discussed, every perf optimization is good :)

@atamazov atamazov merged commit c682975 into develop Dec 22, 2021
@junliume junliume deleted the gwss-skip-zeroes branch January 7, 2022 18:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@averinevg averinevg averinevg left review comments

@junliume junliume junliume approved these changes

@DrizztDoUrden DrizztDoUrden DrizztDoUrden approved these changes

@asroy asroy Awaiting requested review from asroy

Assignees
No one assigned
Labels
performance TESTING_CI_PASSED urgency_normal value_low
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@atamazov @DrizztDoUrden @junliume @averinevg