Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Move GPU ukernel selection to KernelConfig #19440

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

bjacob
Copy link
Contributor

@bjacob bjacob commented Dec 10, 2024

This moves the logic deciding whether an op should be a ukernel out of the GPULowerToUKernels pass, into KernelConfig.

So KernelConfig decides whether the op should be a ukernel, and encodes that into the resulting lowering_config, in a new parameter, that is a new attribute, UKernelSpecAttr. That attribute is directly modeled after the equivalent C++ data structure that we have had in LowerToUKernels passes, FnNameAndDefAttrs, which it replaces. If the attribute is present, it means that the op was selected for ukernel lowering, with the fields telling the ukernel name and some function definition attributes (to import any dependencies, such as the rocm module for runtime support symbols).

All the details about supplying the ukernel bitcode in a hal.executable.object are also moved there, becoming a side effect of KernelConfig.

The GPULowerToUKernels becomes much simpler, since all the decision-making was already done for it. It just looks at the LoweringConfigAttr and if it's there, it performs the requested lowering.

The motivation for this split is that we need to know in KernelConfig whether it's going to be a ukernel, because ops that will get lowered to a ukernel require a different configuration. The important example for us is multi_mma, which in the ukernel case needs to avoid reduction-dimension tiling to 1 so that the ukernel gets to see the reduction loop.

A few simplifications arise already in the current argmax ukernel logic, confirming that this was the right design choice: the old ukernel's matching logic was checking that the distribution tile sizes matched what the ukernel could handle; now that is turned upside down: the ukernel matching happens as a helper within KernelConfig where we know we are setting the appropriate tile sizes on purpose.

Another nice improvement is that this puts just enough distance between ukernel selection (which creates the hal.executable.object) and ukernel lowering, that we are able to insert HoistExecutableObjectsPass in between, simplifying the ukernel lowering as it doesn't need to worry anymore about preserving the hal.executable.object.

@bjacob bjacob changed the title GPUSelectUKernelsPass Move GPU ukernel selection to KernelConfig Dec 12, 2024
@bjacob bjacob marked this pull request as ready for review December 12, 2024 21:55
@hanhanW hanhanW self-requested a review December 13, 2024 03:07
Signed-off-by: Benoit Jacob <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant