[Global opt] add flag to generalize batch matmul ops #17877
Conversation
Signed-off-by: Ian Wood <[email protected]>
compiler/src/iree/compiler/GlobalOptimization/GeneralizeLinalgNamedOps.cpp
Outdated
Show resolved
Hide resolved
compiler/src/iree/compiler/GlobalOptimization/test/generalize_named_ops.mlir
Outdated
Show resolved
Hide resolved
| @@ -23,6 +23,10 @@ static llvm::cl::opt<bool> clEnableQuantizedMatmulReassociation( | |||
| llvm::cl::desc( | |||
| "Enables reassociation of quantized matmul ops (experimental)."), | |||
| llvm::cl::init(false)); | |||
| static llvm::cl::opt<bool> | |||
| clGeneralizeLinalgMatmulOps("enable-generalize-linalg-matmul-ops", | |||
There was a problem hiding this comment.
This needs to be codified somewhere, but there is some thoughts about adding -experimental to flag names that are not really intended to be user facing (i.e. papering over backend issues that are WIP), per this discussion: #17788 (comment). Maybe something like this, but open to suggestions.
| clGeneralizeLinalgMatmulOps("enable-generalize-linalg-matmul-ops", | |
| clGeneralizeLinalgMatmulOps("iree-global-opt-generalize-linalg-matmul-ops-experimental", |
There was a problem hiding this comment.
Thats a good idea, added that to the name
|
My concern is that we are making stable passes be experimental passes. The |
This reverts commit 1e1d51e. Signed-off-by: Ian Wood <[email protected]>
Signed-off-by: Ian Wood <[email protected]>
|
@hanhanW Good point, this is definitely more of a preprocessing pass than anything else |
|
Agree with @hanhanW for now this is a one-off preprocessing. If all the backends can handle these appropriately, then we do this always in the generalize named ops pass. |
|
Ok I will defer to you both then, but if this is the only way to enable fusion of broadcasts, I would not consider this preprocessing, but rather a bug in the backend that is being worked around with an experimental flag we intend to turn on by default later. |
It seems like a pre-mature lowering to me, and the real bug is not having a |
|
oooo if the solution is |
iree-org/iree#17877 --------- Co-authored-by: Quinn Dawkins <[email protected]>
Helps when the producer is a broadcast op. After adding the flag to sdxl scripts, I saw a decent decrease in the number of dispatches. Initially, I was trying to manually generalize+fuse broadcasts [branch here](https://github.com/IanWood1/iree/tree/broadcast_matmul), but quinn saw good results with just this. --------- Signed-off-by: Ian Wood <[email protected]> Signed-off-by: Lubo Litchev <[email protected]>
After adding the flag to sdxl scripts, I saw a decent decrease in the number of dispatches.
Initially, I was trying to manually generalize+fuse broadcasts branch here, but quinn saw good results with just this.