Proposal for 6.0: Quantized kernels

[krzysz00]

A few months ago, MIOpen asked if we could support quantization in our kernels. To my understanding, they were interested in both uniform and per-channel quantization. We haven't had much followup from them since, so we don't know how much they're interested in this on their end.

MLIR has some support for quantization upstream, both in Tosa and in the quantization dialect, that we could take advantage of, and we have good fusion infrastructure.

What this is

• Determine the requirements from MIOpen, and determine if they're smaller steps that they'd be OK with
• Extend our kernel flow with support for quantized types. We'd either have quantized-type memref inputs, or we'd have quantization attributes that we use to quantize/dequantize within the kernel. This depends on what MIOpen wants

Why?

Gets us another kernel type that we're in a good position for, and should be applicable to GEMM just fine.

People wanted it

Why not?

Adds a bit of extra complexity to the kernel flow and how we pass problems to and from clients

Do people want it?

[jungpark-mlir]

Also I haven seen any support for zero_point support.
There are different quantization formats (including option for per-tensor, per-channel quantization). We need to clarify the specification we want to support.