Streamk v0.3

[xiaohuguo2023]

streamk gemm script v0.3

• new persistent gemm kernel
• gemm benchmark tool using nearest neighbor approach.
• new grid_model from Ryan
• updated kernel interfaces and tuning scripts
• reimplement stream-k atomic kernel which has no spills for large tile size
• add bf16 support

[neoblizz]

Looks good to me, my biggest comment here is I wonder if there's a way to merge streamk_kernel and streamk_kernel_atomic with some constexpr, so it uses the same code but uses atomics for the partials instead. That seems to be the fundamental change between the two but a bunch of redundant code is being used.

[xiaohuguo2023]

Looks good to me, my biggest comment here is I wonder if there's a way to merge streamk_kernel and streamk_kernel_atomic with some constexpr, so it uses the same code but uses atomics for the partials instead. That seems to be the fundamental change between the two but a bunch of redundant code is being used.

the reason to put in atomics kernel here is for Ilia to test some accuracy issue of atomic_add, I would like to keep it separate for now just because we don't want to complicate the opt for our remainder part. We have next big task for reducing vgpr spills for the remainder.

[neoblizz]

Looks good to me, my biggest comment here is I wonder if there's a way to merge streamk_kernel and streamk_kernel_atomic with some constexpr, so it uses the same code but uses atomics for the partials instead. That seems to be the fundamental change between the two but a bunch of redundant code is being used.

the reason to put in atomics kernel here is for Ilia to test some accuracy issue of atomic_add, I would like to keep it separate for now just because we don't want to complicate the opt for our remainder part. We have next big task for reducing vgpr spills for the remainder.

Looks good, thanks!