[QST] Can cutlass split warp tiles in the k dimension of gemmuniversal?

[Xseventh]

I am trying to tune cutlass gemm_with_broadcast kernel. I set block tile [128, 128, 64] and warp tile [64, 64, 32] and set epilogue beta is 1 for read source tensor, I encountered an error of illegal memory access. Is this because it is incorrect and causes the expected error? Will there be similar problems with setting tile in gemm without broadcast?

[Xseventh]

Not only this kind of tile division will cause illegal memory access errors. I observed that all the tiles that reported errors are warm tile k is not equal with block tile k. Can you explain the behavior of cutlass when block tile k is different from warp tile k?

[hwu36]

if warp tile k is smaller than block tile k, they will use slice k. we have unit tests for them and some sizes are enabled in the profiler. however, we don't observe much benefits on ampere when using slice k.

[Xseventh]

thanks! It seems that this error is unexpected. I will continue to check the cause of the illegal memory error.

[Xseventh]

if warp tile k is smaller than block tile k, they will use slice k. we have unit tests for them and some sizes are enabled in the profiler. however, we don't observe much benefits on ampere when using slice k.

I understand that the split-k within the same block can be reduced in shared memory. I also see another type of split-k that reduces in global memory across different blocks. Did I understand these two types of split-k methods correctly? Would the former split-k method have better performance than the latter?

[hwu36]

the latter is useful when m/n is small and k is very big.

the former is only useful in some very corner cases.

[github-actions]

This issue has been labeled inactive-30d due to no recent activity in the past 30 days. Please close this issue if no further response or action is needed. Otherwise, please respond with a comment indicating any updates or changes to the original issue and/or confirm this issue still needs to be addressed. This issue will be labeled inactive-90d if there is no activity in the next 60 days.

[mnicely]

Closing due to inactivity. Feel free to reopen if needed.