ROCM Support

[micmelesse]

This is just an issue to track the progress of rocm support.

• Create first pr with empty kernel (uses hipify)

• see add rocm enabling changes #231

• upgrade rocm to work as its own module (without hipify)

• see [GENERAL] Removed deprecated driver files and added basic compatibility with rocm #268

• add support for load and store operators

• see enable unary and most binary operators #314
• see [ROCM] enable matmul(dot) and others #391

• add support for an element wise kernel

• see enable unary and most binary operators #314

• add support for addition and subtraction kernel

• see enable unary and most binary operators #314

• add support for multiplication kernel

• see enable unary and most binary operators #314

• add support for matmul kernel

• see [ROCM] enable matmul(dot) and others #391

• add support for softmax kernel
• add support for conv kernel
• add support for remaining kernels

• exp, log, sin, cos

• see [ROCM] enable matmul(dot) and others #391

• add amd intrinsic support to improve performance

[ptillet]

Great idea!

I'm optimistic that just getting test_language.py to pass will be enough for the 4 last tasks to "just work". That said, we should also extend test_language.py as needed :) Note that there is a new test_dot unit test that will make it easier to get matmul working https://github.com/openai/triton/blob/master/python/test/test_language.py#L437

[weixingzhang]

@micmelesse Great to see ROCm support. What is the plan to support matrix core in MI100? Or is it already supported?

[micmelesse]

See #1073